'Django/Pandas/Website - Querying a pandas dataframe instead of database, is it possible?
My table has some advanced features and queries that Pandas can accomplish but a database can't, so I am wondering about the implications of using a pandas dataframe on the server as a read-only database.
The setup would be the basetable being a big pandas dataframe (400mb) on the webserver, and users submit GET requests that filter it in unique ways in order to make charts off of the data.
The question is are there serious costs and/or bottlenecks when many users query a webservers dataframe? I know databases are meant to serve many people concurrently, but is this a problem for my case?
Further, after a user filters the table, I imagine this filtered table is temporarily saved during their session....could this lead to memory requirements of the webserver gettings wildly out of hand? (Ex: 1000 users x 200MB avg table size = 200GB of memory demanded)
Any explanation and tips would be appreciated!
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|