when using dcc.Store() for large data, which storage type is good, memory, local or session?
I got an error of
IOPub data rate exceeded.
The notebook server will temporarily stop sending output
to the client in order to avoid crashing it.
To change this limit, set the config variable
`--NotebookApp.iopub_data_rate_limit`.
Current values:
NotebookApp.iopub_data_rate_limit=1000000.0 (bytes/sec)
NotebookApp.rate_limit_window=3.0 (secs)The webpage also crashes showing below figure:
2 Answers
There are limitations to using dcc.Store. It stores the data in the browser, which is not ideal or even intended for large/very large datasets, as performance issues will arise. Here is a list of those limitations for the component.
If you intend to use visualise large datasets using Dash, I recommend a great tool that combines Django with Dash called django-plotly-dash. Essentially, you'll be handling the backend all in django using its model-view approach, and then building your dashboard in dash that will be integrated with the django backend. So rather than using dcc.Store to store any large datasets, you can query your database in the callbacks, hence visualising large datasets more efficiently.
this component is not for storing large datasets... It's a good component to store some data that will be used through the application;
Also, Dash is not a good framework for big data projects like the one you did mention.
If you need some extra help feel free to message me pvt.
4
