'Is it possible to optimize the pymongo cursor to close the connection and continue where it's finished?
We use MongoDB:4.12.14 in our project its collection, Buzz contains more than ~3m records. The object is too complex to write here, but We run a query for X:<str> and Y:<datetime> fields which look like this:
{X: <str>, Y: {$gte: <datetime>, $lte: <datetime>}}
There is index for that query > {X: 1, Y: 1}
Our problem is We use MongoDB via pymongo:3.12 and over TLS, but the query runs more than 1 minute despite there is index on the collection. After pymongo returns a cursor, We call list to exhaust the cursor and retrieve all records, but after it reaches the TLS timeout threshold it comes back with that > socket.timeout: The read operation timed out and it comes from
pymongo/pool.py:751pymongo/network.py:200pymongo/network.py:291python3.7/ssl.py:1071python3.7/ssl.py:929
My question is what should we do with that now? What could be the next step? Is it possible somehow to close the connection and continue where it finished? Would it be the proper way at all?
Thanks for your attention in advance.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
