'How to assert that all rows have been retrieved from Oracle when using pagination?
I need to create an API which is responsible for extracting 100K records from Oracle and insert the same to Mongo after some processing.
I plan to use Pagination & Row Limiting (Based on following approach by Tom Kyte : https://blogs.oracle.com/connect/post/on-rownum-and-limiting-results) at oracle end to obtain a chunk of data from oracle. At Java end , create multiple runnables where each thread is responsible for extracting data from Oracle, process and insert it to Mongo after processing.
Although Tom Kyte in his article suggests to not obtain the total count of documents at any point (as it is a heavy operation), should the total count be obtained at initial point so that number of threads needed can be decided or is there any other way through which I can assert that all my documents have been retrieved.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|