mongoose cursor batchSize

Question

how to iterate the cursor batch documents in case that the batchSize is defined ? example, when the batchSize defined to be equal to 50 , is there any way to iterate that 50 sub documents ?

var myCursor = collection.find().cursor({batchSize:50});
mycursor('on',function(doc){

})

Answer 1

Try this:

var myCursor = collection.find({}).cursor({batchSize:50});
myCursor.eachAsync((doc) => {
   ...
});

Batch sizes are just for performance optimisation and will not give you a 50 object chunk.

You will still have to process each doc individually.

Answer 2

Giving an Update to May 2022 Solution for whom want to add Populate within Cursor BatchSize.

• The usecase here is that for every Streamed Document, corresponding to 1 Query to MongoDB. For example collection has 1000 records. Say you use Stream APIs or equivalent

(async () => {
  /** @description - Good Performance */
  const stream = MongooseModel.find({}).cursor();
  for await (const doc of stream) {
    console.log(doc);
  }
  /** @description - Worse Performance */
  const stream = MongooseModel.find({}).populate({path: 'fieldX'}).batchSize(1).cursor();
  for await (const doc of stream) {
    console.log(doc);
  }
})();

• This will create a huge dropdown on Performance approximately 50 times from my comparison (25000 records / sec Non-Populate) vs (500 records / sec Populate 1 field)
• The reason is that for each streamed / 1-cursor document created. Mongoose send 1 query to MongoDB via its Driver.
• So we use batchSize along with cursor, for example {batchSize: 100}. So for the collection that has 1000 records. Mongoose populate and left-join 1000 / 100 == 10 times instead of 1000 times

(async () => {
  /** @description - Better Performance - Theoretically without BUGs */
  const stream = MongooseModel.find({}).populate({path: 'fieldX'}).batchSize(100).cursor();
  for await (const doc of stream) {
    console.log(doc);
  }
})();

• However. Mongoose has BUGS !!! Older Mongoose version: Before v6.2 even though you specify batchSize it still not reduce the Query !!!
• @see {@link https://github.com/Automattic/mongoose/issues/9365 }
• @see {@link https://github.com/Automattic/mongoose/issues/11509 }
• @refer {@link https://thecodebarbarian.com/cursors-in-mongoose-45 }
• @alternative {@link https://github.com/Automattic/mongoose/issues/3683 }