'How to retrieve large datasets from Firestore in chunks?

My app has around 50K objects. And I need to retrieve atleast 10000 objects to build charts for stats; I am not able to get more than 600 objects in a single call. So I tried doing it in chunks using pagination. The code is below:

late DocumentSnapshot _lastDocument;
List<Booking> entries = List.empty(growable: true);

Constructor(){
    getEntries();
}

void getEntries() async {
    var q = await FirebaseFirestore.instance
        .collection('bookings')
        .orderBy('entryTimestamp')
        .where('entryTimestamp',isGreaterThanOrEqualTo: getWeekFilter()[0]/1000, isLessThanOrEqualTo: getWeekFilter()[1]/1000)
        .limitToLast(50).get();
    entries.addAll(q.docs.map((e) => Booking.fromSnapshot(e)));
    notifyListeners();
    print('${entries.length}');
    if(entries.length < 2000){
      _lastDocument = q.docs[q.docs.length - 1];
      getMoreEntries();
    }
  }

void getMoreEntries() async {
    print('WEEK_DEBUG | Getting more entries after $_lastDocument.');
    var q = await FirebaseFirestore.instance
        .collection('bookings')
        .orderBy('entryTimestamp')
        .where('entryTimestamp',isGreaterThanOrEqualTo: getWeekFilter()[0]/1000, isLessThanOrEqualTo: getWeekFilter()[1]/1000)
        .startAfterDocument(_lastDocument)
        .limitToLast(50).get();

    entries.addAll(q.docs.map((e) => Booking.fromSnapshot(e)));
    notifyListeners();
    _lastDocument = q.docs[q.docs.length - 1];
    if(entries.length < 2000){
      getMoreEntries();
    }
  }

 List getWeekFilter(){
    var now = DateTime.now();
    var firstDay = now.subtract(const Duration(days: 7));
    var lastDay = now.add(const Duration(days: 7));
    var from = DateTime(firstDay.year, firstDay.month, firstDay.day).millisecondsSinceEpoch;
    var to = DateTime(lastDay.year, lastDay.month, lastDay.day).millisecondsSinceEpoch;
    return [from, to];

  }

When I run the code, getMoreEntries() gets called once while the total length of entries remains 50. I get the first 50 objects but nothing more. What am I doing wrong? What's a better way to retrieve large datasets for charts? Any help is greatly appreciated.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source