'How to retrieve large datasets from Firestore in chunks?
My app has around 50K objects. And I need to retrieve atleast 10000 objects to build charts for stats; I am not able to get more than 600 objects in a single call. So I tried doing it in chunks using pagination. The code is below:
late DocumentSnapshot _lastDocument;
List<Booking> entries = List.empty(growable: true);
Constructor(){
getEntries();
}
void getEntries() async {
var q = await FirebaseFirestore.instance
.collection('bookings')
.orderBy('entryTimestamp')
.where('entryTimestamp',isGreaterThanOrEqualTo: getWeekFilter()[0]/1000, isLessThanOrEqualTo: getWeekFilter()[1]/1000)
.limitToLast(50).get();
entries.addAll(q.docs.map((e) => Booking.fromSnapshot(e)));
notifyListeners();
print('${entries.length}');
if(entries.length < 2000){
_lastDocument = q.docs[q.docs.length - 1];
getMoreEntries();
}
}
void getMoreEntries() async {
print('WEEK_DEBUG | Getting more entries after $_lastDocument.');
var q = await FirebaseFirestore.instance
.collection('bookings')
.orderBy('entryTimestamp')
.where('entryTimestamp',isGreaterThanOrEqualTo: getWeekFilter()[0]/1000, isLessThanOrEqualTo: getWeekFilter()[1]/1000)
.startAfterDocument(_lastDocument)
.limitToLast(50).get();
entries.addAll(q.docs.map((e) => Booking.fromSnapshot(e)));
notifyListeners();
_lastDocument = q.docs[q.docs.length - 1];
if(entries.length < 2000){
getMoreEntries();
}
}
List getWeekFilter(){
var now = DateTime.now();
var firstDay = now.subtract(const Duration(days: 7));
var lastDay = now.add(const Duration(days: 7));
var from = DateTime(firstDay.year, firstDay.month, firstDay.day).millisecondsSinceEpoch;
var to = DateTime(lastDay.year, lastDay.month, lastDay.day).millisecondsSinceEpoch;
return [from, to];
}
When I run the code, getMoreEntries() gets called once while the total length of entries remains 50.
I get the first 50 objects but nothing more. What am I doing wrong? What's a better way to retrieve large datasets for charts?
Any help is greatly appreciated.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
