How do you deal with Listview.separated() separators getting rebuilt and changing content of the separators?

Question

I have a ListView which is a bunch of news articles. The news articles have a date associated with them, for example March 3, March 4, etc..

Now if the articles have the same date I want them to be grouped together, so it would go something like this:

March 3

News Article 1
News Article 2

March 4

News Article 3

March 5

News Article 4
News Article 5
News Article 6

What I've tried to do is to create a variable that tracks the last seen date:

final _listofuseddates = [];

The date itself is a String located here: data["date"]

If the date presented by the article data is different, then it will insert a new separator text with the date:

         ListView.separated(
                        itemCount: data.length 
                        itemBuilder: (BuildContext context, int index) {

                          return NewsItem(
                            index: index,
                            data: data[index]
                          );

                        },
                      separatorBuilder: (BuildContext context, int index) {

                        final _thisdate = data["date"];

                        if (_listofuseddates.contains(_thedate)) {
                            return Container();
                        }
                        else
                          {
                          _listofuseddates.add(_thedate);
                          return Text(_thedate.toString());
                          }

                      },
                    ),

Whats happening here is the date of the current article is being compared to the List of dates we've already seen, if its a novel date, then a Separator text is added with that date, for example March 3. If the date was already seen, then no separator, or at least an empty imperceptible separator is put in Container();.

Now this is OK when the list first loads, but as soon as you scroll the list, this all changes and all the dates erase themselves on the fly, because the listview is constantly rebuilding itself and of course the Date was already seen. Worse, yet if you scroll down and then backup the list then things will sometimes reverse themselves, as the first unseen date is the last item as you are scrolling up.

Is there a better approach to this seemingly simple concept of adding date headers to a bunch of list items to keep them "grouped"?