WPF DataGrid makes shadow copy of whole ItemsSource

Question

I recently started working on a WPF application that has a DataGrid with lots of data. The ItemsSource is bound like this:

ItemsSource="{Binding MyData, Mode=OneWay}"

MyData in this case is a custom collection that implements these Interfaces:

public sealed class GridCollection 
    : IList<object[]>, INotifyCollectionChanged, INotifyPropertyChanged

The underlying data is never changed, but only completely reloaded on certain triggers. The Collection holds quite large amounts of data. As the columns can change after regenerating the data source they are rebuild after each ItemsChanged (in a inherited class) like this (most likely not relevant for the problem, but who knows):

using (Dispatcher.DisableProcessing())
{
    Columns.Clear();
    for (var i = 0; i < columns.Count; i++)
    {
        var bindingBase = new Binding("[" + i + "]") {Mode = BindingMode.OneTime};

        var column = new DataGridTextColumn
        {
            Header = columns[i].Name,
            Binding = bindingBase,
            IsReadOnly = true
        };
        Columns.Add(column);
    }
}

Now the problem with this is that the DataGrid seems to hold a complete copy of the data in the collection in its own ObservableCollection, as I can see clearly in a memory profiler (going directly to the large object heap). I can only assume that this behavior is to avoid multithreading issues or that it has something to do with the GridView? It might have to do with that the DataGrid does not recognize the ItemsSource as List and only as IEnumerable? I could also confirm that the data from the collection is never accessed after WPF has made the copy of the ItemsSource.

What I want to achieve is that the DataGrid operates directly on my collection to be more memory and time efficient (as creating this copy certainly takes time).

I already played around with stuff like CollectionSynchronization, as I thought that maybe if I tell it that I take care of synchronization then it does not make a copy,

BindingOperations.EnableCollectionSynchronization(gridCollection, _collectionLock);

but it does not change anything on the behavior. I also made sure that row virtualization is working on the grid.

Any idea why the grid keeps this copy and how to avoid this? I mean sure it probably has to keep some data (at least for the visible rows), but for the whole collection seems overkill...

Update The rows were not duplicated, but the CollectionView created an ObservableCollection with a large backing array referencing all entries in my collection. Implementing ICollectionView fixed that for me. Now I get the weirdest behavior:

public IEnumerator<object[]> GetEnumerator()
{
    lock (_lock)
    {
        if (_backingField == null) // Breakpoint hits when first refreshing view, but not second time!
            yield break;

        // Breakpoint hits second time, _backingField is null --> Exception
        for (var i = 0; i < _backingField.Count; i++) yield return _backingField[i];
    }
}

So somehow the second time I refresh the view the null check is not executed. I put all accesses to _backingField in a lock, the _backingField is volatile and compile optimizations are disabled. Sounds like code reordering to me, but I even put a memory barrier between the null check and the for, it just does not help. Is this something WPF-specific with ICollectionView? I double checked that my breakpoints are always on the main thread. Any ideas?

Update 2 Now it makes total sense that the second check is not hit in the debugger as it is an IEnumerable. Somehow implementing ICollectionView causes the DataGrid (or someone else) to start enumerating and after throwing the OnCollectionChanged reset-event to continue with this. Still no idea why, as the backing field has changed before the resets event, therefore the enumerator is invalid at this point. If you know best practices how to handle this let me know :)