wxDataViewListCtrl is slow with 100k items from another thread

Question

The requirements:

1. 100k lines
2. One of the columns is not text - its custom painted with wxDC*.
3. The items addition is coming from another thread using wxThreadEvent.

• Up until now I used wxDataViewListCtrl, but it takes too long to AppendItem 100 thousand time.
• wxListCtrl (in virtual mode) does not have the ability to use wxDC* - please correct me if I am wrong.

The only thing I can think of is using wxDataViewCtrl + wxDataViewModel. But I can't understand how to add items.

I looked at the samples (https://github.com/wxWidgets/wxWidgets/tree/WX_3_0_BRANCH/samples/dataview), too complex for me. I cant understand them.

I looked at the wiki (https://wiki.wxwidgets.org/WxDataViewCtrl), also too complex for me.

Can somebody please provide a very simple example of a wxDataViewCtrl + wxDataViewModel with one string column and one wxDC* column.

Thanks in advance.

P.S.

Per @HajoKirchhoff's request in the comments, I am posting some code:

// This is called from Rust 100k times.
extern "C" void Add_line_to_data_view_list_control(unsigned int index,
                                                   const char* date,
                                                   const char* sha1) {
    wxThreadEvent evt(wxEVT_THREAD, 44);
    evt.SetPayload(ViewListLine{index, std::string(date), std::string(sha1)});
    wxQueueEvent(g_this, evt.Clone());
}

void TreeWidget::Add_line_to_data_view_list_control(wxThreadEvent& event) {
    ViewListLine view_list_line = event.GetPayload<ViewListLine>();

    wxVector<wxVariant> item;

    item.push_back(wxVariant(static_cast<int>(view_list_line.index)));
    item.push_back(wxVariant(view_list_line.date));
    item.push_back(wxVariant(view_list_line.sha1));

    AppendItem(item);
}

Answer 1

Appending 100k items to a control will always be slow. That's because it requires moving 100k items from your storage to the controls storage. A much better way for this amount of data is to have a "virtual" list control or wxGrid. In both cases the data is not actually transferred to the control. Instead when painting occurs, a callback function will transfer only the data required to paint. So for a 100k list you will only have "activity" for the 20-30 lines that are visible.

With wxListCtrl see https://docs.wxwidgets.org/3.0/classwx_list_ctrl.html, specify the wxLC_VIRTUAL flag, call SetItemCount and then provide/override

• OnGetItemText
• OnGetItemImage
• OnGetItemColumnImage

Downside: You can only draw items contained in a wxImageList, since the OnGetItemImage return indizes into the list. So you cannot draw arbitrary items using a wxDC. Since the human eye will be overwhelmed with 100k different images anyway, this is usually acceptable. You may have to provide 20/30 different images beforehand, but you'll have a fast, flexible list.

That said, it is possible to override the OnPaint function and use that wxDC to draw anything in the list. But that'll get difficult pretty soon.

So an alternative would be to use wxGrid, create a wxGridTableBase derived class that acts as a bridge between the grid and your actual 100k data and create wxGridCellRenderer derived classes to render the actual data onscreen. The wxGridCellRenderer class will get a wxDC. This will give you more flexibility but is also much more complex than using a virtual wxListCtrl.

Answer 2

The full example of doing what you want will inevitably be relatively complex. But if you decompose in simple parts, it's really not that difficult: you do need to define a custom model, but if your list is flat, this basically just means returning the value of the item at the N-th position, as you can trivially implement all model methods related to the tree structure. An example of such a model, although with multiple columns can be found in the sample, so you just need to simplify it to a one (or two) column version.

Next, you are going to need a custom renderer too, but this is not difficult neither and, again, there is an example of this in the sample too.

If you have any concrete questions, you should ask them, but it's going to be difficult to do much better than what the sample shows and it does already show exactly what you want to do.