Graph Neural Network for Image Classification

Question

I have a task about image classification using Graph Neural Network. Can you give me some references for it? I just found on the internet GCN is used for CSV data classification. thanks :)

Answer 1

Here is a survey of image based Graph Neural Networks for classification - https://arxiv.org/abs/2106.06307

and a tutorial - https://medium.com/@BorisAKnyazev/tutorial-on-graph-neural-networks-for-computer-vision-and-beyond-part-1-3d9fada3b80d

Answer 2

I currently work on similar topic and here is my observation of what works

Step 1: Use SLIC segmentation to get the superpixels of the image
Step 2: Region adjacency graph can be build form the superpixel labels (output is networkx graph)
Step 3: Encode any special feature to discriminate your graph (just like the images) eg. px intensities of rgb channels can be embedded as node features using a vector etc

superpixels_labels = segmentation.slic(fmri, compactness=30, n_segments=72, multichannel=False) + 1 

def build_rag(labels, image):
    g = nx.Graph()
    footprint = ndi.generate_binary_structure(labels.ndim, connectivity=1)
    _ = ndi.generic_filter(labels, add_edge_filter, footprint=footprint,
                           mode='nearest', extra_arguments=(g,))
    for n in g:
        g.nodes[n]['total color'] = np.zeros(34, np.double)
        g.nodes[n]['pixel count'] = 0
    for index in np.ndindex(labels.shape):
        n = labels[index]
        g.nodes[n]['total color'] += image[index]
        g.nodes[n]['pixel count'] += 1
    return g


#to add node features from image
nx.set_node_attributes(g, [0.2, 0.7, 0.5], "ndata")
g.nodes[1]["ndata"]

Useful Reference

1. https://arxiv.org/abs/2201.12633
2. https://graph-neural-networks.github.io/static/file/chapter20.pdf
3. https://distill.pub/2021/gnn-intro/