Cross layer in pytorch

Question

How can I add a cross layer in pytorch (https://arxiv.org/abs/1708.05123). I would like to add such a cross layer into my neural network that comes before the deep layers as a way to better combine my features that are coming from a range of sources.

Thanks!

Answer 1

There is a PyTorch implementation of the DCN architecture from paper you linked in the DeepCTR-Torch library. You can see the implementation here: https://github.com/shenweichen/DeepCTR-Torch/blob/master/deepctr_torch/models/dcn.py

If you pip install deepctr-torch then you can just use the CrossNet layer as a torch.Module:

from deepctr_torch.layers.interaction import CrossNet

crossnet = CrossNet(in_features=512)