'Use only the firsrt 4 layers of XLNET
First sorry for my bad english.
Short version :Can anyone tell me how to use only the first n layers of XLNET for classification ?
Long Version : I have a dataset composed of texts and their summary. The goal is to detect if the summary is generated by a bot or not. So I thought of using bert and give him as input "[CLS] "+Text+" [SEP]"+summary then take the representation of the "[CLS] " token and detect using a classifier if the summary was written by a bot. Th problem is bert takes no more than 512 words as input. So I thought of using XLNET. But here another problem appeared : My gpu (RTX 2060) can't handle a batch of size 1. So I Thought of using only like the first 4 layers of XLNET but the problem is: I don't know how to do it. So my code to load the model is model=XLNetForSequenceClassification.from_pretrained("xlnet-base-cased", num_labels = 2) can anyone tell me what to add to use only a part of the network please ?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|