Custom Bert and unresolved self.init_weights()

Question

currently I get a warning about unresolved attribute reference self.iniit_weights() in my customized Bert. Do you know how I can fix it? Neural networks and transformers are very new to me. I use thew newest version of transformers 4.9.1 and python 3.8.

My Bert class looks like this and is very similar to an example found online, it only has more classifiers.

class MyBert(BertPreTrainedModel):

    def __init__(self, config, num_labels1=None, num_labels2=None, num_labels3=None):
        super().__init__(config)
        self.num_labels1 = num_labels1
        self.num_labels2 = num_labels2
        self.num_labels3 = num_labels3
        self.bert = BertModel(config)
        self.dropout = torch.nn.Dropout(config.hidden_dropout_prob)
        self.classifier1 = torch.nn.Linear(config.hidden_size, num_labels1)
        self.classifier2 = torch.nn.Linear(config.hidden_size, num_labels2)
        self.classifier3 = torch.nn.Linear(config.hidden_size, num_labels3)
        self.init_weights() # unresolved attribute reference

Thank you for any help.

Answer 1

I don't know whether you still need help for so long, but here are things about your model:

Since you were using transformers I believe the your model is a subclass of BertPreTrainedModel which is a subclass of PreTrainedModel, and this is where the method init_weights() comes from.

The link to the documentation is here