'PyTorch Lightning (Trainable Params - Wrong)
I am employing MULTI-GPU training using pytorch lightning. The below output displays the model:
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1,2,3]
┏━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃ ┃ Name ┃ Type ┃ Params ┃
┡━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│ 0 │ encoder │ Encoder │ 2.0 M │
│ 1 │ classifier │ Sequential │ 8.8 K │
│ 2 │ criterion │ BCEWithLogitsLoss │ 0 │
│ 3 │ train_acc │ Accuracy │ 0 │
│ 4 │ val_acc │ Accuracy │ 0 │
│ 5 │ train_auc │ AUROC │ 0 │
│ 6 │ val_auc │ AUROC │ 0 │
│ 7 │ train_f1 │ F1Score │ 0 │
│ 8 │ val_f1 │ F1Score │ 0 │
│ 9 │ train_mcc │ MatthewsCorrCoef │ 0 │
│ 10 │ val_mcc │ MatthewsCorrCoef │ 0 │
│ 11 │ train_sens │ Recall │ 0 │
│ 12 │ val_sens │ Recall │ 0 │
│ 13 │ train_spec │ Specificity │ 0 │
│ 14 │ val_spec │ Specificity │ 0 │
└────┴────────────┴───────────────────┴────────┘
Trainable params: 2.0 M
Non-trainable params: 0
I have set Encoder to be untrainable using the below code:
ckpt = torch.load(chk_path)
self.encoder.load_state_dict(ckpt['state_dict'])
self.encoder.requires_grad = False
Shouldn't trainable params be 8.8 K rather than 2.0 M ?
My optimizer is the following:
optimizer = torch.optim.RMSprop(filter(lambda p: p.requires_grad, self.parameters()), lr =self.lr, weight_decay = self.weight_decay)
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
