Converting PyTorch Boolean target to regression target

Question

Question

I have code that is based on Part 2, Chapter 11 of Deep Learning with PyTorch, by Luca Pietro Giovanni Antiga, Thomas Viehmann, and Eli Stevens. It's working just fine. It predicts the value of a Boolean variable. I want to convert this so that it predicts the value of a real number variable that happens to be always between 0 and 34.

There are two parts that I don't know how to convert. First, this part:

    pos_t = torch.tensor([
            not candidateInfo_tup.isNodule_bool,
            candidateInfo_tup.isNodule_bool
        ],
        dtype=torch.long,
    )

(Why are two values passed in here when one is completely determined by the other?)

and then this part:

    self.head_linear = nn.Linear(1152, 2)
    self.head_softmax = nn.Softmax(dim=1)

How do I do this?

Guess

I don't want people to think I haven't thought about this at all, so here is my guess:

First part:

    age_t = torch.tensor(candidateInfo_tup.age_int, dtype=torch.double)

Second part:

    self.head_linear = nn.Linear(299520, 1)
    self.head_relu = nn.ReLU()

I'm also guessing that I need to change this:

    loss_func = nn.CrossEntropyLoss(reduction='none')

to something like this:

    loss_func = nn.L1Loss()

My guesses are based on this article by Christian Versloot.

Answer 1

The example from the book is working but it has some redundant elements which confuse you. Normally output size of 1 is enough for a binary classification problem. To bring it to 0 or 1, one may use sigmoid and then rounding, like in the example here: PyTorch Binary Classification - same network structure, 'simpler' data, but worse performance? Or just put after single output neuron this:

y_pred_binary = torch.round(torch.sigmoid(y_pred))

The book example uses output size of 2 and then applies softmax to get to 0 and 1. This works, but such technique is typically used in multi-class classification.

For prediction of 0-34 variable:

• if these are discrete variables - it is called "multiclass classification" as Ken indicated. Use size 35 output and softmax in this case. Search for "pytorch multiclass classification" for examples.
• if this is a regression - than your changes seem in the right direction, except 'Second part'. instead of RELU - clip output at both ends at [0, 34]. Also 299520 - is too much for the previous layer. Use whatever input size there was before. Search for "pytorch regression" for examples.