Using Recurrent Neural Networks for binary values prediction

Question

EDIT: The problems stated have been solved, you'll first find the solution, the initial question is stated below!

SOLUTION:

Applying the .unsqueeze(0) to my inputs solved the problem with the dimensions.

Second Problem:

RuntimeError: expected scalar type Double but found Float

This was caused by a wrong datatype of my training set. The networks wanted the X_train as dtype=torch.float32, the y however as dtype=torch.long. I do not know an explanation for this however, maybe someone could answer this as a comment.

Also, the input_size for the model was changed to 35, as my data has 35 features. The output was fixed to 2, since I do want a prediction 1 for good and 0 for bad.

model = RNN(35, hidden_size, num_layers, 2).to(device)

PROBLEM:

I'm trying to predict 10 different binary values for every sample based on a training set with 35 features per sample. My approach was using an RNN with 2 layers, but fail to implement it properly. As I try to train the model with batches of size 64, the return simply is

RuntimeError: input.size(-1) must be equal to input_size. Expected 64, got 35

If I do set the batch size to 35 to see what's next, I receive the

RuntimeError: expected scalar type Double but found Float

Why does the model use the features as input size and where could the double problem come from, as I did not find similar notations in other forums or posts.

Thank you in advance for your help.

The RNN:

class RNN(nn.Module):
    def __init__(self, input_size, hidden_size ,num_layers, output_size):
        super(RNN, self).__init__()
        self.num_layers = num_layers
        self.hidden_size = hidden_size
        self.rnn = nn.RNN(input_size, hidden_size, num_layers, nonlinearity='relu',batch_first=True)

        self.fc = nn.Linear(hidden_size, output_size)
        self.softmax = nn.Softmax(dim=1)
        

    def forward(self, x):
        
        # self.hidden_size).to(device)
        h = torch.zeros(self.num_layers, x.size(0), self.hidden_size)

        out, _ = self.rnn(x, h)

        out = out.contiguous().view(-1, self.hidden_size)

        out = self.fc(out)

        out = self.softmax(out)

        return out

Implementing the model by:

model = RNN(batch_size, hidden_size, num_layers, output_size).to(device)

and using the training loop:

for epoch in range(num_epochs):
    for index, (inputs, labels) in enumerate(dataloader):  
        # forward, backward, update
        print(f'inputs {inputs.shape}')
        

        # Forward pass
        outputs = model(inputs.unsqueeze(1))
        loss = criterion(outputs, labels)
        
        # Backward and optimize
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        
        print (f'Epoch [{epoch+1}/{num_epochs}], Step [{index+1}/{n_total_steps}], Loss: {loss.item():.4f}')

The unsqueeze is used in order to add a dimension of the input, as the model requires 3 dimensions, but only 2 were given by the dataset

Answer 1

The Network definition should be independent of the batch size. The input_size is the number of features.