RuntimeError: output with shape [1] doesn't match the broadcast shape [10]

Question

Hi I trying to make RBM Model code using pytorch module but got a issue in visible layer to hidden layer. Here is the problem part code.

        h_bias = (self.h_bias.clone()).expand(10)
        v = v.clone().expand(10)
        
        p_h = F.sigmoid(
            F.linear(v, self.W, bias=h_bias)
        )

        sample_h = self.sample_from_p(p_h)
        return p_h, sample_h

and each parameters size is here.

h_bias           v                self.W
torch.Size([10]) torch.Size([10]) torch.Size([1, 10])
1                1                2

Traceback (most recent call last):
  File "/Users/bahk_insung/Documents/Github/ecg-dbn/model.py", line 68, in <module>
    v, v1 = rbm(sample_data)
  File "/Users/bahk_insung/miniforge3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/bahk_insung/Documents/Github/ecg-dbn/RBM.py", line 54, in forward
    pre_h1, h1 = self.v_to_h(v)
  File "/Users/bahk_insung/Documents/Github/ecg-dbn/RBM.py", line 36, in v_to_h
    F.linear(v, self.W, bias=h_bias)
  File "/Users/bahk_insung/miniforge3/lib/python3.9/site-packages/torch/nn/functional.py", line 1849, in linear
    return torch._C._nn.linear(input, weight, bias)
RuntimeError: output with shape [1] doesn't match the broadcast shape [10]

I think dimension and size are not matched thats reason why happened. But I can't get any solutions. Please help me guys. Thank you.

Answer 1

I solved with torch.repeat() function. As mandias said...

you are trying to create an output of size "1" with an input size of "10". The linear transform does not know how to change your inputs of size 10 into an output of size 1.

That was a my problem. So I changed weight input like this.

w = self.W.clone().repeat(10, 1)

Originally, self.W size is [1, 10]. After used repeat function then changed as [10, 10]. Input is 10 size, and Output is 10.

Tbh not sure this code is right thing but I HAVE TO run this code quickly... anyway thank you guys.

Answer 2

If you look at the pytorch functional.linear documentation it shows the weight parameter can be either 1D or 2D: "Weight: (out_features, in_features) or (in_features)". Since your weight is 2D ([1, 10]) it indicates that you are trying to create an output of size "1" with an input size of "10". The linear transform does not know how to change your inputs of size 10 into an output of size 1. If your weight will always be [1, N] then you can use squeeze to change it to 1D like so:

F.linear(v, self.W.squeeze(), bias=h_bias)

This would create an output of size 10.