Dimension issues for LSTM sequence model on Keras

Question

I'd like to train a simple LSTM model for sequence data with 128 time steps with 6 features for 118 multi-classes.

The dimensions of the dataset are shown below:

X_train, X_test shape: (batch, timesteps, num_features) = (batch, 128, 6)
y_train, y_test shape: (batch, 118)

where labels are represented by one-hot encoding with 118 classes.

model = keras.Sequential([
    tf.keras.layers.LSTM(units = 32, kernel_initializer = 
    tf.initializers.zeros()),
    tf.keras.layers.Dense(units = 6)
])

model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(X_train, y_train, validation_data = (X_test, y_test), epochs=20, verbose=0)

There is an error below after executing the code above:

ValueError: Shapes (None, 118) and (None, 6) are incompatible

How to fix the dimension issue?

Answer 1

The units parameter in tf.keras.layers.Dense() is the dimensionality of the output space.

Since you have used 6 units in the last dense layer, the model will process the input and return a tensor of shape (None, 6) and compare it with the labels of shape (None, 118), which would be incompatible.

Try changing the number of units to 118 in the last dense layer to get a compatible model.

Answer 2

I think the issue is because your network will output a vector of size 6, while your expected output is 118. so You have two ways. 1- You can change ur "tf.keras.layers.Dense(units = 6)" to "tf.keras.layers.Dense(units = 118)"

2- you can add one more dense layer to reshape the output to 118. so the final model will be

model = keras.Sequential([tf.keras.layers.LSTM(units = 32, kernel_initializer = tf.initializers.zeros()), tf.keras.layers.Dense(units = 6),tf.keras.layers.Dense(units = 118)])

Another issue you might face is the None at the shape. I'm not sure why but u might need to check the data or something.