'TensorFlow Error: Dimensions must be equal but are 6558 and 6562 for '{{node mean_squared_error/SquaredDifference}}

I am working on a 1D Convolutional Neural Network for the classification of malware types. I am currently running into a problem that I cannot seem to figure out.

Here is the need to know info on the error:

    File "C:\Users\Admin\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\keras\losses.py", line 1329, in mean_squared_error
        return backend.mean(tf.math.squared_difference(y_pred, y_true), axis=-1)

    ValueError: Dimensions must be equal, but are 6561 and 6562 for '{{node mean_squared_error/SquaredDifference}} = SquaredDifference[T=DT_FLOAT](sequential/output_layer/BiasAdd, IteratorGetNext:1)' with input shapes: [1,6561,13], [6562,1].

I will put my code following that:

X, _ = d.get_encoded_data()
y = d.get_y()
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=1)

LAYERS = 52 # arbitrary number for now
EPOCHS = 100 # arbitrary

# TODO: define all the testing data

def getModel():
    # all three layers are arbitrarily chosen, don't know input shape
    return keras.Sequential(
        [   
            layers.Conv1D(32, 4, padding='same', activation='relu', name="layer1", input_shape=X_train.shape[1:]),
            layers.Dropout(.3),
            layers.Conv1D(32, 2, activation='relu'),
            layers.Dense(LAYERS / 4, activation='relu', name='layer3'),
            layers.Dropout(.3),
            layers.Dense(13, name='output_layer')
        ])

def plot(history, epochs, metric, type):
    loss_train = history.history[metric]
    epochs = range(1,epochs)
    plt.plot(epochs, loss_train, 'g', label=metric)
    plt.title(type, metric)
    plt.xlabel('Epochs')
    plt.ylabel(metric)
    plt.legend()
    plt.show()

if __name__ == '__main__':
    X_train = X_train.reshape(1, 6562, 50)
    y_train = y_train.reshape(1, 6562, 1)
    
    X_train = X_train.astype(np.float32)
    y_train = y_train.astype(np.float32)
    
    train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))
    
    model = getModel()
    optimizer = keras.optimizers.Adam(learning_rate=0.001)
   
    model.compile(optimizer, 'mse', metrics=[keras.metrics.Accuracy(),
                                                keras.metrics.Precision(), 
                                                keras.metrics.Recall()])

    model.build()
    model.summary()
    history = model.fit(
        train_dataset,
        epochs=EPOCHS
    )
   
    for metric in history.history:
        plot(history, EPOCHS, metric, "Training")
   
    print("Evaluate on test data")
    results = model.evaluate(X_test, y_test)
    print("test loss, test acc:", results)

The main issue here is in the getModel() function and the fit function. My input sizes are (6562, 50) for the X_train and (6562,) for the y_train. Why am I getting this error?

For your info, d.get_encoded_data gets the data as it says, but the _ represents the y. I want y to be one hot encoded tho, which is the line following that. I can't seem to find good tutorials on Conv1D that help at all. Any help is greatly appreciated.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source