'The encoder of my autoencoder returns zero values
i'm trying to reduce the dimension of my data set by using an auto encoder my data set contains vectors of size 21300 where the vectors contain a lot of 0's at the end.
encoder_input = keras.Input(shape=(21300,), name='sum')
encoded = keras.layers.Dense(units=13000, activation='relu')(encoder_input)
encoded = keras.layers.Dense(units=10000, activation='relu')(encoded)
encoded = keras.layers.Dense(units=7500, activation='relu')(encoded)
encoded = keras.layers.Dense(units=3000, activation='relu')(encoded)
encoded = keras.layers.Dense(units=1500, activation='relu')(encoded)
encoded = keras.layers.Dense(units=500, activation='relu',name="bf4enc1")(encoded)
encoder = keras.layers.Dense(units=40, activation='relu',name='encoder')(encoded)
decoded = keras.layers.Dense(units=500, activation='relu', name='afterenc1')(encoder)
decoded = keras.layers.Dense(units=1500, activation='relu')(decoded)
decoded = keras.layers.Dense(units=3000, activation='relu')(decoded)
decoded = keras.layers.Dense(units=7500, activation='relu')(decoded)
decoded = keras.layers.Dense(units=10000, activation='relu')(decoded)
decoded = keras.layers.Dense(units=13000, activation='relu')(decoded)
decoder_output = keras.layers.Dense(units=21300)(decoded)
opt = tf.keras.optimizers.Adam(learning_rate=0.001, decay=1e-6)
autoencoder = keras.Model(encoder_input, decoder_output, name='autoencoder')
autoencoder.compile(opt, loss='mse')
print("STARTING FITTING")
history = autoencoder.fit(
movies_vector,
movies_vector,
epochs=25,
batch_size=16
)
#autoencoder.save(f"models/AE-{epoch+1}.model")
#encoder_input.save(f"models/EE-{epoch+1}.model")
print("ENCODER READY")
#USING THE MIDDLE LAYER
encoder = keras.Model(inputs=autoencoder.input,
outputs=autoencoder.get_layer('encoder').output)
however after using the encoder on the exact same data set it was trained on it doesn't compress my vectors, instead all compressed vectors have 0 as their values. i've tried changing the epochs from 25 to 1000 to no avail. how can the decoder scale the compressed vector from 0 values into the original vector with 0.29 loss?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
