'TimeDistributed effect on LSTM performance

I have read in previous posts that using the TimeDistributed parameter in the LSTM model does not change the performance or how the layers work (It only gives an additional information on each timestep). Except that I noticed a big difference in accuracy between :

model = Sequential()
model.add(LSTM(120, input_shape = (468, 1000)))
model.add((Dense(1)))
model.compile(loss='binary_crossentropy', optimizer=adam.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, amsgrad=False), metrics=['accuracy'])

accuracy on my data = 0.94

model = Sequential()
model.add(LSTM(120, input_shape = (468, 1000), return_sequences = True))
model.add(TimeDistributed(Dense(1)))
model.compile(loss='binary_crossentropy', optimizer=adam.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, amsgrad=False), metrics=['accuracy'])

accuracy on my data = 0.52

I can't see this difference on toy examples, but the difference on my data is huge (accuracy is avereged over 10 iterations) even on the same train/test split. Is it normal ?



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source