Should I shuffle the test image dataset in ImageDataGenerator? I have different results with False and True

Question

I trained CNN on the train set with shuffle = True . Then, if I set shuffle = True on my test dataset i have accuracy about 50%, while with shuffle = False I have 94-95% accuracy. Interestingly, with shuffle = False both labels and data are not shuffled, which is understandable, but with shuffle = True labels remain in the same order, while data is reshuffled. As a result, in the last case the accuracy drops dramatically to 50%. Why are labels not reshuffled?

Answer 1

https://gist.github.com/RyanAkilos/3808c17f79e77c4117de35aa68447045

According to this discussion, the answer is no, you should not shuffle the test dataset.