Why does image_dataset_from_directory return a different array than loading images normally?

Question

I noticed that the output from TensorFlow's image_dataset_from_directory is different than directly loading images (either by PIL, Keras' load_img, etc.). I set up an experiment: I have a single RGB image with dimensions 2400x1800x3, and tried comparing the resulting numpy arrays from the different methods:

from PIL import Image
from tensorflow.keras.utils import image_dataset_from_directory, load_img, img_to_array


train_set = image_dataset_from_directory(
    '../data/',
    image_size=(2400, 1800), # I'm using original image size
    label_mode=None,
    batch_size=1
)

for batch in train_set:
    img_from_dataset = np.squeeze(batch.numpy()) # remove batch dimension
    img_from_keras = img_to_array(load_img(img_path))
    img_from_pil = img_to_array(Image.open(img_path))


    print(np.all(img_from_dataset == img_from_keras)) # False
    print(np.all(img_from_dataset == img_from_pil))   # False
    print(np.all(img_from_keras == img_from_pil))     # True

So, even though all methods return the same shape numpy array, the values from image_dataset_from_directory are different. Why is this? And what can/should I do about it?

This is a particular problem during prediction time where I'm taking a single image (i.e. not using image_dataset_from_directory to load the image).

Answer 1

This is strange but I have not figured out exactly why but if you print out a pixel values from the img_from_dataset, img_from_keras and img_from_pil I found that the pixel values for img_from_data are sometimes lower by 1, that is it looks like some kind of rounding is going on. All 3 are supposed to return float32 so I can't see why they should be different. I also tried using ImageDataGenerator().flow_from_directory and it matches the data for img_from_keras and img_from_pil. Now img_from_dataset return a A tf.data.Dataset object it yields float32 tensors of shape (batch_size, image_size[0], image_size[1], num_channels). I used this code to detect the pixel value difference where I used a 224 X 224 X3 image

match=True
for i in range(224):
    for j in range(224):
        for k in range (3):
            if img_from_dataset[i,j,k] != img_from_keras[i,j,k]:
                match=False
                print(img_from_dataset[i,j,k], img_from_keras[i,j,k], i, j, k)
                break
        if match==False:
            break
    if match == False:
        break
print(match)

An example output of the code is

86.0 87.0 0 0 2
False

If you ever figured out why the difference let me know. I expect one will have to go through the detailed code. I took a quick look. Even though you specified the image size as being the same as the original image, image_dataset_from_directory still resizes the image using tf.image.resize with the iterpolation as interpolation='bilinear'. Maybe the load_img(img_path) and PIL image.open use different interpolations.