training stops after a while

Question

i'm trying to solve classification problem(cat vs dog) in the middle of the first epoch my training got interrupted i'm using image_dataset_from_directory to read my images:

train_dataset = image_dataset_from_directory(data_set_path+'/train' , labels='inferred',label_mode='binary', batch_size=32, image_size=(180,180), seed= 348)
val_dataset = image_dataset_from_directory(data_set_path+'/val' , labels='inferred', label_mode='binary', batch_size=32, image_size=(180,180), seed= 348)
test_dataset = image_dataset_from_directory(data_set_path+'/test' , labels='inferred', label_mode='binary', batch_size=32, image_size=(180,180), seed= 348)

and my model is as follow:

inputs = keras.Input(shape=(180,180,3))
x = data_augmentation(inputs)
x = layers.Rescaling(1./255)(x)
x = layers.Conv2D(32,3,padding = 'same', use_bias = False)(x)
for filter_size in (32,64,128,256,512):
    residual = x
    x = layers.BatchNormalization()(x)
    x = layers.Activation('relu')(x)
    
    x = layers.SeparableConv2D(filter_size,3,padding= 'same' ,use_bias=False)(x)
    x = layers.BatchNormalization()(x)
    x = layers.Activation('relu')(x)
    
    x = layers.SeparableConv2D(filter_size,3,padding= 'same', use_bias=False)(x)
    x = layers.BatchNormalization()(x)
    x = layers.Activation('relu')(x)
    
    x = layers.MaxPooling2D(3, strides = 2 , padding= 'same')(x)
    
    residual = layers.Conv2D(filter_size,1,strides = 2,padding= 'same', use_bias= False) 
    (residual)
    x = layers.add([x, residual])
x = layers.GlobalAveragePooling2D()(x)
x = layers.Dropout(0.5)(x)
output = layers.Dense(1, activation = 'sigmoid')(x)
model = keras.Model(inputs = inputs, outputs=output)

in the middle of first epoch i get this error:

InvalidArgumentError: Graph execution error:

2 root error(s) found.
  (0) INVALID_ARGUMENT:  Input is empty.
     [[{{node decode_image/DecodeImage}}]]
     [[IteratorGetNext]]
     [[model_1/sequential/random_rotation/Shape/_4]]
  (1) INVALID_ARGUMENT:  Input is empty.
     [[{{node decode_image/DecodeImage}}]]
     [[IteratorGetNext]]
0 successful operations.
0 derived errors ignored. [Op:__inference_train_function_7730]

i would be grateful if anyone can help me

Answer 1

I am not sure if it is the case for your error but I had problems with dogs vs cats dataset before. A few of the images were corrupted and needed to be removed before training could proceed.

Try reading the images with opencv or skimage and remove the ones with exception.

Issue with dataset, https://discuss.pytorch.org/t/pil-unidentifiedimageerror-in-dataloader/73577.