Keras model with 2 inputs

Question

I am dealing with a binary classification problem that feeds a network with two inputs (images),

model_vgg16_conv = VGG16(weights='imagenet', include_top=False)
for layer in model_vgg16_conv.layers:
    layer.trainable = False
model_vgg16_conv.summary()


input1 = Input(shape=(60,36,3))
input2 = Input(shape=(60,36,3))
concate_input = concatenate([input1, input2])
input = Conv2D(3, (3, 3), 
                     padding='same', activation="relu")(concate_input)
#Use the generated model 
output_vgg16_conv = model_vgg16_conv(input)

#Paso la salida del modelo por varas capas. 
x = GlobalAveragePooling2D()(output_vgg16_conv) 
x = Dense(512,activation='relu')(x)
predictions = Dense(1, activation='sigmoid')(x)
#Create your own model 
my_model = Model(inputs=[input1, input2], outputs=predictions)

#In the summary, weights and layers from VGG part will be hidden, but they will be fit during the training
my_model.summary()


my_model.compile(loss='binary_crossentropy',                # función de pérdida para problemas de clasificación multi-clase
              optimizer=optimizers.Adam(learning_rate=1e-4),  # optimizador Adam
              metrics=['accuracy'])

Model: "model_5"
__________________________________________________________________________________________________
 Layer (type)                   Output Shape         Param #     Connected to                     
==================================================================================================
 input_17 (InputLayer)          [(None, 60, 36, 3)]  0           []                               
                                                                                                  
 input_18 (InputLayer)          [(None, 60, 36, 3)]  0           []                               
                                                                                                  
 concatenate_5 (Concatenate)    (None, 60, 36, 6)    0           ['input_17[0][0]',               
                                                                  'input_18[0][0]']               
                                                                                                  
 conv2d_5 (Conv2D)              (None, 60, 36, 3)    165         ['concatenate_5[0][0]']          
                                                                                                  
 vgg16 (Functional)             (None, None, None,   14714688    ['conv2d_5[0][0]']               
                                512)                                                              
                                                                                                  
 global_average_pooling2d_5 (Gl  (None, 512)         0           ['vgg16[0][0]']                  
 obalAveragePooling2D)                                                                            
                                                                                                  
 dense_10 (Dense)               (None, 512)          262656      ['global_average_pooling2d_5[0][0
                                                                 ]']                              
                                                                                                  
 dense_11 (Dense)               (None, 1)            513         ['dense_10[0][0]']               
                                                                                                  
==================================================================================================
Total params: 14,978,022
Trainable params: 263,334
Non-trainable params: 14,714,688

My question is if I am feeding the network correctly? The Vgg16 layer displays the value (none, none, none, 3). Is it right?

Answer 1

It's ok, the problem is that this architecture introduces overfitting. The solution is to use weighted weights or oversampling.