How can I save a few best Keras models?

Question

I need to save a few best models. I have defined ModelCheckpoint from Keras, which saved only one best model.

checkpoint = tf.keras.callbacks.ModelCheckpoint(
     filepath=checkpoint_path, 
     monitor='val_loss', 
     verbose=1, 
     save_best_only=True,
     save_weights_only=True, 
     mode='min'
)

Any ideas?

Answer 1

save_best_only=False

This way you will save the model after each epoch,and not only when the model is considered "the best".

Remember to define a path that changes for each epoch or the model will be overwritten.

AFTER THE COMMENT:

If your callback is saving only one model is probably because you have defined the path to be static, you need to use the keras formatting, so that the name on the file where the model is saved changes every epoch, otherwise it will be overwritten.

You need to define the path in something like this:

filepath= model_folder_path + "/epoch:{epoch:02d}-val_loss:{val_loss:.2f}.hdf5"

If you define the path to be static every time keras goes to save the new best model the model saved just before will be overwritten and you will remain with only one of them (the last one saved).

Go check the doc for more info on how to write the path:

https://keras.io/api/callbacks/model_checkpoint/

AFTER THE SECOND COMMENT:

Ok now what you want is to save only some of this best models and not everyone of them, I achieved this result using a callback like this:

class too_many_models(Callback):
  def __init__(self, mypath, max_num_of_models):
    self.path = mypath
    self.mnom = max_num_of_models

  def  on_epoch_end(self, epoch, logs=None):
    onlyfiles = [f for f in listdir(self.path) if isfile(join(self.path, f))]
    onlyfiles.sort()
    if len(onlyfiles) > self.mnom:
      os.remove(join(self.path, onlyfiles[0]))

model_folder = "supertest"
!mkdir "/content/gdrive/My Drive/"$model_folder

folderpath = "/content/gdrive/My Drive/" + model_folder
filepath = folderpath + "/epoch:{epoch:02d}-val_loss:{val_loss:.2f}.hdf5"
 
my_callbacks = [
    tf.keras.callbacks.ModelCheckpoint(filepath=filepath, save_weights_only=False, monitor='val_loss', mode='min', save_best_only=True),
    too_many_models(mypath=folderpath, max_num_of_models=5)
    ]

you want to use this callback like this:

model.fit(train, steps_per_epoch=len(train), validation_data=valid, callbacks=my_callbacks)

A few things to note:

the callback too_many_models is called after the callback that saves the model.

The folder needs to be empty before starting the training, you need a different folder for every training (but I usually do this, so I don't see it as a problem).

I sort the files, in this way I am sure the model I am removing (at index 0) is always the older one saved (with the lower number in epoch), this works only because i defined the names of the models in that way, the number of the epoch sort the files as I want them. (I hope this point is clear)

you can change the number of models saved with max_num_of_models, with 5 it will save the last best 5 models.

folderpath needs to be the path where you are saving the models. (pay attention on how I defined the different paths)

I just tested everything to be sure it works as intended.

I know it's a pretty rough solution, but it gets the job done :D