How to save the best model instead of the last one for Detectron2

Question

I want to save the best model instead of the last model for detectron2. The evaluation metric I want to use is AP50 or something similar. The code I currently have is:

trainer.register_hooks([
            EvalHook(eval_period=20, eval_function=lambda:{'AP50':function?}), 
            BestCheckpointer(eval_period=20, checkpointer=trainer.checkpointer, val_metric= "AP50", mode="max")
            ])

But I have no idea what I have to substitute for the function in EvalHook. I use a subset of the coco dataset to train the model, and I saw that detectron2 contains some evaluation measures for the coco dataset, but I have no idea how to implement this.

Answer 1

This notebook has an implementation of what you asked and what I am searching for...

trainer.resume_or_load(resume=False)
if cfg.TEST.AUG.ENABLED:
  trainer.register_hooks(
    [hooks.EvalHook(0, lambda: trainer.test_with_TTA(cfg, trainer.model))]   #this block uses a hook to run evalutaion periodically
  )                                           #https://detectron2.readthedocs.io/en/latest/modules/engine.html#detectron2.engine.hooks.EvalHook
trainer.train()

Try this...

I will report here if it works.