'What is the best Learning Rate Scheduler for your model?

My purpose in this discussion - Find the good method to choose best Learning Rate Scheduler in model. I know No Meal For Free, continuously training, trying many Scheduler to find the last method with the best performance sometime is the last solution. But I hope you could share some your experiences, we can collect and use in future.

Firstly, I searched kernels that introducing some Learning Rate Scheduler (LRS)

https://www.kaggle.com/code/tolgadincer/tf-keras-learning-rate-schedulers/

https://www.kaggle.com/code/snnclsr/learning-rate-schedulers/

https://www.kaggle.com/code/isbhargav/guide-to-pytorch-learning-rate-scheduling

Secondly, I have some questions for beginning

  1. What points do you think when choosing a LRS?
  2. If we choose 1 in 2 LRS, what standard do you get result for the better one?


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source