'Is Hyperband tuning algorithm from KerasTuner appropriate for Learning rate?
I'm doing hyperparameter tuning with KerasTuner. I usually work with Hyperband but I have the feeling it would not be adapted for learning rate, since the algorithm train the models for a very few number of epoch in the initial stages, then train further only the more promising hyperparameter combinations. So, with all others hyperparameter remaining constant, high Lr (e.g. 1e-2) will have more chance to get selected in the early stages of Hyperband compared to smaller Lr (e.g. 1e-3), but without guarantee it would indeed be the best after a full training.
Am I wrong ?
Best,
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|