Desk rejected at ICLR 2027
CAUTIOUS DRIVER: A ROBUST, ZERO PRE-TUNING LEARNING RATE SCHEDULER
Desk rejected at ICLR 2027, so left out of the paper lists and the map.
Abstract
Choosing the correct learning rate is one of the most important parts of machine learning. Much work has been done to try to make this process more efficient, but there is still not a fully satisfactory way to do so. This paper introduces ”Cautious Driver”, a method to tune learning rates requiring no pre-tuning and no expensive overhead, that can achieve close to optimal performance across 9 orders of magnitude in learning rate.
Related papers
Discussion (0)
Sign in to comment.