CAUTIOUS DRIVER: A ROBUST, ZERO PRE-TUNING LEARNING RATE SCHEDULER

Choosing the correct learning rate is one of the most important parts of machine learning. Much work has been done to try to make this process more efficient, but there is still not a fully satisfactory way to do so. This paper introduces ”Cautious Driver”, a method to tune learning rates requiring no pre-tuning and no expensive overhead, that can achieve close to optimal performance across 9 orders of magnitude in learning rate.