acceptodds
Desk rejected at ICLR 2027

CAUTIOUS DRIVER: A ROBUST, ZERO PRE-TUNING LEARNING RATE SCHEDULER

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Choosing the correct learning rate is one of the most important parts of machine learning. Much work has been done to try to make this process more efficient, but there is still not a fully satisfactory way to do so. This paper introduces ”Cautious Driver”, a method to tune learning rates requiring no pre-tuning and no expensive overhead, that can achieve close to optimal performance across 9 orders of magnitude in learning rate.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Scheduler, Learning Rate

Related papers

Discussion (0)

Sign in to comment.