Auto-LTOpt: Automated Optimizer Evolution for Extreme Class Imbalance

Practical learning tasks frequently exhibit imbalanced long-tailed data distributions, posing significant challenges for standard optimizers to generalize to tail classes. Existing long-tailed optimizers rely heavily on hand-crafted rules and limited gradient adjustment strategies, resulting in poor flexibility and generalization. In this paper, we propose Auto-LTOpt, a novel automated framework that uses large language models to automatically design tailored optimizers for long-tailed learning without human intervention. By exploring a rich search space of gradient manipulation operations through program evolution, Auto-LTOpt can autonomously discover effective optimization strategies to balance head and tail classes. Extensive experiments on standard long-tailed benchmarks show that Auto-LTOpt outperforms state-of-the-art handcrafted optimizers, while achieving strong and stable performance across diverse imbalanced settings. Our code is available at Supplement.