- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Auto-LTOpt, a framework that uses large language models and evolutionary search to discover optimizers for learning under class imbalance. It explores modifications to update rules, utility functions, and optimizer classes, using validation performance to retain promising candidates. Experiments on CIFAR, ImageNet-LT, and iNaturalist 2018 show competitive performance across several training objectives, supported by component ablations and analyses of the search process. Quite interesting.
Critical strengths
- The paper explores automated optimizer design for class imbalance, offering greater flexibility than tuning existing update rules.
- Experiments cover four benchmarks and diverse training objectives, providing encouraging evidence of broad applicability.
- The discovered optimizer generally improves overall accuracy over strong baselines, including Muon, with promising results on ImageNet-LT and iNaturalist 2018.
Critical weaknesses
- The contribution beyond adapting ReEvo could be clearer. Comparisons with LLM sampling and ReEvo under equal budgets would strengthen the evaluation.
- The proposed connection between small gradient norms and tail classes needs validation. The implementation also appears to overwrite the direct row scaling. More clarification is encouraged.
- Several gains over Muon are small, and tail accuracy occasionally decreases. More explanation is encouraged.
- The stated search configuration is difficult to reconcile with the 109 evaluated candidates. The cost analysis also excludes LLM overhead.
- The theoretical results mainly describe general properties of elitist search. They provide limited explanation of the discovered optimizer’s effectiveness for rare classes.
Questions for authors
Please refer to weaknesses.