Local Look-Ahead Ensembling: Do We Really Need a Second Network for Learning with Noisy Labels?
Desk rejected at ICLR 2027, so left out of the paper lists and the map.
Abstract
Recent learning with noisy labels (LNL) methods are commonly formulated as semi-supervised learning problems, where a teacher is used for sample selection and pseudo-label generation. Among existing approaches, co-training-based methods have achieved strong performance by maintaining two networks that exchange information during training. However, this design nearly doubles the computational cost, raising the question of whether a second network is truly necessary. In this paper, we introduce Local Look-Ahead Ensembling (LLE), a teacher construction strategy designed to replace the peer network in co-training-based LNL methods. Instead of training a second model from scratch, LLE constructs several local models by performing a small number of look-ahead optimization steps from the current model and aggregates their predictions to form a teacher. As a result, existing two-network LNL frameworks can be converted into single-network methods without changing their overall training logic. We first analyze LLE in a controlled semi-supervised learning setting and show that it primarily modifies pseudo-labels on low-confidence samples while providing substantially more accurate predictions on these examples. We then integrate LLE into representative co-training-based LNL methods, including DivideMix, UniCon, and C2D. Experimental results on CIFAR-10 and CIFAR-100 demonstrate that LLE preserves performance at low and moderate noise rates, consistently improves accuracy under severe label noise, and significantly reduces training cost by eliminating the need for a second network.
- Keywords:
- learning with noisy labels, label correction, co training