Local Look-Ahead Ensembling: Do We Really Need a Second Network for Learning with Noisy Labels?

Recent learning with noisy labels (LNL) methods are commonly formulated as semi-supervised learning problems, where a teacher is used for sample selection and pseudo-label generation. Among existing approaches, co-training-based methods have achieved strong performance by maintaining two networks that exchange information during training. However, this design nearly doubles the computational cost, raising the question of whether a second network is truly necessary. In this paper, we introduce Local Look-Ahead Ensembling (LLE), a teacher construction strategy designed to replace the peer network in co-training-based LNL methods. Instead of training a second model from scratch, LLE constructs several local models by performing a small number of look-ahead optimization steps from the current model and aggregates their predictions to form a teacher. As a result, existing two-network LNL frameworks can be converted into single-network methods without changing their overall training logic. We first analyze LLE in a controlled semi-supervised learning setting and show that it primarily modifies pseudo-labels on low-confidence samples while providing substantially more accurate predictions on these examples. We then integrate LLE into representative co-training-based LNL methods, including DivideMix, UniCon, and C2D. Experimental results on CIFAR-10 and CIFAR-100 demonstrate that LLE preserves performance at low and moderate noise rates, consistently improves accuracy under severe label noise, and significantly reduces training cost by eliminating the need for a second network.