- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies learning with noisy labels using frozen base model representations. By comparing standard cross-entropy loss and noise-sensitive linear outputs against clean label references, the paper show limited accuracy gains under moderate noise conditions, but significant improvements under more severe interference. The paper also examines probabilistic reliability separately and evaluates the AEC method. Experiments on both synthetic and real-world noisy label benchmarks show that high accuracy can coexist with poor calibration, while AEC generally improves calibration performance, though it may also lead to a certain degree of performance degradation.
Critical strengths
The problem setup is clear and well-defined, and the use of fixed encoder and readout family makes the comparison results easier to interpret. The matched clean-label reference is useful, and the paper appropriately avoids treating it as a mathematical upper bound. I also appreciate the cautious conclusions in the paper: it reports cases where robust training still leads to significant accuracy gains, while also pointing out scenarios in which AEC may degrade calibration performance.
Critical weaknesses
AEC selects the top 10% of examples with the highest confidence in each predicted class and adjusts their average confidence to 0.995. However, the main paper does not adequately justify why this target is appropriate across different predictors and data corruption settings. The authors acknowledge this limitation, and Figure 1 shows that calibration performance may worsen. A clearer analysis is needed regarding the rationale behind the target setting, the accuracy of anchor predictions, and observed failure cases, to clarify under what conditions AEC should be effective. The direct comparison of calibration methods focuses on a single dataset, which is insufficient to determine AEC's consistent advantage over other calibration strategies. It is particularly important to distinguish whether the gains come from high-confidence anchors, class-conditioned temperature scaling, or more general probability scaling. The NLL and Brier results only show two CE settings, while broader conclusions are primarily based on ECE. Since AEC may increase confidence in incorrect predictions, improvements in ECE alone are insufficient to fully assess probability quality. It is recommended that the main experimental discussion explicitly clarify whether changes in ECE align with NLL/Brier under different training rules and reported failure cases.
Questions for authors
How were the target confidence of 0.995 and the anchor fraction of 0.1 chosen? What evidence connects the target to anchor prediction accuracy, and how sensitive are the conclusions to these choices? Do AEC’s ECE improvements also correspond to improvements in NLL and Brier across GCE, SSR, and DivideMix, including the failure cases? How does its comparison with noisy-validation temperature scaling change in harder settings?