- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies why nearly identical noisy-label fitting can incur different semantic costs. It defines sample-level stable learning events and model-level noise-fitting events. A target-pairing intervention preserves corrupted examples, wrong-target multisets, and class-transition counts to test how example–target relations affect learning order. Gradient alignment serves as an optimization diagnostic, while event-matched linear probes and neighborhood purity assess semantic preservation. The authors also keep example–wrong-target pairs fixed while changing supervision timing, and test functional dependence by retaining the final shallow prefix and rolling the deep suffix back around learning events. Strong augmentation does not prevent eventual noise fitting, but preserves more true-class structure. Later-learned errors depend more on late deep changes and more often recover the true class after rollback.
Critical strengths
The pairing intervention fixes noise rate, corrupted positions, and transition statistics, avoiding changes in error quantity as an explanation for relational effects. Matching noise-fitting events avoids relying only on same-epoch comparisons. At 98% noise fitting, true-label probe accuracies of 48.4% versus 71.5% clearly show that fitting alone does not characterize semantic damage. The fixed-pair crossover of early and delayed supervision provides additional intervention evidence beyond augmentation correlations.
Rollback is compared before and after individual learning events, and the analysis measures whether erased errors recover the true class. Final-margin matching, multivariate analysis, matched late-clean examples, and extensions to ViT and human noise provide complementary checks. I particularly appreciate the explicit restriction to intervention-defined functional suffix dependence, rather than claiming that erroneous information is uniquely stored in deep layers.
Critical weaknesses
-
Equal total active-supervision epochs per example are an important control. However, the early and delayed windows occupy different portions of a cosine learning-rate schedule, and the subsets are evaluated at their own first q=0.98 checkpoints. Effective update magnitude, already learned representations, and total training time can therefore change together. The results support a contribution of when erroneous supervision is applied, not unique attribution to stable learning time itself. The authors already acknowledge this distinction; the wording should remain narrow, with sensitivity to learning rate or effective updates strengthening the interpretation.
-
Combining the final prefix with an earlier suffix can be affected by prefix–suffix co-adaptation, normalization state, and the scale of parameter changes. Pre/post-event comparisons and true-class recovery reduce the concern that arbitrary earlier parameters simply disrupt predictions, while the ViT extension weakens a BatchNorm-only explanation. Nevertheless, the source and recalibration of running statistics, and controls matched by rollback lag or parameter-change magnitude, should be explicit. These are unresolved interpretation conditions, not established implementation errors. The conclusion should remain functional dependence under the specified hybrid-network intervention rather than unique localization or deletion of error memory.
Questions for authors
Q1. Does the fixed-pair timing effect persist with a constant learning rate or approximately matched effective update magnitude? Can you show both common-epoch and common-noise-fit views, including each subgroup’s q=0.98 time and cumulative active supervision before it? Q2. How are normalization parameters and running statistics handled during suffix rollback? Can matched-lag or approximately matched-parameter-change controls distinguish event specificity from cross-time module mismatch? Q3. Please add Table 3’s column headings and specify probe train/test splits and evaluation populations: which results concern all test examples, and which concern corrupted training examples? This would clarify distinctions among decodability, local geometry, and generalization.