- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper audits LSNPC, a latent-shift method for proposing corrections to noisy training labels. It defines four per-correction scores inspired by counterfactual properties and studies whether they distinguish repairs from harmful edits. Across text and image datasets with human and injected noise, confidence changes from anti-informative to informative as the corrector's recovery rate changes. Label-free thresholding generally preserves the gains of unselective correction, but often performs close to matched random rejection. Experiments include clean-label diagnostics, downstream retraining with three fixed model types, Co-teaching comparisons, and seed and encoder analyses.
Critical strengths
- The paper is unusually careful about separating clean-label diagnostics from the label-free setting that would matter in practice. This makes the interpretation of the results more transparent and avoids presenting diagnostic evidence as if it were directly available at deployment time.
- The study covers multiple modalities, both real and injected label noise, and several downstream model types. This breadth, together with the additional transition-level and seed-level analyses, gives the audit a more comprehensive scope than a single-dataset evaluation.
Critical weaknesses
- The strongest selection evidence is diagnostic, not the deployment setting. Table 3 and Section 5.2 use clean-label-referenced scores on a WR/RW subset. In paired label-free runs, noise-robustness is near chance at 0.55 and proximity is 0.66. End-to-end, thresholding is often within half an accuracy point of matched random rejection.
- Evidence across runs is limited. Main retraining results average five corrector seeds and five downstream seeds, while ten-seed replication covers only two settings. Appendix A says the retained run archive does not record the corrector's trusted-set size.
Questions for authors
- Can the authors report paired uncertainty intervals for the end-to-end advantage over matched random rejection in each primary setting and downstream head?
- Can they verify the trusted-set size and full corrector configuration used in the archived runs, and report whether conclusions persist across more corrector seeds?
Thank you all
We are sincerely grateful to all the reviewers. We submitted the work without much confidence that it might be of interest beyond our team. The thoughtful reviews encouraged us and also gave us many real insights that could help us improve the work substantially. Thank you!