- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper challenges the implicit complementarity in label refurbishment: reducing trust in the observed label automatically increases the pseudo-target weight. TRACE separately estimates observed-label and pseudo-target reliability, sets a=s_obs and b=(1−s_obs)s_pseudo, and uses them to construct a corrected target and total sample weight. Observed-label reliability combines loss, local relational stability across network depth, and dual-network prediction agreement; pseudo-target reliability is confidence-based. The interface preserves the base learner’s pseudo-target generator and is evaluated as a plug-in across several learners, synthetic noise, and real-noise datasets. Depth-wise diagnostics motivate separate assessment, and the experiments report both classification performance and pseudo-target quality.
Critical strengths
The weight b=(1−s_obs)s_pseudo combines correction need with replacement reliability and permits both signals to be unreliable. Weighting by a+b also avoids restoring full supervision after target normalization. This is a coherent interface improvement that does not require a new pseudo-label generator.
The paper reports paired gains for learners including DivideMix, RoLR, DISC, and ANNE, with extensions to CIFAR-N, WebVision, Food-101N, and Clothing1M. Table 5 includes a coupled-null control, not merely module removals, to separate observed-label refinement, pseudo-target gating, and sample weighting. The paper also discloses modest gains on difficult human noise and decreases in some global pseudo-target accuracies.
Critical weaknesses
-
In Table 5, Last accuracy is 74.71% for RoLR, 75.86% after observed-label refinement with complementary weights retained, 76.10% after adding the pseudo-target gate, and 76.22% for full TRACE. The gating increment is therefore 0.24 percentage points in this ablation; the full 1.51-point gain should not be attributed to decoupling. Tables 1–2 also mix cited and replicated results; the base rows used for paired attribution should be explicitly identified as matched reruns.
-
Appendix E.1 explains that Figure 1’s “Follow noisy” is computed only on genuinely corrupted observed labels, whereas “Pseudo correct” is the overall pseudo-target accuracy within the confidence stratum. Thus, 99.7% and 74.7% are not complementary quantities on the same population, and presenting their 25.0-point difference as a same-sample mismatch is potentially misleading. The low-clean+noisy subsets in Tables 6/17 are also selected separately by each method, confounding membership changes with target-quality changes. Global pseudo-target accuracy on CIFAR-100N decreases from 71.86% to 71.22%, so benefits are not uniform. The maximum-probability gate remains model-derived and cannot exclude confident errors. Common denominators, fixed evaluation sets, and explicit calibration definitions are needed to substantiate safer replacement supervision.
Questions for authors
Q1. Can you report seed-level paired differences and uncertainty for coupled null, decoupled reliability, and full TRACE across multiple existing noise settings? Please identify which base rows are matched reruns and which are contextual cross-paper references. Q2. Can Figure 1’s statistics be recomputed on the same corrupted-label population within common confidence strata, and can the methods be compared on a fixed shared low-clean set? Please report bin counts and denominators rather than directly contrasting percentages from different populations. Q3. How are calibration and high-confidence errors defined for the sharpened pseudo-target’s maximum probability? On a common set that actually requires replacement, does the gate consistently reduce confident errors relative to coupled null, particularly on CIFAR-100N?
Dear Reviewer G517, with the discussion period ending soon, we would appreciate it if you could take a look at our responses. If anything needs further clarification, we would be happy to continue the discussion. Thank you again for your time.