- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper investigates the relationship between memorization and generalization in networks trained with noisy labels. Using the Topo^2 framework, the authors interpret within-class and cross-class topological structures as generalization and memorization channels. They introduce overlay and stripping interventions and propose a quantitative relationship between memorization and validation-accuracy loss, with a coefficient near 0.38 in several reference settings
Critical strengths
- The paper addresses an interesting question: how much generalization is lost when a network fits incorrect labels.
- The FM0 reference and controlled exposure to flipped samples provide a useful experimental setting for investigating this trade-off.
- The authors report multi-seed calibration and explicitly acknowledge limitations involving model capacity, noise structure, and noise range.
Critical weaknesses
- The claim of causal separability is not sufficiently established. Section 3.4 states that the generalization channel is frozen, but does not specify what is actually fixed. Continuing training with noisy samples does not itself demonstrate selective intervention on an independent memorization channel.
- The topological channels lack precise operational definitions. It is unclear how within-class and cross-class are distinguished, and how the interpolation in Section 3.2 is performed and motivated. These details are essential for reproducing and interpreting the central measurements.
- The proposed functional form is not convincingly justified. Section 4.3 reports that alternative forms fit similarly, while Section 5.2 provides only a heuristic explanation for . This is insufficient to establish the specific equation as a quantitative causal law
- The experimental scope is limited relative to the generality claimed. Results mainly concern CIFAR/SVHN settings, and matched parameter counts still yield substantially different coefficients across architectures. Broader validation across dataset scales and model families is needed.
- The claim that memory is “fully invertible” (Line 312) is stronger than the recovery results support. After stripping, memorization remains at 0.11 and accuracy reaches 0.92 versus the FM0 reference of 0.96, demonstrating substantial recovery rather than complete inversion
Questions for authors
See weaknesses.