- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper claims that a single data point's isolated loss spike can come from noise and thus should not be the sole heuristic for replay. Built on this claim, the paper proposes a method called SPHERE that decides which memories get replayed and how much weight each one gets based on its neighboring loss landscape.
Critical strengths
- The motivation to disentangle label noise from real forgetting is intuitive.
- SPHERE is a simple plug-in method for example selection and does not interfere with the memory storage policy or training pipeline.
- SPHERE only uses information from a single sample, enabling online deployment.
- The evaluation is comprehensive across SFT for vision and language, and GRPO for coding.
Critical weaknesses
- The claim that corrupted labels appear as isolated loss spikes is not supported by experiments (or toy experiments).
- Suppose 1. is supported by experiments: SPHERE smooths out high-risk spikes; Corrupted labels produce such spikes. These two together do not imply SPHERE achieves better continual learning by reducing the risk of replaying corrupted supervision. Spikes can come from correctly labeled outliers or under-represented samples; smoothing them might just sacrifice the model's capability to fit the long tail and better fit the majority, resulting in better overall performance. A suggested experiment is to rerun the noisy-label experiments with an oracle buffer that contains only correctly labeled samples. If SPHERE’s gain comes from avoiding corrupted replay, it should largely vanish under this setting. If not, the gain should be attributed to additional factors.
- The gain attribution to spatial coherence is minor and loose. 3.1 SPHERE relies on mixing with standard ER, indicating the method itself is not stable and only works incrementally on standard ER. 3.2 Two baselines that use no neighborhood information (Loss-prop, MIR-prop) match SPHERE's performance in instruction tuning within an error margin. Shuffled neighborhoods match SPHERE's performance in code RL within an error margin.
Questions for authors
- Is spatial the correct term here? I assume spatial here means a high-dimensional embedding space, while my first impression of spatial is something related to real-world geometry.
- What fraction of replay falls on corrupted memories for ER, MIR-prop, and SPHERE at different noise levels?
- How well does SPHERE perform on more realistic corrupted labels such as CIFAR-100N rather than random noise?
2 comments
Reply to Reviewer WpSo (Part 2 of 2)
W3: The gain attribution to spatial coherence is minor and loose. 3.1 SPHERE relies on mixing with standard ER, indicating the method itself is not stable and only works incrementally on standard ER. 3.2 Two baselines that use no neighborhood information (Loss-prop, MIR-prop) match SPHERE's performance in instruction tuning within an error margin. Shuffled neighborhoods match SPHERE's performance in code RL within an error margin.
A3: We address these two points separately.
3.1. Mixing with ER. SPHERE does not require uniform blending to achieve improvements. Without the uniform floor, it still outperforms ER in instruction tuning (FP 0.673 vs. 0.661, minimum run 0.608 vs. 0.423) and improves over X-DER under 60% label noise (9.3 vs. 8.3). Thus, uniform blending is a design choice to improve robustness, rather than a requirement for SPHERE's effectiveness.
3.2. Spatial coherence and baseline comparisons. We acknowledge that Loss-prop and MIR-prop are competitive in instruction tuning, particularly with a small buffer. However, Loss-prop's only advantage over SPHERE is slightly lower tail forgetting (0.142 vs. 0.143), a difference of just 0.1 percentage points. SPHERE achieves better performance on the other reported metrics, including lower observed FP variability and higher minimum-run performance, with clearer gains at buffer size 3000 (FP 0.658 vs. 0.618). More importantly, controlled ablations directly support the contribution of spatial information. Removing the neighborhood block reduces final accuracy by 4.42 percentage points (Table 2), while permuting geometry reduces FP from .690 to .666 (Table 3). In Code RL, although permuted geometry achieves similar final accuracy, SPHERE improves worst-task accuracy (43.67 vs. 41.67) and BWT (−1.56 vs. −2.89), further supporting the benefit of meaningful geometric alignment.
Q1: Is spatial the correct term here? I assume spatial here means a high-dimensional embedding space, while my first impression of spatial is something related to real-world geometry.
A4: Thank you for pointing out this ambiguity. Yes, by "spatial" we refer to the geometry of the high-dimensional representation space, rather than physical space. Specifically, each memory is mapped to a representation , and the distance defines its neighborhood. Spatial coherence means that nearby memories tend to exhibit similar expected loss changes under the same model update, as formalized in Assumption 1.
This representation-space geometry is used for both neighborhood aggregation (Eq. 4) and distance-aware replay allocation (Eq. 7). We will clarify this definition when "spatial" first appears in the paper and explicitly refer to it as "representation-space coherence" where appropriate to avoid confusion.
Q2: What fraction of replay falls on corrupted memories for ER, MIR-prop, and SPHERE at different noise levels?
A5: Thank you for the question. We measured the fraction of replayed samples with corrupted labels, normalized by the corrupted fraction of the buffer. A value of 1.0 corresponds to uniform replay, while values below 1.0 indicate reduced replay of corrupted samples.
| Dataset | Noise | ER | MIR-prop | ER + SPHERE | ER-ACE | ER-ACE + SPHERE |
|---|---|---|---|---|---|---|
| CIFAR-10 | 20% | 1.00 | 0.97 | 0.67 | 1.00 | 0.54 |
| CIFAR-10 | 40% | 1.00 | 0.98 | 0.79 | 1.00 | 0.68 |
| CIFAR-10 | 60% | 1.00 | 0.99 | 0.90 | 1.00 | 0.87 |
| CIFAR-100 | 20% | 0.99 | 0.93 | 0.60 | 0.99 | 0.40 |
| CIFAR-100 | 40% | 1.00 | 0.93 | 0.65 | 1.00 | 0.40 |
| CIFAR-100 | 60% | 1.00 | 0.96 | 0.77 | 1.00 | 0.56 |
Across both datasets and all noise levels, SPHERE consistently reduces the replay share allocated to corrupted samples, by up to 60% relative to uniform replay, whereas MIR-prop shows only minor changes. The reduction becomes smaller at higher noise levels, suggesting that its effectiveness depends on the degree of corruption. These measurements provide direct evidence that SPHERE shifts replay allocation away from corrupted supervision, complementing the oracle-buffer findings in A2. We will add these results to the paper.
Q3: How well does SPHERE perform on more realistic corrupted labels such as CIFAR-100N rather than random noise?
A6: Thanks for the question. We additionally evaluated SPHERE on CIFAR-100N with the same configuration, the results shows SPHERE consistently improves all four evaluated replay hosts under realistic label noise.
| Method | Host | + SPHERE |
|---|---|---|
| ER | 13.8 ± 1.3 | 15.3 ± 1.3 |
| X-DER | 15.7 ± 6.1 | 16.9 ± 5.5 |
| PuriDivER (buffer fitting) | 34.3 ± 1.3 | 36.0 ± 0.7 |
| ER-ACE | 16.5 ± 4.4 | 18.4 ± 2.9 |
| MIR-prop | 15.1 ± 1.0 | |
| DER++ | 11.3 ± 4.6 | |
| GDumb (buffer fitting) | 18.0 ± 0.6 | |
| EWC / LwF (no memory) | 2.5 ± 0.9 / 7.1 ± 1.3 |
Reply to Reviewer WpSo (Part 1 of 2)
We sincerely thank the reviewer for the insightful questions and concrete experimental suggestions, particularly regarding the role of spatial coherence and corrupted supervision. We address the reviewer's question in detail below:
A1: Thanks for the comment. We would like to clarify that we do not claim corrupted labels necessarily produce isolated loss spikes. However, label corruption can introduce misleading positive loss increments that may appear as isolated spikes when neighboring samples do not exhibit similar responses. As illustrated in Appendix B.1, a model update may increase the loss under a corrupted label while improving the prediction of the true class. This provides a concrete mechanism through which corrupted supervision can generate unreliable pointwise forgetting signals, rather than implying that all corrupted labels produce isolated spikes.
Importantly, SPHERE is not limited to noisy-label learning. Its central motivation is that neighborhood agreement in latent space provides additional evidence for assessing prospective forgetting risks, allowing unsupported spikes to be attenuated while preserving spatially coherent increases. Label corruption is one important setting where this mechanism can be beneficial, but it is neither a necessary condition nor the only source of unreliable loss increments.
A2: Thank you for suggesting this experiment. We conducted the proposed oracle-buffer evaluation on CIFAR-100, where stream labels remain noisy but the replay buffer contains only correctly labeled samples. Results are averaged over 10 seeds paired with the original experiments:
The oracle-buffer results directly address the reviewer's hypothesis: SPHERE's gains are substantially reduced when corrupted samples are excluded from replay. With oracle buffers, the improvements over ER become smaller and are no longer significant, while the ER-ACE integration underperforms its baseline. These findings support reducing corrupted replay as an important source of SPHERE's gains under noisy supervision.
Importantly, corruption is only one benefit of SPHERE's broader mechanism, rather than an assumption on which the method depends. SPHERE leverages neighborhood agreement to assess the reliability of prospective forgetting risks, without explicitly identifying corrupted labels. The clean-label ablation in Table 6 further supports the value of neighborhood information even without label corruption. We also acknowledge the potential trade-off highlighted by the reviewer: when replay labels are already clean, suppressing isolated responses may downweight genuinely difficult examples. Again, the performance gains should come from leveraging information from neighboring nodes rather than from handling corruption, as the main focus of our paper is not on addressing mislabels.