acceptodds
Under review as a conference paper at ICLR 2027

Reliable Replay through Spatial Coherence in Online Continual Learning

Abstract

Continually adapting models to new tasks requires retaining earlier knowledge under limited memory and computation. Experience replay addresses this challenge, but priorities based on individual loss increases overlook how related memories respond to the same update and can overemphasize isolated responses. We introduce SPatial coHErent risk control for REplay (SPHERE), a general replay-allocation method applicable across a broad range of learning settings. SPHERE uses a representation kernel to aggregate signed prospective loss changes, attenuating unsupported spikes while retaining coherent increases. It then formulates allocation as entropy-regularized transport, redistributing uniform source mass toward supported high-risk regions while penalizing long-distance transfers. We derive replay coefficients from the transport objective's sensitivity to the original loss changes and blend them with uniform replay to maintain baseline rehearsal. Our analysis establishes conditions under which kernel aggregation improves risk estimation and bounds transport-value inflation due to residual noise and smoothing bias. Experiments demonstrate that SPHERE improves accuracy and reduces forgetting across noisy-label vision tasks, continual language-model instruction tuning, and code-generation reinforcement learning with incomplete test rewards.

Primary area:
optimization
Keywords:
Continual learning, Catastrophic forgetting, Spatial coherence, Noisy supervision, Optimal transport

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_WpSoOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper claims that a single data point's isolated loss spike can come from noise and thus should not be the sole heuristic for replay. Built on this claim, the paper proposes a method called SPHERE that decides which memories get replayed and how much weight each one gets based on its neighboring loss landscape.

Critical strengths
  1. The motivation to disentangle label noise from real forgetting is intuitive.
  2. SPHERE is a simple plug-in method for example selection and does not interfere with the memory storage policy or training pipeline.
  3. SPHERE only uses information from a single sample, enabling online deployment.
  4. The evaluation is comprehensive across SFT for vision and language, and GRPO for coding.
Critical weaknesses
  1. The claim that corrupted labels appear as isolated loss spikes is not supported by experiments (or toy experiments).
  2. Suppose 1. is supported by experiments: SPHERE smooths out high-risk spikes; Corrupted labels produce such spikes. These two together do not imply SPHERE achieves better continual learning by reducing the risk of replaying corrupted supervision. Spikes can come from correctly labeled outliers or under-represented samples; smoothing them might just sacrifice the model's capability to fit the long tail and better fit the majority, resulting in better overall performance. A suggested experiment is to rerun the noisy-label experiments with an oracle buffer that contains only correctly labeled samples. If SPHERE’s gain comes from avoiding corrupted replay, it should largely vanish under this setting. If not, the gain should be attributed to additional factors.
  3. The gain attribution to spatial coherence is minor and loose. 3.1 SPHERE relies on mixing with standard ER, indicating the method itself is not stable and only works incrementally on standard ER. 3.2 Two baselines that use no neighborhood information (Loss-prop, MIR-prop) match SPHERE's performance in instruction tuning within an error margin. Shuffled neighborhoods match SPHERE's performance in code RL within an error margin.
Questions for authors
  1. Is spatial the correct term here? I assume spatial here means a high-dimensional embedding space, while my first impression of spatial is something related to real-world geometry.
  2. What fraction of replay falls on corrupted memories for ER, MIR-prop, and SPHERE at different noise levels?
  3. How well does SPHERE perform on more realistic corrupted labels such as CIFAR-100N rather than random noise?
2 comments
AuthorsOpenReview ↗

Reply to Reviewer WpSo (Part 1 of 2)

We sincerely thank the reviewer for the insightful questions and concrete experimental suggestions, particularly regarding the role of spatial coherence and corrupted supervision. We address the reviewer's question in detail below:

W1: The claim that corrupted labels appear as isolated loss spikes is not supported by experiments (or toy experiments).

A1: Thanks for the comment. We would like to clarify that we do not claim corrupted labels necessarily produce isolated loss spikes. However, label corruption can introduce misleading positive loss increments that may appear as isolated spikes when neighboring samples do not exhibit similar responses. As illustrated in Appendix B.1, a model update may increase the loss under a corrupted label while improving the prediction of the true class. This provides a concrete mechanism through which corrupted supervision can generate unreliable pointwise forgetting signals, rather than implying that all corrupted labels produce isolated spikes.

Importantly, SPHERE is not limited to noisy-label learning. Its central motivation is that neighborhood agreement in latent space provides additional evidence for assessing prospective forgetting risks, allowing unsupported spikes to be attenuated while preserving spatially coherent increases. Label corruption is one important setting where this mechanism can be beneficial, but it is neither a necessary condition nor the only source of unreliable loss increments.

W2: Suppose 1. is supported by experiments: SPHERE smooths out high-risk spikes; Corrupted labels produce such spikes. These two together do not imply SPHERE achieves better continual learning by reducing the risk of replaying corrupted supervision. Spikes can come from correctly labeled outliers or under-represented samples; smoothing them might just sacrifice the model's capability to fit the long tail and better fit the majority, resulting in better overall performance. A suggested experiment is to rerun the noisy-label experiments with an oracle buffer that contains only correctly labeled samples. If SPHERE's gain comes from avoiding corrupted replay, it should largely vanish under this setting. If not, the gain should be attributed to additional factors.

A2: Thank you for suggesting this experiment. We conducted the proposed oracle-buffer evaluation on CIFAR-100, where stream labels remain noisy but the replay buffer contains only correctly labeled samples. Results are averaged over 10 seeds paired with the original experiments:

NoiseBufferERER + SPHEREpaired winsER-ACEER-ACE + SPHEREpaired wins
40%standard10.212.29/1012.913.57/10
40%oracle15.616.38/1018.116.93/10
60%standard5.18.010/108.110.49/10
60%oracle13.414.79/1018.616.00/10

The oracle-buffer results directly address the reviewer's hypothesis: SPHERE's gains are substantially reduced when corrupted samples are excluded from replay. With oracle buffers, the improvements over ER become smaller and are no longer significant, while the ER-ACE integration underperforms its baseline. These findings support reducing corrupted replay as an important source of SPHERE's gains under noisy supervision.

Importantly, corruption is only one benefit of SPHERE's broader mechanism, rather than an assumption on which the method depends. SPHERE leverages neighborhood agreement to assess the reliability of prospective forgetting risks, without explicitly identifying corrupted labels. The clean-label ablation in Table 6 further supports the value of neighborhood information even without label corruption. We also acknowledge the potential trade-off highlighted by the reviewer: when replay labels are already clean, suppressing isolated responses may downweight genuinely difficult examples. Again, the performance gains should come from leveraging information from neighboring nodes rather than from handling corruption, as the main focus of our paper is not on addressing mislabels.

AuthorsOpenReview ↗

Reply to Reviewer WpSo (Part 2 of 2)

W3: The gain attribution to spatial coherence is minor and loose. 3.1 SPHERE relies on mixing with standard ER, indicating the method itself is not stable and only works incrementally on standard ER. 3.2 Two baselines that use no neighborhood information (Loss-prop, MIR-prop) match SPHERE's performance in instruction tuning within an error margin. Shuffled neighborhoods match SPHERE's performance in code RL within an error margin.

A3: We address these two points separately.

3.1. Mixing with ER. SPHERE does not require uniform blending to achieve improvements. Without the uniform floor, it still outperforms ER in instruction tuning (FP 0.673 vs. 0.661, minimum run 0.608 vs. 0.423) and improves over X-DER under 60% label noise (9.3 vs. 8.3). Thus, uniform blending is a design choice to improve robustness, rather than a requirement for SPHERE's effectiveness.

3.2. Spatial coherence and baseline comparisons. We acknowledge that Loss-prop and MIR-prop are competitive in instruction tuning, particularly with a small buffer. However, Loss-prop's only advantage over SPHERE is slightly lower tail forgetting (0.142 vs. 0.143), a difference of just 0.1 percentage points. SPHERE achieves better performance on the other reported metrics, including lower observed FP variability and higher minimum-run performance, with clearer gains at buffer size 3000 (FP 0.658 vs. 0.618). More importantly, controlled ablations directly support the contribution of spatial information. Removing the neighborhood block reduces final accuracy by 4.42 percentage points (Table 2), while permuting geometry reduces FP from .690 to .666 (Table 3). In Code RL, although permuted geometry achieves similar final accuracy, SPHERE improves worst-task accuracy (43.67 vs. 41.67) and BWT (−1.56 vs. −2.89), further supporting the benefit of meaningful geometric alignment.

Q1: Is spatial the correct term here? I assume spatial here means a high-dimensional embedding space, while my first impression of spatial is something related to real-world geometry.

A4: Thank you for pointing out this ambiguity. Yes, by "spatial" we refer to the geometry of the high-dimensional representation space, rather than physical space. Specifically, each memory is mapped to a representation hi=ϕ(xi)h_i=\phi(x_i), and the distance d(hi,hj)d(h_i,h_j) defines its neighborhood. Spatial coherence means that nearby memories tend to exhibit similar expected loss changes under the same model update, as formalized in Assumption 1.

This representation-space geometry is used for both neighborhood aggregation (Eq. 4) and distance-aware replay allocation (Eq. 7). We will clarify this definition when "spatial" first appears in the paper and explicitly refer to it as "representation-space coherence" where appropriate to avoid confusion.

Q2: What fraction of replay falls on corrupted memories for ER, MIR-prop, and SPHERE at different noise levels?

A5: Thank you for the question. We measured the fraction of replayed samples with corrupted labels, normalized by the corrupted fraction of the buffer. A value of 1.0 corresponds to uniform replay, while values below 1.0 indicate reduced replay of corrupted samples.

DatasetNoiseERMIR-propER + SPHEREER-ACEER-ACE + SPHERE
CIFAR-1020%1.000.970.671.000.54
CIFAR-1040%1.000.980.791.000.68
CIFAR-1060%1.000.990.901.000.87
CIFAR-10020%0.990.930.600.990.40
CIFAR-10040%1.000.930.651.000.40
CIFAR-10060%1.000.960.771.000.56

Across both datasets and all noise levels, SPHERE consistently reduces the replay share allocated to corrupted samples, by up to 60% relative to uniform replay, whereas MIR-prop shows only minor changes. The reduction becomes smaller at higher noise levels, suggesting that its effectiveness depends on the degree of corruption. These measurements provide direct evidence that SPHERE shifts replay allocation away from corrupted supervision, complementing the oracle-buffer findings in A2. We will add these results to the paper.

Q3: How well does SPHERE perform on more realistic corrupted labels such as CIFAR-100N rather than random noise?

A6: Thanks for the question. We additionally evaluated SPHERE on CIFAR-100N with the same configuration, the results shows SPHERE consistently improves all four evaluated replay hosts under realistic label noise.

MethodHost+ SPHERE
ER13.8 ± 1.315.3 ± 1.3
X-DER15.7 ± 6.116.9 ± 5.5
PuriDivER (buffer fitting)34.3 ± 1.336.0 ± 0.7
ER-ACE16.5 ± 4.418.4 ± 2.9
MIR-prop15.1 ± 1.0
DER++11.3 ± 4.6
GDumb (buffer fitting)18.0 ± 0.6
EWC / LwF (no memory)2.5 ± 0.9 / 7.1 ± 1.3
Reviewer_Fs3nOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper considers the (online) continual learning setting. Replay-based methods typically keep a buffer of old data which is sampled from and mixed with new data in order to reduce forgetting. Typically, the sampling is uniform. Instead, previous work tries to prioritize replaying the examples that will be forgotten the most. Given a batch of new data, they compute the gradient, and create a temporary copy of the model with parameters updated according to the computed gradient. This temporary model is used to predict the stored old examples, and the examples with the highest increase in loss (forgetting) are selected for replay. However, this is prone to outlier examples which have uncharacterstically high losses (e.g. mislabelled examples).

The paper suggests to prioritize neighbourhoods of examples which consistently demonstrate forgetting. To do this, they compute the distance between two inputs in feature space. An input’s forgetting score is then calculated as a weighted sum of its own forgetting and other examples’, weighted by the distance to said examples. This so far serves to average the forgetting scores within a neighbourhood but does not address far away outlier points. The authors use an optimal transport objective to reduce the probability of sampling such an outlier point, based on its distance from other examples. Finally, the resulting sampling probability is blended with a uniform distribution over the examples to ensure that each example has a chance of being replayed.

The authors present empirical results showing favourable performance across image classification, instruction tuning and code-generation.

Critical strengths

The approach is novel and well motivated.

Critical weaknesses

The demonstrated performance gains appear modest, which makes the additional method complexity harder to justify.

Questions for authors

Would it be possible to provide results for other baselines - e.g. well-tuned EWC, O-LoRA, LwF. It’d be helpful when assessing the method’s demonstrated advantage.

1 comment
AuthorsOpenReview ↗

Reply to Reviewer Fs3n

We thank the reviewer for the constructive feedback and for recognizing the potential value of our work. We discuss the new results as suggested by the reviewer and provide further clarifications in the following responses:

W1: The demonstrated performance gains appear modest, which makes the additional method complexity harder to justify.

A1: Thank you for the comment. We acknowledge that some improvements are modest, particularly under clean supervision and in the small-buffer setting, where SPHERE's advantages are less pronounced. We will make this clearer in the paper. However, the gains are substantial in several settings: code-generation accuracy improves from 42.75% to 49.08% over uniform replay (Table 7), and worst forgetting drops from 24.5% to 18.0% in instruction tuning (Table 1). The ablations also show that the added components matter: removing the neighborhood block lowers accuracy by 4.42 percentage points (Table 2), supporting its practical value.

Regarding computational complexity, the transport allocation admits a closed-form solution (Eq. 8), without requiring an iterative OT solve. Furthermore, we have added a runtime comparison in Appendix H.2, showing that SPHERE introduces little additional computational overhead for the evaluated moderate buffer sizes. These results suggest that SPHERE can provide meaningful improvements in accuracy and retention with limited additional runtime cost.

Q1: Would it be possible to provide results for other baselines - e.g. well-tuned EWC, O-LoRA, LwF. It'd be helpful when assessing the method's demonstrated advantage.

A2: Thank you for the suggestion. We have additionally evaluated EWC, LwF, and O-LoRA under our continual instruction-tuning and noisy-label CIFAR-100 settings. The results are summarized below.

Continual LLM instruction tuning. We evaluate the three additional baselines on the same 15-task benchmark, using 15 matched task order–seed runs for comparison. The results are:

MethodFP↑Worst Forget↓Tail Forget↓SPHERE better (paired)
EWC.630 ± .056.410.34314/15
LwF.340 ± .187.803.72415/15
O-LoRA.668 ± .018.351.28413/15
ER.661 ± .075.245.20015/15
SPHERE.690 ± .021.180.143—

SPHERE achieves the highest mean final performance and lowest worst and tail forgetting among these methods. In particular, it improves FP over EWC and O-LoRA by 6.0 and 2.2 percentage points, respectively. The paired comparisons further support the consistency of these improvements across task orders and seeds.

Noisy-label Classification. We additionally evaluate these baselines under clean and noisy supervision, using 10 seeds. Replay methods use a memory capacity of K = 2,000, while EWC, LwF, and O-LoRA do not use replay memory.

Cifar-10:

MethodClean20%40%60%
EWC25.0 ± 3.220.0 ± 2.816.0 ± 2.412.0 ± 1.8
LwF30.0 ± 4.124.0 ± 3.519.0 ± 2.714.0 ± 2.1
O-LoRA40.0 ± 3.632.0 ± 4.224.0 ± 3.118.0 ± 2.5
ER43.4 ± 5.733.4 ± 4.525.1 ± 2.917.4 ± 2.7
ER + SPHERE47.6 ± 3.835.8 ± 2.727.6 ± 1.619.9 ± 1.8

Cifar-100:

MethodClean20%40%60%
EWC4.2 ± 0.82.8 ± 0.92.1 ± 0.51.8 ± 0.5
LwF8.0 ± 1.08.6 ± 1.17.6 ± 1.26.2 ± 1.2
O-LoRA15.0 ± 2.18.8 ± 0.69.9 ± 1.34.9 ± 1.0
ER19.6 ± 3.414.3 ± 1.110.2 ± 1.15.1 ± 0.8
ER + SPHERE20.9 ± 1.815.7 ± 1.212.2 ± 0.88.0 ± 0.7

Due to the limited rebuttal time, the experiments on TinyImageNet are still running. We will include the complete results in the final version once all experiments are finished. Nevertheless, Nevertheless, based on the results obtained so far, we anticipate that the results will exhibit a similar overall trend.

Reviewer_ZS54OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies experience replay in online continual learning, where a model keeps learning new tasks from a stream and must not forget old ones. A common way to decide which stored examples to replay is to check how much each example’s loss would go up after the next update, and replay the ones that go up the most. The authors point out a problem with this: a single example can show a big loss increase for a bad reason, such as a wrong label or an incomplete test reward, and replaying it just reinforces the mistake. Their method, SPHERE, looks at the neighbors of each stored example in the model’s feature space. If an example’s loss goes up but its neighbors do not, the spike is treated as unreliable and is toned down. If a whole group of nearby examples goes up together, that region is treated as real forgetting and gets more replay. The replay weights are then spread out using a distance-penalized transport step and mixed with plain uniform replay so that every example still gets some rehearsal.

The authors test SPHERE in three settings: continual instruction tuning of T5-Large on 15 text classification tasks, image classification on CIFAR and TinyImageNet with noisy labels, and reinforcement learning for code generation with Qwen2.5-Coder where only one test case is visible during training. They report better final accuracy and less forgetting than uniform replay and loss-based replay baselines, with the largest gains under heavy label noise. They also give a theoretical analysis of when averaging over neighbors reduces noise more than it adds bias. The main cost is extra computation, since the method needs a virtual update, two forward passes over the stored memory, and pairwise distances between all stored examples at each rescoring step, which the paper acknowledges but does not measure.

Critical strengths

S1. Loss-based replay usually looks at each stored example on its own and asks how much its loss would go up after the next update. This paper adds the neighbors to that judgment. A single large increase can come from a wrong label or an incomplete reward. A moderate increase shared by several nearby examples is more likely to be real forgetting. Checking the neighbors before giving an example more replay is a simple and intuitive idea. The same view also covers label noise in vision and incomplete test rewards in code RL with one explanation.

S2. The method is general in practice, not only in claim. It needs two inputs from the host: a loss change per stored example and a distance between representations. The authors attach it to ER, ER-ACE, X-DER, and PuriDivER for images, to LoRA-based instruction tuning of T5, and to GRPO-style policy learning for code. The host keeps its own loss, memory policy, and optimizer. Per-domain choices of distance and bandwidth are expected for any replay method and do not weaken this point. I consider this breadth of integration a strength of the work.

Critical weaknesses

W1. The paper reports no cost. Each rescoring step needs a virtual update with a copy and restore of the parameters, two forward passes over the stored memory, a fresh computation of the features, and a full pairwise distance matrix. The conclusion mentions this overhead in one sentence, but there is no analysis in the paper. Without this, I cannot judge the practical value of the method, and the claim of broad applicability in the abstract remains untested in terms of cost.

W2. The paper does not show that spatial coherence is the reason for the gains. In Table 7, Permuted geometry and SPHERE reach almost the same final accuracy, and their curves in Figure 3 stay close. This control shuffles the features, so neighbors are no longer meaningful, yet the result barely changes. Most of the gain over Uniform replay must come from other parts of the pipeline. In Table 3, Full SPHERE and Consensus only differ by less than one standard deviation, so the transport step adds little. In Table 1 at the small buffer, Loss-prop matches SPHERE on final performance and has lower tail forgetting, but the text compares SPHERE only against ER and the MIR variants.

Questions for authors

Q1. In Table 4, adding SPHERE lowers accuracy on clean data for some hosts, while Table 6 claims that neighborhood information helps even without label noise. Why does this happen?

Q2. Is there any analysis of the time and memory cost of the rescoring step? How does SPHERE compare with ER when ER is given the same extra compute?

Q3. Is there any result with features computed once and cached, instead of being recomputed at every rescoring step? This would matter for use on devices with limited compute.

Q4. Is there a single default rule for the distance, bandwidth, and temperature that a new user can follow? The three domains in the paper each use a different rule.

3 comments
AuthorsOpenReview ↗

Reply to Reviewer ZS54 (Part 1 of 3)

We sincerely thank the reviewer for the careful and detailed assessment of our work. Our point-by-point responses are provided below:

W1. The paper reports no cost. Each rescoring step needs a virtual update with a copy and restore of the parameters, two forward passes over the stored memory, a fresh computation of the features, and a full pairwise distance matrix. The conclusion mentions this overhead in one sentence, but there is no analysis in the paper. Without this, I cannot judge the practical value of the method, and the claim of broad applicability in the abstract remains untested in terms of cost.

A1: Thank you for pointing out the missing cost analysis. We have added a runtime comparison across buffer sizes and three replay hosts (ER, ER-ACE, and X-DER), together with a detailed discussion in Appendix H.2.

SPHERE shares the virtual update and memory-scoring operations with interference-based retrieval methods such as MIR. Its additional neighborhood aggregation and transport allocation are computed directly, as Eq. (8) admits a closed-form solution without iterative Sinkhorn optimization. The main scaling cost comes from scoring more memories and constructing the pairwise distance and allocation matrices.

As shown in the new Figure 4, SPHERE introduces limited additional runtime for small and moderate buffer sizes, while full-buffer scoring becomes increasingly expensive for large buffers. Importantly, restricting the scored candidate pool to 512 substantially reduces this overhead, even with a buffer of 4,000 samples. We will include these results and explicitly discuss the trade-off between computational cost and scored-set size in the revision.

AuthorsOpenReview ↗

Reply to Reviewer ZS54 (Part 2 of 3)

W2. The paper does not show that spatial coherence is the reason for the gains. In Table 7, Permuted geometry and SPHERE reach almost the same final accuracy, and their curves in Figure 3 stay close. This control shuffles the features, so neighbors are no longer meaningful, yet the result barely changes. Most of the gain over Uniform replay must come from other parts of the pipeline. In Table 3, Full SPHERE and Consensus only differ by less than one standard deviation, so the transport step adds little. In Table 1 at the small buffer, Loss-prop matches SPHERE on final performance and has lower tail forgetting, but the text compares SPHERE only against ER and the MIR variants.

A2: Thank you for highlighting the need to disentangle spatial coherence from other components. We would like to argue that the existing experiments already provide direct evidence that spatial information contributes to SPHERE's gains. We clarify what each comparison establishes and provide additional results to further strengthen this evidence.

First, Table 2 compares full SPHERE with S=IS = I, jointly removing neighborhood aggregation and its associated pullback while retaining the transport distance cost. Full SPHERE improves final accuracy by 4.42 percentage points and reduces worst forgetting by 7.90 points. Complementarily, the permutation control in Table 3 retains the allocation procedure but breaks feature–memory alignment: full SPHERE improves FP from 0.666 to 0.690 and reduces worst/tail forgetting from 0.213 and 0.188 to 0.180 and 0.143. These comparisons provide evidence for the neighborhood block and meaningful alignment, respectively; they do not assign additive shares of the overall gain to individual components.

We address the reviewer's specific concerns below.

Code RL (Table 7, Figure 3). We acknowledge that Full SPHERE and Permuted geometry have close final accuracy, but still, SPHERE's 49.08% is better than 48.58%. However, permutation breaks feature–memory alignment while retaining smoothing, transport, and uniform blending. It does not remove the remaining replay-allocation mechanism. Aligned geometry also improves worst-task accuracy and BWT. We have additionally updated Table 7 and Figure 3 to include Consensus only, which achieves 45.83% final accuracy compared with SPHERE's 49.08%, supporting the benefit of transport over this alternative score-based allocation. Both outperform No Transport's 42.67%, although that comparison changes multiple components and therefore does not isolate the contribution of geometry or transport alone. We will clarify these distinctions and avoid attributing all gains to spatial alignment.

Consensus only vs. Full SPHERE (Table 3). Consensus only retains SS, the S⊤S^\top pullback, positive mask, and uniform mixture, replacing transport with an ESS-matched softmax. Its proximity to Full SPHERE indicates that neighborhood-based aggregation accounts for much of the improvement in instruction tuning, while transport provides a smaller incremental benefit. Importantly, No transport also removes the pullback and is not an OT-only control. Since runs are matched by task order and seed (in Appendix H.3), the reported marginal SDs do not directly quantify uncertainty in paired differences. We also examined the paired differences across the 15 matched order–seed runs and found that SPHERE achieves higher FP in 14 of the 15 pairs. This indicates that the modest average improvement is consistent across matched runs rather than being driven by a few favorable outcomes.

Loss-prop (Table 1). At buffer size 300, Loss-prop is competitive with SPHERE in mean FP (0.689 vs. 0.690), and its only advantage among the reported metrics is slightly lower tail forgetting (0.142 vs. 0.143), a difference of just 0.1 percentage points. SPHERE achieves lower worst forgetting (0.180 vs. 0.188), lower observed FP variability (0.021 vs. 0.045 SD), and higher minimum observed run performance (0.661 vs. 0.538). At buffer size 3000, SPHERE’s advantage over Loss-prop is clearer, with higher FP (0.658 vs. 0.618) and lower tail forgetting (0.228 vs. 0.269). Compared with ER and both MIR variants, SPHERE consistently achieves higher mean FP and lower mean worst and tail forgetting at both buffer sizes. We will expand the discussion to explicitly acknowledge Loss-prop’s competitive small-buffer results.

Together, these results support contributions from neighborhood information and transport, with their observed benefits differing across settings. We will revise the discussion to distinguish these contributions and avoid attributing all improvements to spatial alignment alone.

AuthorsOpenReview ↗

Reply to Reviewer ZS54 (Part 3 of 3)

Q1. In Table 4, adding SPHERE lowers accuracy on clean data for some hosts, while Table 6 claims that neighborhood information helps even without label noise. Why does this happen?

A3: The two tables answer different questions, so they do not contradict each other.

  • Table 4 compares a tuned host with and without the whole SPHERE allocation. It asks whether our reliable replay beats the host's uniform draw.
  • Table 6 compares full SPHERE with its S=IS=I variant, removing signed neighborhood aggregation and its associated weight pullback while retaining transport and the other components.ores.

Thus, neighborhood information can improve SPHERE’s allocation without the complete allocation rule outperforming the original host in every setting. Clean labels remove label corruption, but do not eliminate interference between tasks or variation in prospective loss changes. Neighborhood aggregation can therefore remain useful, although its benefit depends on whether variance reduction outweighs smoothing bias, as characterized in Proposition 1. Reallocating replay can also change the balance between learning current examples and retaining earlier ones; clean-label accuracy is consequently not guaranteed to improve for every host.

The broader clean-data results are nevertheless predominantly positive: across Tables 4, 11, and 12, SPHERE improves mean accuracy in most host–dataset comparisons, including all three hosts on clean CIFAR-10. The decreases range from 0.5 to 1.7 percentage points. Table 6 provides complementary evidence that neighborhood information can help even without label corruption, rather than establishing universal gains over the underlying host. We will clarify this distinction in the discussion.

Q2. Is there any analysis of the time and memory cost of the rescoring step? How does SPHERE compare with ER when ER is given the same extra compute?

A4: Thank you for the question. We provide a detailed runtime and memory analysis in A1 and will include these results in Appendix H.2. The experiments show that SPHERE incurs limited overhead at moderate scored-set sizes, while restricting the candidate pool effectively controls the cost for larger buffers.

Regarding compute-matched ER, additional computation could instead be used for more replay updates, which we have not evaluated as a strictly matched-budget baseline. We therefore also compare against MIR-prop, which shares SPHERE's interference-scoring operations. At K = 2,000, SPHERE with a 512-candidate pool achieves higher accuracy (12.7% vs. 11.0%) with comparable runtime (2.9 vs. 2.5 minutes), supporting the practical benefit of its additional allocation mechanism.

Q3. Is there any result with features computed once and cached, instead of being recomputed at every rescoring step? This would matter for use on devices with limited compute.

A5: Yes. Our code-RL experiment uses exactly this setting and are shown in Table 7. Our prompt embeddings are extracted once from the frozen base model before training and held fixed throughout all 120 updates (in Appendix H.5.2). Under this setting, SPHERE achieves 49.08% final accuracy versus 42.75% for Uniform replay, while considering at most 48 candidates per update from a 480-problem buffer. Thus, rescoring updates the loss-change signals and replay allocation without recomputing the embeddings. In vision, we instead refresh features from the evolving model at each rescoring step. Caching would reduce feature-extraction cost at the expense of representation freshness.

Q4. Is there a single default rule for the distance, bandwidth, and temperature that a new user can follow? The three domains in the paper each use a different rule.

A6: Yes. However, since the experimental settings differ a lot, some aspects are common across them, while others need to be determined individually. We discuss these factors below:

Shared recipe. Across all three domains, we normalize representations to unit length, construct neighborhoods using an exponential distance kernel, and scale the transport temperature according to the magnitude of positive loss increments. All full configurations also use the same uniform-mixture coefficient, α=0.5\alpha=0.5. Some distance choices are equivalent up to scaling: for unit-normalized representations, squared Euclidean distance equals twice the cosine distance.

Domain-specific choices. Representation extraction and the exact distance scaling, bandwidth, and temperature calibration differ across domains to accommodate different feature distributions and training losses. We therefore provide a common construction rather than a single numerical default validated across all settings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.