acceptodds
Under review as a conference paper at ICLR 2027

Beyond Class Marginals: Bounding Rehearsal Gaps without Freezing Class Co-occurrence

Abstract

Class-balanced replay controls class frequency but does not determine the interval between successive replay appearances of a class. We study this interval, the rehearsal gap, separately from the class marginal and class co-occurrence, and introduce randomised-pass replay (RPR), which visits each resident class once per shuffled pass. For a fixed set of resident classes and replay batch size , RPR preserves the balanced time-averaged class marginal and bounds every gap by ; a churn-conditional bound applies while the resident set changes. The scheduler uses no future class information and adds no replay examples or forward passes. In a linear-head ER-ACE diagnostic, joint absence from the incoming and replay batches produces a one-sided classifier-bias gradient. Longer absence episodes are associated with larger negative bias displacement, and removing the incoming-loss mask attenuates the scheduling effect. In the primary ER-ACE experiments, RPR improves final average accuracy by – percentage points relative to independent class-balanced retrieval under reservoir storage, with positive effects also observed under balanced storage. Pretrained ViTs show positive effects on the tested LT10 streams with small replay batches, while matched larger-batch controls show no material effect. Fixed-cycle and reused-pass controls change more than one temporal statistic, so the experiments do not isolate rehearsal-gap length from all other forms of temporal dependence. The accuracy effects depend on the learner and operating regime.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
online continual learning, experience replay, class-balanced replay, rehearsal scheduling, rehearsal gaps, randomized-pass replay, class co-occurrence, long-tailed learning, replay buffer, catastrophic forgetting

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_EzZUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposed Randomised-Pass Replay (RPR) for reducing the rehearsal gap in continual learning. Experimental results show the effectiveness of the proposed RPR compared with i.i.d and fixed cycle though the improvement depends on batchsize and buffer capacity.

Critical strengths

The concept of rehearsal gap is interesting. Theoretical definition of upper bound of rehearsal gap is also the strength of this paper.

Critical weaknesses

Experimental results are insufficient.

1-1. The authors discuss only the relation between batchsize and the number of class. However, there is the relation among the number of class, batchsize, and buffer capacity. It would be better to include the analysis about it.

1-2. The dataset you used is too small to show the effectiveness of your method in real situation. In real applications, the number of class is large, the buffer capacity and batchsize are small. Will be the proposed RPR effective in the real situation? The authors should evaluate your method on larger datasets such as iNaturalist.

1-3. Table 3 shows that the proposed method is worse than the baseline when b=32. There is no discussion and analysis about it. The authors should clarify the reason why the proposed RPR decreases the accuracy.

1-4. Similary with 1-3, there is no ablation study regarding the batchsize and buffer capacity. For example, in Table 3, the authors evaluate the result only for buffer capacity of 500 when b=32. The authors show the accuracy while changing batchsize and buffer capacity, and clarify the limitation.

1-5. The experimental setting in Table 1 remains unclear. Implementation details noted that buffer capacities range is from 200 to 5,120. The authors should clarify buffer capacity of each dataset.

This paper is hard to read and interpret.

2-1. There is little discussion and analysis regarding experimental results.

2-2. There is little explanation of Figures and Tables.

2-3. This paper refers to the supplementary material as if they were part of the main text. The main manuscript should be self-contained and understandable without the supplementary material.

Questions for authors

Please respond to the Weakness I pointed out.

Reviewer_KgY7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the temporal spacing of replayed classes, termed the rehearsal gap, in online continual learning. The authors argue that while standard class-balanced replay guarantees long-term class marginal frequencies, it suffers from an unbounded geometric tail in the intervals between visits of the same class. To mitigate this without freezing class co-occurrence (as done in fixed class-cycling), the authors propose Randomised-Pass Replay (RPR). RPR maintains a persistent deck of resident classes and samples without replacement across shuffled passes, guaranteeing a worst-case gap bound of 2[C/b]−1 on a fixed resident set. The paper provides a mathematical diagnostic for linear-head ER-ACE showing that joint class absence produces one-sided negative classifier bias drift, and reports experiments on CIFAR-100, Tiny-ImageNet, and ImageNet-R across various replay budgets and loss functions.

Critical strengths
  1. Clear problem definitions: The paper clearly decouples three distinct properties of replay schedules: class marginals, rehearsal gap distribution, and class co-occurrence.
  2. Rigorous combinatorial proofs: The derivation of the fixed-set and churn-conditional bounds (Propositions 1–3) is mathematically complete and clean.
Critical weaknesses
  1. Limited novelty. The proposed method, RPR, is fundamentally standard random reshuffling (sampling without replacement per epoch/pass) applied to a buffer of class IDs. Random reshuffling is a textbook heuristic in stochastic optimization. Porting it to order a list of resident classes in an experience replay buffer does not represent a significant conceptual or algorithmic advance for an ICLR contribution.
  2. RPR is sentitive to hyperparameters and losses. In Table 3, RPR only produces positive differences when the replay batch size is tiny (b=8). When scaled to a modest batch size (b=32), RPR exhibits negative or null effects. In Table 12 and Section 5.2, when evaluating common OCL objectives other than masked ER-ACE (e.g., DER, DER++, OCM), RPR yields null, dataset-dependent, or contradictory results.
Questions for authors
  1. Across the ViT experiments (Table 13) and several ResNet settings (Table 6), RPR shows no significant improvement over deterministic fixed class-cycling. What empirical evidence directly demonstrates that "resampling class co-occurrence" is actually beneficial to continual representation learning?
  2. What is the formal mathematical mechanism explaining why bounding rehearsal gaps improves performance on cosine classifiers?
Reviewer_yRZ2OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the order in which stored classes are revisited during experience replay in online continual learning. Sampling classes independently at each step can leave a class unvisited for a long stretch, while cycling in a fixed order bounds the wait time but freezes which classes share a batch. The authors keep a shuffled deck of the classes currently in memory, pop classes without replacement to build each replay batch, reshuffle when the deck empties, and insert or drop classes as memory changes. They connect long replay gaps to classifier-bias drift under ER-ACE and test the scheduler across reservoir and balanced buffers, long-tailed streams, other replay losses, and a pretrained ViT.

Critical strengths
  • Separating replay frequency, wait time between visits, and batch pairings is a clean way to frame retrieval, and crossing storage rules with retrieval schedules nicely separates what is in the buffer from when it is replayed.
  • I appreciate the candidness about the ViT runs, other replay losses, and larger replay batches come back negative or inconclusive rather than overselling the method.
  • The ER-ACE mask ablation is a good diagnostic. Turning off the incoming mask removes most of the scheduling gain in the linear-head runs.
Critical weaknesses
  • Keeping a shuffled pass over stored classes is a very small change (the authors note shuffled passes are standard), and the propositions are mostly short counting and pigeonhole arguments plus Jensen's inequality. That puts the weight on how general the gains are, but outside ER-ACE the evidence is thin: standard DER++ gains little unless the ER-ACE mask is added, DER, DER++/Refresh, and OCM give inconclusive or dataset-dependent results, and the RPR numbers for DER and DER++/Refresh aren't reported.
  • The mechanism section doesn't explain the main results. The authors say the bias-drift argument only covers the linear-head diagnostic, and the gain is no smaller with the bias-free cosine head used in the main runs than with the linear head at the same buffer and replay batch, so the bias itself doesn't seem to be what matters. Measuring weight-direction or feature drift in the cosine head would be more convincing than showing that a bias pushed down at every step drifts further over longer gaps, which is largely expected from the sign of the gradient.
  • What separates RPR from fixed cycling is that it renews which classes share a batch, but the evidence that this helps accuracy is weak, as the authors partly acknowledge. RPR beats class-ID cycling in most ResNet runs, yet the ViT comparisons are mostly inconclusive, and reusing one shuffled pass for many traversals does about as well as reshuffling every pass. Comparing against a single fixed random permutation would show whether part of the ResNet gap comes from cycling in class-ID order rather than from frozen pairings.
  • The headline gains also come from a fairly narrow regime. Final accuracies on the long-tailed streams are around ten percent, and the buffer behind the main table, only given in the appendix, holds a quarter to two-fifths of the CIFAR-100 long-tailed streams. In the appendix buffer sweep, which uses the linear head, the gain is small or not significant at the two smallest buffers, and on the ViT clear gains only appear on long-tailed streams with the smallest replay batch.
  • The authors are upfront that matched class frequencies aren't established once memory changes, but per-class replay counts are never reported. A newly stored class is inserted into the rest of the current pass, so it gets an early visit while its task still dominates the incoming stream, and reporting these counts would show whether the gains come from shorter gaps or from shifted replay frequency.
Questions for authors
  • How does a single fixed random class permutation compare with class-ID cycling and per-pass reshuffling on the ResNet?
  • What are the per-class replay counts for head, mid, and tail classes under RPR and the independent balanced draw in the online runs?
  • How much do the cosine-head class weights rotate during long replay gaps?
  • Does the gain hold on from-scratch ER-ACE with more steps per incoming batch, where baseline accuracies are closer to usual levels?

Minor

  • The buffer size for the main table isn't given in its caption or the main text, and the appendix MIR table doesn't say which stream it uses.
  • The CIFAR-10 control is mentioned in the setup and conclusion, but its numbers never appear.

Comments on the paper

1 comment

Dear Chairs and Reviewers,

Thank you for the time and care you have devoted to our submission. To respect your time, we briefly summarise the main concerns, the additional evidence, and its implications.

  • Contribution beyond reshuffling and fixed cycling (KgY7, yRZ2). We acknowledge that reshuffling is standard and clarify the replay-specific theoretical contribution. To test whether the advantage over numeric cycling merely reflects class-ID ordering, we added a permanent random-order comparator. RPR gains 0.527 percentage points across 30 paired seeds, providing evidence beyond the numeric-order comparison, although this does not isolate the benefit of renewed pairings.

  • Replay-frequency confounding (yRZ2). We report head/mid/tail replay counts and acknowledge that online counts are not exactly matched. An additional control fixes per-class replay counts, pair co-occurrence totals, and buffer-label trajectories. RPR order gains 0.367 points over permuted batches, supporting an effect of temporal order after matching these totals, without isolating gap length from other temporal dependencies.

  • Mechanism for cosine classifiers (KgY7, yRZ2). We added a conditional margin-stability analysis and direct weight-rotation and signed-margin measurements. Longer absence is associated with greater margin degradation, but RPR does not uniformly reduce rotation or aggregate episode-level margin loss. These diagnostics examine the proposed explanation directly; they do not establish causal mediation of the final accuracy gains.

  • Memory, budget, scale, and generality (EzZU, yRZ2, KgY7). A matched memory-by-batch experiment gives +1.060 points at memory 500/replay 8, addressing dependence on the large primary buffer; replay-32 effects remain inconclusive. With three updates per incoming batch, RPR retains a +0.624-point advantage over independent retrieval. Following EzZU’s request, a naturally imbalanced 1,000-species iNaturalist evaluation gives +0.960 points with memory 500, extending the evidence to a larger vocabulary. These exploratory results broaden the evaluation while retaining the negative and inconclusive findings across losses and backbones.

We also clarified experimental settings and expanded the main-text explanations, addressing EzZU’s and yRZ2’s presentation concerns. Additional comparisons are labelled exploratory with unadjusted paired intervals.

Thank you again for your constructive feedback, which has helped us strengthen the evaluation and state the contribution and limitations more precisely.

Kind regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.