acceptodds
Under review as a conference paper at ICLR 2027

Distribution Matching Guided Replay Sample Selection for Online Class-Incremental Learning

Abstract

Online Class-Incremental Learning (OCIL) focuses on learning from continuously evolving data streams that are ubiquitous in dynamic environments. Data replay is an effective OCIL method, which maintains a replay buffer and learns from a mixture of replay samples and new samples to mitigate catastrophic forgetting. Previous studies have proposed various strategies for selecting replay samples; however, they often suffer from limited data availability and error accumulation, and only achieve suboptimal performance in practice. To this end, we propose a new replay sample selection strategy called Distribution Matching Replay (DMR). By continuously aligning the distribution of replay samples with that of all historical data, DMR significantly alleviates the issues of data availability and error accumulation. To make second-order distribution matching memory-efficient in the streaming setting, we further introduce a randomized sketch of the RFF covariance statistics, which approximates Maximum Covariance Discrepancy (MCD) without explicitly storing per-class covariance matrices. Extensive experiments on multiple benchmarks demonstrate that DMR outperforms state-of-the-art selection strategies and can be seamlessly integrated into various OCIL methods to further improve their performance.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Continual Learning, Online Class-Incremental Learning, Replay Sample Selection, Maximum Mean Discrepancy, Random Fourier Features, Randomized Sketching

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Ms92OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Distribution Matching Replay (DMR) for task-agnostic, single-pass online class-incremental learning. DMR maintains class-wise statistics of all observed samples in a fixed random Fourier feature (RFF) space and scores removals and insertions by their effects on mean and covariance discrepancies between the buffer and historical summaries. Generally, DMR achieves satisfactory performance on some small-scale and middle-scale datasets.

Critical strengths
  1. The problem studied is a very important topic: a small replay buffer is an imperfect proxy for the historical stream, and retaining compact statistics of discarded samples is a sensible way to improve selection under a single-pass constraint. The proposed method shows that more informative statistics about historical examples can be preserved.

  2. The shared, fixed RFF map and additive statistics provide a coherent mechanism for historical distribution matching without retaining all past examples. Appendix D appropriately distinguishes sketch approximation from the separate RFF approximation and acknowledges adaptive-selection limitations.

Critical weaknesses
  1. It seems that the proposed method can also address many typical problems of learning from streaming data. Therefore, can this method be applied to concept-drifted (or distributionally drifting) streaming data? If the streaming data has significant distribution shift, can this method still have satisfactory performance? This experiment can make the proposed method more general for online learning.

  2. Some baseline methods are missing. For example, online coreset selection with a streaming k-center algorithm is not selected as a baseline method. Moreover, the baseline methods are quite old since they were proposed before 2024. Section 2 describes herding as matching class means and PICS as selecting visual-similarity clusters, yet neither appears in the comparison. Additionally, the setting is a continual learning setup, so more powerful continual learning baseline methods should be considered in experiments. It would be better if the authors can consider more recently proposed update and sampling strategies for reply buffers, especially in the continual learning domain.

  3. How can the proposed method maintain examples of all seen classes? Will it happen that some minor classes in previously seen data are fully swept out? The experimental setup in this paper is unrealistic. The class imbalance problem sometimes is more troublesome in the setting of online learning or CIL, since in practice, the instances for each class are not equally important. But class-conditional matching does not fully specify the intended buffer distribution. Eq. (8) and Algorithm 1 compare marginal changes in each candidate's class-wise discrepancy, while the replacement quota is global and replay sampling is uniform over stored examples. Even perfect class-conditional matching does not match the joint distribution if buffer class proportions differ from the desired priors. No explicit class weights, minimum class occupancy, or allocation constraint is given. It would be better if the authors can explain these problems.

  4. Kernel calibration may depend strongly on the beginning of the stream. Appendix H sets bandwidths from the buffer when it first reaches capacity, then freezes the RFF map to keep statistics comparable. This is consistent, but calibration occurs before many later classes are observed. Input scaling, augmentation, and the precise representation used by z(x) are also unclear, although they determine RBF geometry. Clarify these choices and test shuffled class orders and bandwidth perturbations, particularly on Tiny-ImageNet. The unresolved question is whether geometry calibrated on early classes remains informative for later classes.

Questions for authors

see weaknesses

6 comments
AuthorsOpenReview ↗

Response to Reviewer Ms92 (W1)

We sincerely thank the reviewer for the constructive suggestions on distribution shifts, baselines, class preservation, and kernel calibration.

W1. Robustness to Distribution Shift and Concept Drift

We evaluate substantial distribution shifts using a heterogeneous, class-disjoint OCIL stream. This tests shifts across image domains and class semantics. As for within-class temporal drift, systematically adapting to such nonstationarity is more closely related to Online Domain-Incremental Learning (ODIL) and lies beyond the class-disjoint setting evaluated here. Therefore, as discussed in Section 5 of the paper, we leave this as a direction for future work.

Cross-dataset experiment. We introduce Seq-MultiDataset, comprising five sequential tasks from distinct datasets: CIFAR-10 (Natural Objects and Scenes) →\rightarrow FashionMNIST (Clothing and Footwear) →\rightarrow SVHN (Real-world Street House Numbers) →\rightarrow KMNIST (Cursive Japanese Characters) →\rightarrow EuroSAT-RGB (Satellite Imagery of Land Use and Terrain). Each task contains 10 disjoint classes with 500 training samples per class, yielding unified 50-way classification. Images are resized to 32×32×332\times32\times3 (replicating grayscale channels) and normalized with mean and standard deviation of 0.5. Reservoir sampling (RS) and DMR use the same ER learner under single-pass, task-agnostic training with K=500K=500, differing only in buffer updates. Results are mean ± std over five seeds; overall accuracy equally weights the class-macro test accuracies of the five datasets to address the imbalance in test-set sizes across these datasets.

DatasetRS Acc. ↑\uparrowDMR Acc. ↑\uparrowRS Fgt. ↓\downarrowDMR Fgt. ↓\downarrow
CIFAR-1027.5 ± 3.635.2 ± 3.513.3 ± 4.17.7 ± 3.1
FashionMNIST71.7 ± 4.875.3 ± 2.34.1 ± 4.32.7 ± 1.7
SVHN55.8 ± 2.760.3 ± 4.214.0 ± 3.613.5 ± 6.6
KMNIST68.8 ± 6.769.0 ± 5.512.1 ± 7.511.9 ± 5.3
EuroSAT-RGB67.6 ± 3.669.7 ± 6.1——
Overall58.3 ± 1.961.9 ± 2.210.9 ± 2.28.9 ± 1.2

DMR improves overall final accuracy by 3.6 pp and reduces average forgetting by 2.0 pp. Mean accuracy improves on all five constituent datasets, supporting DMR's effectiveness under the evaluated cross-dataset distribution/semantic shifts.

AuthorsOpenReview ↗

Response to Reviewer Ms92 (W2)

W2. Comparison with Additional Replay-Selection and Continual-Learning Methods

Additional selection baselines. We compare DMR against GCR [1], LWP [2] and CSReL [3] under the common ER learner. Both LWP and CSReL are 2025 methods; Low Weight Permutation (LWP) uses prediction-based representativeness, while CSReL measures sample importance via reducible loss. All comparisons in this section use the paper's dataset–buffer settings and corresponding training/evaluation protocols, with mean ± std over five runs. The table reports final average accuracy (%); DMR values are from the original Table 1.

DatasetBufferGCR (2022)LWP (2025)CSReL (2025)DMR (ours)
Seq-MNIST5060.7 ± 4.462.8 ± 1.165.4 ± 3.276.4 ± 1.4
Seq-MNIST10071.4 ± 3.174.8 ± 1.976.0 ± 1.582.1 ± 1.6
Seq-CIFAR-1020027.2 ± 6.330.2 ± 2.229.8 ± 2.831.7 ± 1.0
Seq-CIFAR-1050036.4 ± 4.340.2 ± 3.240.8 ± 1.443.9 ± 1.2
Seq-CIFAR-100100013.7 ± 0.816.7 ± 2.416.1 ± 0.418.3 ± 2.1
Seq-CIFAR-100200015.7 ± 0.819.6 ± 1.220.0 ± 2.120.9 ± 1.4
Seq-Tiny-ImageNet10008.6 ± 1.19.6 ± 1.09.7 ± 0.310.1 ± 0.2
Seq-Tiny-ImageNet200013.0 ± 0.313.5 ± 0.913.8 ± 0.414.1 ± 0.8

DMR has higher mean accuracy than all three compared baselines in all eight settings, and lower average forgetting as well (will be reported in the revised manuscript). It is worth noting that, although CSReL is a task-aware method, our proposed DMR manages to outperform it without leveraging task boundary/identity information.

K-center, herding and PICS. The original Table 1 already includes DSS, a 2-approximate k-center selection method evaluated under our common online protocol. As for herding and PICS, they rely on task boundary to perform candidate set selection or visual-similarity cluster construction. Consequently, neither of these methods can directly satisfy our task-agnostic, single-pass access constraints. That said, we do acknowledge that herding is indeed an important and widely adopted sample selection strategy. Hence, we additionally evaluate the original iCaRL-style herding. The results reveal that DMR achieves higher mean accuracy (+2.5 to +8.0 pp) and lower forgetting (-6.7 to -2.3 pp) than herding in all eight settings; this task-aware comparison will appear together with CSReL in the revised appendix.

Additional learners. We integrate DMR into PCR [4] and LOR [5], replacing only the buffer-update policy while preserving each learner's objective and training procedure, using paired runs. LOR is a recent method proposed in 2026; it explores lookahead model states along different weighted combinations of current-batch and replay gradients, then optimizes a first-order Log-Sum-Exp surrogate that emphasizes the worst-performing sampled directions.

DatasetBufferPCR (2023)PCR + DMRLOR (2026)LOR + DMR
Seq-MNIST5082.4 ± 4.885.2 ± 1.986.5 ± 1.988.0 ± 1.0
Seq-MNIST10087.3 ± 2.590.6 ± 2.290.9 ± 0.592.1 ± 0.3
Seq-CIFAR-1020042.9 ± 4.544.1 ± 3.956.7 ± 0.958.9 ± 1.0
Seq-CIFAR-1050050.8 ± 4.052.9 ± 2.460.7 ± 1.662.1 ± 1.6
Seq-CIFAR-100100025.8 ± 1.029.2 ± 1.224.5 ± 1.226.3 ± 0.4
Seq-CIFAR-100200032.9 ± 0.934.1 ± 1.526.9 ± 1.228.1 ± 0.8
Seq-Tiny-ImageNet100015.6 ± 2.018.7 ± 1.919.3 ± 1.022.9 ± 0.5
Seq-Tiny-ImageNet200019.6 ± 0.921.8 ± 0.324.9 ± 1.626.5 ± 0.9

DMR improves mean accuracy in all 16 comparisons (PCR: +1.2 to +3.4 pp; LOR: +1.2 to +3.6 pp), and reduces average forgetting in all 16, extending the original Table 2 to additional mechanisms and a 2026 learner.

AuthorsOpenReview ↗

Response to Reviewer Ms92 (W3)

W3. Class Preservation and Class-Imbalanced Streams

Allocation and class priors. We agree that matching PR(x∣y)P_R(x\mid y) alone does not guarantee matching the joint distribution PR(x,y)P_R(x,y), which also depends on PR(y)P_R(y). However, historical class proportions are not necessarily the optimal replay allocation under a limited memory budget. For example, classes with limited intra-class variation may be adequately represented by fewer exemplars, leaving more capacity for classes with greater diversity. DMR evaluates candidate replacements through changes in class-conditional discrepancy, allowing allocation to respond to per-class matching errors rather than enforcing historical-frequency quotas.

Class preservation. All DMR experiments in the submission and rebuttal used two safeguards: an incoming class absent from the buffer (nR(y)=0n_R(y)=0) receives insertion score +∞+\infty, prioritizing admission when slots are available; deleting a class's last buffered sample (nR(y)=1n_R(y)=1) incurs removal cost +∞+\infty. Since removal minimizes cost, represented classes are preserved when the deletion quota satisfies k≤K−CRk\le K-C_R, where kk is the number of deletions, KK the buffer size, and CRC_R the number of represented classes. This condition holds in all tested configurations. The submitted version's Appendix E, including Algorithm 1 and its empty-class convention, describes an obsolete earlier implementation; it differs from the current implementation only in the safeguards, and all experiments used the implmentation with safeguards. We apologize for this inconsistency and will revise it to reflect the safeguards used in the experiments.

DMR also has a potential implicit balancing tendency. For nR(y)>1n_R(y)>1, removing feature ziz_i changes the class mean by (μR(y)−zi)/(nR(y)−1)(\mu_R^{(y)}-z_i)/(n_R(y)-1); adding one changes it by (zi−μR(y))/(nR(y)+1)(z_i-\mu_R^{(y)})/(n_R(y)+1). Thus, under comparable conditions, changes to smaller classes can be larger. This potentially makes their samples costlier to remove, although candidate scores also depend on feature geometry and the current discrepancy.

Original benchmark buffer diagnostics. Across all eight settings, DMR had zero missing classes in all 40 final-task snapshots and lower task-averaged class-count coefficient of variation (CV) than RS. RS lost classes on Seq-Tiny-ImageNet (K=1000K=1000) while DMR did not. Representative results are as follows; full results will appear in the revision.

SettingFinal missing, RS / DMRMean CV across tasks, RS ↓\downarrowMean CV across tasks, DMR ↓\downarrow
Seq-CIFAR-100, K=1000K=10000 / 00.214 ± 0.0210.152 ± 0.008
Seq-CIFAR-100, K=2000K=20000 / 00.151 ± 0.0090.105 ± 0.002
Seq-Tiny-ImageNet, K=1000K=10001.60 ± 1.15 / 00.326 ± 0.0230.182 ± 0.006
Seq-Tiny-ImageNet, K=2000K=20000 / 00.229 ± 0.0020.120 ± 0.009

Class-imbalanced streams. We evaluate task-wise class-imbalanced Seq-CIFAR-100, K=1000K=1000, with IF=Nmax⁡/Nmin⁡∈{3,10}\mathrm{IF}=N_{\max}/N_{\min}\in\{3,10\}. Training counts decay exponentially within each 10-class task; the test set is unchanged. RS and DMR share five paired streams and seeds. Minority denotes the least frequent 30% of classes.

IFMethodFinal Acc. ↑\uparrowFgt. ↓\downarrowMean CV across tasks ↓Final missingMinority coverage
3RS9.6 ± 0.913.9 ± 1.00.403 ± 0.0090100%
3DMR12.8 ± 2.011.2 ± 1.00.161 ± 0.0050100%
10RS4.7 ± 1.216.8 ± 0.70.731 ± 0.0231.60 ± 1.8294.67 ± 6.06%
10DMR8.0 ± 1.410.8 ± 0.60.389 ± 0.0840100%

For IF=3/10, DMR improves accuracy by 3.2/3.3 pp and reduces forgetting by 2.7/6.0 pp, retaining all seen classes in the final buffer across the tested seeds.

An explicitly balanced variant of DMR. We also tested DMR with approximately uniform feasible class quotas for removal/insertion. Its accuracy is lower than original DMR in all eight settings (e.g., 15.7 ± 1.3% vs. 18.3 ± 2.1% on Seq-CIFAR-100, K=1000K=1000). This suggests that rigid uniform quotas do not improve DMR in these settings.

Reviewer_XuFhOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a novel buffer update method called Distribution Matching Replay (DMR) for Online Class-Incremental Learning (OCIL) by matching the distributions of the replay buffer and the historical data. Existing studies have proposed various strategies for selecting replay samples; however, they often suffer from limited data availability and error accumulation, and only achieve suboptimal performance in practice. Hence, DMR maintains a representative replay buffer by minimizing RFF-MMD together with a memory efficient sketched RFF-MCD between the buffer and the historical data. Extensive experiments on multiple benchmarks demonstrate that DMR outperforms state-of-the-art selection strategies and can be seamlessly integrated into various OCIL methods to further improve their performance.

Critical strengths
  1. The research problem of this paper is very important. The paper identified a clear online information bottleneck and provided a design that directly corresponds to it.

  2. The proposed method has a coherent mathematical structure, with each module serving the same goal.

  3. The ablation study can identify the main source of performance improvement.

Critical weaknesses
  1. The paper demonstrates the practicality of fixed RFF distribution matching on the measured benchmark, but the relationship between the proxy target and the classification related sample value during the training process has not been fully validated.

  2. The experiment does not cover CSReL [1], GCR [2], PCR [3], and UER [4]. Although these methods rely on additional data access, task boundaries, or training sample screening, these tasks are directly related to memory construction or online coreset selection, and I still want to see if DMR has any advantages.

  3. Lack of class imbalance scenarios in the experiment.

[1] Tong R, Liu Y, Shi J Q, et al. Coreset selection via reducible loss in continual learning[C]//International Conference on Learning Representations. 2025, 2025: 13964-13999.

[2] Tiwari R, Killamsetty K, Iyer R, et al. Gcr: Gradient coreset based replay buffer selection for continual learning[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022: 99-108.

[3] Lin H, Zhang B, Feng S, et al. Pcr: Proxy-based contrastive replay for online class-incremental continual learning[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2023: 24246-24255.

[4] Lin H, Feng S, Zhang B, et al. Uer: A heuristic bias addressing approach for online continual learning[C]//Proceedings of the 31st ACM International Conference on Multimedia. 2023: 96-104.

Questions for authors
  1. Does better RFF distribution matching lead to greater replay utility throughout training?

  2. How does DMR compare with CSReL [1] and GCR [2] under comparable settings? What is the effect of using DMR on PCR [3] and UER [4]?

  3. How does DMR perform on class-imbalanced streams?

7 comments
AuthorsOpenReview ↗

Response to Reviewer XuFh (W1 & Q1)

We sincerely thank the reviewer for the constructive suggestions on replay utility, additional comparisons, and class-imbalanced evaluation.

W1 & Q1. Does Better Distribution Matching Lead to Greater Replay Utility?

Replay-utility experiment. We conduct a controlled replay-utility experiment that evaluates whether DMR-selected buffers provide greater training benefits than reservoir sampling (RS) throughout the learning process. On Seq-CIFAR-100 (K=1000K=1000), we save model/optimizer checkpoints and buffers at the midpoints of Tasks 2, 4, 6, 8, and 10. We use a 2×22\times2 cross-evaluation, pairing each checkpoint trained with DMR or reservoir sampling (RS) with a DMR- or RS-selected buffer. Starting from identical model and optimizer states, we perform 20 updates using the same sequence of incoming mini-batches, hyperparameters, and replay batch sizes. The only difference between runs is the replay buffer used, whose contents remain fixed throughout the 20 updates.

We define utility as (i) the decrease in cross-entropy (CE) and (ii) increase in accuracy on all training examples observed up to the checkpoint excluding samples in either buffer. We report DMR's utility advantage, ΔU=U(MDMR;θt)−U(MRS;θt)\Delta U=U(\mathcal M_{\mathrm{DMR}};\theta_t)-U(\mathcal M_{\mathrm{RS}};\theta_t). Five replay-sampling repetitions per combination are averaged within each of five training seeds; results are mean ± std across seeds.

Distribution matching throughout training. We compare class-averaged RFF-MMD² between the replay buffer and the historical reference, using the same RFF map and historical reference for both policies:

Task midpointRS MMD² ↓DMR MMD² ↓
20.0075 ± 0.00070.0026 ± 0.0001
40.0172 ± 0.00130.0056 ± 0.0001
60.0254 ± 0.00110.0091 ± 0.0001
80.0348 ± 0.00180.0124 ± 0.0002
100.0443 ± 0.00170.0164 ± 0.0002
Average0.0258 ± 0.00090.0092 ± 0.0001

Across all five checkpoints, the RFF-MMD² values are lower with DMR; the average RFF-MMD² also decreases by 64.3%, indicating an improvement in the matching of historical distributions at each evaluation stage.

Replay utility. The average utility advantage of DMR buffer across these checkpoints is:

Checkpoint sourceCE utility advantage ↑Accuracy utility advantage ↑ (pp)
DMR-trained+0.0824 ± 0.0322+1.315 ± 0.545
RS-trained+0.0951 ± 0.0327+1.380 ± 0.185

Mean CE and Acc utility advantages are positive at all five checkpoints for both model sources. Benefits on RS-trained models show that the selected buffers' utility extends beyond models previously trained with DMR. These results connect better distribution matching to greater short-horizon classification-relevant utility at the evaluated training stages.

AuthorsOpenReview ↗

Response to Reviewer XuFh (W2 & Q2)

W2 & Q2. Additional Replay-Selection Baselines and Integration with OCIL Learners

CSReL and GCR. We evaluate GCR using its official implementation and CSReL using its official streaming selection mechanism, with the same ER learner and evaluation protocol as DMR. Comparisons in this section use the submission's dataset–buffer settings and corresponding training/evaluation protocols, reporting final average accuracy (%) as mean ± std over five runs. DMR values below are from the original Table 1.

DatasetKKGCR (2022)CSReL (2025)DMR (ours)
Seq-MNIST5060.7 ± 4.465.4 ± 3.276.4 ± 1.4
Seq-MNIST10071.4 ± 3.176.0 ± 1.582.1 ± 1.6
Seq-CIFAR-1020027.2 ± 6.329.8 ± 2.831.7 ± 1.0
Seq-CIFAR-1050036.4 ± 4.340.8 ± 1.443.9 ± 1.2
Seq-CIFAR-100100013.7 ± 0.816.1 ± 0.418.3 ± 2.1
Seq-CIFAR-100200015.7 ± 0.820.0 ± 2.120.9 ± 1.4
Seq-Tiny-ImageNet10008.6 ± 1.19.7 ± 0.310.1 ± 0.2
Seq-Tiny-ImageNet200013.0 ± 0.313.8 ± 0.414.1 ± 0.8

DMR achieves the highest mean accuracy in all eight settings. It is worth noting that we have retained CSReL's task-dependent mechanisms: its auxiliary selection model excludes stored examples from previous tasks, and its retraining schedule uses seen_tasks. This means, although CSReL is a task-aware method, our proposed DMR manages to outperform it without leveraging task boundary/identity information. CSReL will be reported as a comparison using additional task information (task-aware) in the revised appendix, separately from the task-agnostic methods in Table 1.

PCR integration. We replace only PCR's buffer-update policy with DMR, preserving its learning objective and training procedure:

DatasetKKPCR (2023)PCR + DMR
Seq-MNIST5082.4 ± 4.885.2 ± 1.9
Seq-MNIST10087.3 ± 2.590.6 ± 2.2
Seq-CIFAR-1020042.9 ± 4.544.1 ± 3.9
Seq-CIFAR-1050050.8 ± 4.052.9 ± 2.4
Seq-CIFAR-100100025.8 ± 1.029.2 ± 1.2
Seq-CIFAR-100200032.9 ± 0.934.1 ± 1.5
Seq-Tiny-ImageNet100015.6 ± 2.018.7 ± 1.9
Seq-Tiny-ImageNet200019.6 ± 0.921.8 ± 0.3

DMR improves PCR's mean accuracy in all eight settings (+1.2–3.4 pp) and reduces average forgetting in all eight. UER is also under evaluation; we will report its results in a follow-up comment if available during the discussion period.

AuthorsOpenReview ↗

Response to Reviewer XuFh (W3 & Q3)

W3 & Q3. Performance on Class-Imbalanced Streams

We evaluate task-wise class-imbalanced Seq-CIFAR-100 (K=1000K=1000) with IF=Nmax⁡/Nmin⁡∈{3,10}\mathrm{IF}=N_{\max}/N_{\min}\in\{3,10\}. Within each task, the training sample counts of its 10 classes decrease exponentially, while the original test set remains unchanged. RS and DMR use the same five paired streams and seeds. The table reports final average accuracy and forgetting (%), task-averaged class-count coefficient of variation (CV), and the number of classes absent from the final buffer, as mean ± std.

IFMethodFinal Acc. ↑\uparrowFgt. ↓\downarrowMean CV across tasks ↓\downarrowFinal missing ↓\downarrow
3RS9.6 ± 0.913.9 ± 1.00.403 ± 0.0090
3DMR12.8 ± 2.011.2 ± 1.00.161 ± 0.0050
10RS4.7 ± 1.216.8 ± 0.70.731 ± 0.0231.60 ± 1.82
10DMR8.0 ± 1.410.8 ± 0.60.389 ± 0.0840

DMR improves final accuracy by 3.2/3.3 pp and reduces forgetting by 2.7/6.0 pp for IF=3/10, respectively. It also yields a lower class-count CV under both imbalance factors and retains all seen classes in the final replay buffer across the tested seeds, including all minority classes. Under the stronger imbalance (IF=10\mathrm{IF}=10), RS loses 1.60 classes on average, while DMR does not lose any. These results demonstrate that DMR maintains its performance advantage and class coverage under the evaluated class-imbalanced streams.

We hope the controlled replay-utility experiments, more baseline comparisons and integrations, and class-imbalanced evaluation address your concerns. We are happy to provide further details or discuss any remaining concerns.

Reviewer_VTtdOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Distribution Matching Replay (DMR), a buffer-update policy for online class-incremental learning. DMR selects buffer samples by matching their class-wise distributions to those of all observed data through mean and covariance alignment. It uses Random Fourier Features and compact online statistics to avoid retaining historical samples, and a randomized covariance sketch to reduce storage costs. Extensive experimetns are conducted to show improvements over existing selection strategies and demonstrate compatibility with several replay-based learning methods.

Critical strengths
  1. This paper investigates the distribution issue for the Online Class-Incremental Learning task. It proposes a Distribution Matching Replay solution to alleviate the distribution mismatch.
  2. This paper is simple and easy to understand. For instance, Figure 1 can help the reviewers understand the purpose of this article at a glance.
  3. The design of online sufficient statistics is concise and can support accurate addition, deletion, and update.
  4. Covariance compression addresses practical storage bottlenecks and provides theoretical analysis.
Critical weaknesses
  1. My major concern is that the research in this paper is not advanced enough. Most of the references are at least two years ago, with only one [1] from 2025. Therefore, I am not convinced that many of the claims in it still hold true. More latest methods should be analyzed and compared.
  2. There is still a lack of direct mechanism evidence between distribution matching and interference management in OCL. Specifically, in the OCL scenario, conflicts between new and old tasks are a key issue. However, a buffer with a more matching distribution cannot guarantee sufficient handling of this gradient conflict problem.
  3. There is a lack of controlled direct comparison between complete covariance and compressed covariance.
  4. The validation of 'method independent plugins' still lacks several discriminative learning mechanisms.

[1] Tong R, Liu Y, Shi J Q, et al. Coreset selection via reducible loss in continual learning[C]//International Conference on Learning Representations. 2025, 2025: 13964-13999.

Questions for authors
  1. How does DMR advance beyond recent OCL methods published in 2025–2026?
  2. Can the authors demonstrate how distribution matching helps mitigate new–old task interference?
  3. How does covariance compression affect accuracy, memory usage, and runtime compared with full covariance?
  4. Does DMR consistently improve methods with different learning mechanisms, such as [1-3]?

[1] Lai S, Ma C, Zhu F, et al. Gradient-guided epsilon constraint method for online continual learning[J]. Advances in Neural Information Processing Systems, 2026, 38: 124209-124238. [2] Lai S, Zhao Z, Zhu F, et al. Beyond Myopic Alignment: Lookahead Optimization for Online Class-Incremental Learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026: 18053-18062. [3] Ye F, Zhao Y, Liu Q, et al. Dynamic Siamese Expansion Framework for Improving Robustness in Online Continual Learning[J]. Advances in Neural Information Processing Systems, 2026, 38: 71293-71312.

8 comments
AuthorsOpenReview ↗

Response to Reviewer VTtd (W1 & Q1)

We sincerely thank the reviewer for the constructive suggestions on recent methods, interference analysis, covariance compression, and learner integration.

W1 & Q1. Comparisons with Recent Methods

Relation to recent methods. DMR uses compact class-wise statistics of all observed data and RFF-MMD/sketched RFF-MCD for task-agnostic, single-pass buffer selection. This complements the gradient-constrained optimization of GEC (Lai et al., 2026), lookahead optimization of LOR (Lai et al., 2026), and expert expansion of DSEF (Ye et al., 2026). We have already completed the LOR integration: replacing only LOR's buffer-update policy with DMR improves mean accuracy and reduces forgetting in all eight dataset–buffer settings (see our response to W4). We are also evaluating DMR integrated with GEC; if completed during the discussion period, we will report the results in the revised appendix, given that GEC is a task-aware approach.

Additional selection baselines. We compare DMR against GCR [1] and two 2025 methods, Low Weight Permutation (LWP) [2] and CSReL [3], under the same dataset–buffer settings and ER training/evaluation protocol. Final average accuracy (%) is mean ± std over five runs.

DatasetKKGCR (2022)LWP (2025)CSReL (2025)DMR
Seq-MNIST5060.7 ± 4.462.8 ± 1.165.4 ± 3.276.4 ± 1.4
Seq-MNIST10071.4 ± 3.174.8 ± 1.976.0 ± 1.582.1 ± 1.6
Seq-CIFAR-1020027.2 ± 6.330.2 ± 2.229.8 ± 2.831.7 ± 1.0
Seq-CIFAR-1050036.4 ± 4.340.2 ± 3.240.8 ± 1.443.9 ± 1.2
Seq-CIFAR-100100013.7 ± 0.816.7 ± 2.416.1 ± 0.418.3 ± 2.1
Seq-CIFAR-100200015.7 ± 0.819.6 ± 1.220.0 ± 2.120.9 ± 1.4
Seq-Tiny-ImageNet10008.6 ± 1.19.6 ± 1.09.7 ± 0.310.1 ± 0.2
Seq-Tiny-ImageNet200013.0 ± 0.313.5 ± 0.913.8 ± 0.414.1 ± 0.8

DMR achieves the highest mean accuracy in all eight settings, including against both 2025 methods. We will add GCR and LWP to Table 1 of the revised manuscript; CSReL will appear in the appendix because its original implementation uses task-boundary information, whereas Table 1 compares task-agnostic selection strategies.

AuthorsOpenReview ↗

Response to Reviewer VTtd (W2 & Q2)

W2 & Q2. Distribution Matching and Gradient Interference

We test whether DMR-selected replay better approximates historical gradients, and whether it makes joint updates more favorable to old-task retention.

Controlled diagnostics. On Seq-CIFAR-100 (K=1000K=1000), we experiment on checkpoints trained with DMR or reservoir sampling (RS) at the midpoints of Tasks 2, 4, 6, 8, and 10. At each fixed checkpoint, we compare their selected buffers with identical model parameters, current mini-batch, and replay batch size.

Let goldg_{\mathrm{old}} be the full-parameter gradient on previously learned tasks, gnewg_{\mathrm{new}} the current-mini-batch gradient, and grepg_{\mathrm{rep}} the replay gradient. We measure replay alignment cos⁡(grep,gold)\cos(g_{\mathrm{rep}},g_{\mathrm{old}}) to evaluate how well the replay gradients approximate the gradients of past data. We also measure joint-update alignment IB=⟨gold,gnew+λgrep⟩,I_B=\langle g_{\mathrm{old}}, g_{\mathrm{new}}+\lambda g_{\mathrm{rep}}\rangle, where current and replay losses have equal weight (λ=1\lambda=1). Under a first-order gradient-descent approximation, larger IBI_B implies a more favorable change in old-task loss.

We consider two replay modes: natural replay samples normally from each buffer; class-matched replay draws the same label sequence from both, controlling class exposure. The table reports DMR−RS\text{DMR} - \text{RS} (higher is better). Paired differences are averaged over five replay-sampling repetitions per combination, then five checkpoints, then summarized across five training seeds (mean ± std).

Checkpoint sourceReplay modeΔcos⁡(grep,gold)\Delta\cos(g_{\mathrm{rep}},g_{\mathrm{old}}) ↑ΔIB\Delta I_B ↑
DMR-trainedNatural+0.0527 ± 0.0215+30.92 ± 7.42
DMR-trainedClass-matched+0.0506 ± 0.0175+33.07 ± 5.69
RS-trainedNatural+0.0320 ± 0.0220+14.78 ± 9.44
RS-trainedClass-matched+0.0336 ± 0.0164+11.97 ± 6.58

DMR improves both metrics across all four comparisons, including at RS-trained checkpoints. The improvements persist under class-matched replay, indicating that the gains arise in part from the selected examples themselves, beyond differences in class exposure. With gnewg_{\mathrm{new}} and goldg_{\mathrm{old}} fixed, the positive ΔIB\Delta I_B implies that DMR yields a relatively more favorable joint-update direction for old-task retention.

We also repeat the diagnostics using gso-farg_{\mathrm{\text{so-far}}}, the gradient of all data observed so far (including the current task), as the reference in both metrics. This target more directly reflects DMR's historical-distribution matching objective. DMR again improves both metrics in all four comparisons: cosine-similarity gains range from +0.0219 to +0.0392, and update-alignment gains from +11.05 to +30.63. These results complement the old-task diagnostics by showing more faithful approximation of the full observed-data training signal.

Connection to matching and utility. The controlled replay-utility experiment reported in our response to reviewer XuFh W1 uses the same checkpoints and replay buffers as this experiment, providing complementary downstream evidence. The historical RFF-MMD² averaged across these checkpoints is 64.3% lower with DMR. In a controlled 20-update experiment with frozen buffers and otherwise identical training, DMR-selected buffers improve historical accuracy utility over RS by +1.315 / +1.380 pp on DMR-trained / RS-trained checkpoints, respectively. Together, these diagnostics connect better matching with improved replay gradients and short-horizon classification utility in the tested setting.

AuthorsOpenReview ↗

Response to Reviewer VTtd (W3 & Q3)

W3 & Q3. Full vs. Sketched Covariance

Controlled full vs. sketched comparison. We compare full RFF-MCD with the default sketch (r=64r=64), keeping the RFF dimension (d′=512d'=512), mixing weight (α=0.5\alpha=0.5), RFF map, streams, backbone, optimizer, and training budget identical. Accuracy is mean ± std over five paired seeds.

DatasetCovarianceFinal Acc. ↑\uparrowDMR resident memory ↓\downarrowCPU update time ↓\downarrow
Seq-MNIST (K=100K=100)Full82.4 ± 1.122.07 MiB11.01 ms/step
Sketch (r=64r=64)82.1 ± 1.62.40 MiB1.29 ms/step
Seq-CIFAR-10 (K=500K=500)Full45.1 ± 1.232.88 MiB177.51 ms/step
Sketch (r=64r=64)43.9 ± 1.213.41 MiB10.68 ms/step

Memory is the measured resident footprint of DMR-related data structures rather than theoretical estimates. Runtime covers the complete buffer update, including RFF encoding, candidate scoring, buffer modifications, and statistic updates, excluding one-time random-parameter initialization; it is amortized over all stream mini-batch steps in one run. For frequent, relatively small operations, GPU kernel-launch and synchronization overheads, together with parallel execution, can mask differences arising from the algorithms' underlying computational complexity. Therefore, we use CPU timing to make differences in the computational cost of full versus sketched covariance updates easier to observe.

At the default r=64r=64, sketching incurs only 0.3 pp lower mean accuracy on Seq-MNIST and 1.2 pp on Seq-CIFAR-10 relative to full covariance, while substantially reducing memory and update time (59.22% to 89.12% memory footprint reduction and 8.55× to 16.62× speedup). The sketch-dimension sweep already reported in Appendix J, Fig. 5 further shows relatively stable accuracy (r∈{64,128,256,512}r\in\{64,128,256,512\}): 81.5 to 82.1% on Seq-MNIST, and 43.4 to 44.3% on Seq-CIFAR-10. Overall, these controlled results demonstrate that the covariance sketch makes second-order distribution matching considerably more memory- and update-efficient, with only a modest accuracy cost in the tested settings. We will include the direct full-versus-sketch comparison and measurement protocol in the revised manuscript.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Updated Manuscript

We sincerely thank the Area Chair and all reviewers for their constructive feedback and engagement during the discussion period.

We have uploaded the revised manuscript and appendix (with all changes [highlighted in blue]). The updated PDF incorporates the additional experiments, analyses, and clarifications presented in our rebuttal, including:

  • Implementation and Protocol Clarifications: Updated Appendix E (Algorithm 1) to document the class-preservation safeguards used in all experiments, and clarified the task-agnostic OCIL setting, RFF input normalization, and preprocessing details.
  • Additional Baselines and Learner Integrations: Added comparisons with recent replay selection methods (GCR, LWP, CSReL, MERS and herding) and additional learner integrations (PCR, UER, LOR and GEC).
  • Mechanism and Efficiency Analyses: Added controlled replay-utility experiments, gradient-interference diagnostics, and direct full-vs-sketched covariance comparisons, including memory usage, CPU timing, and measurement protocols.
  • Robustness and Buffer Analyses: Added the cross-dataset shift benchmark (Seq-MultiDataset), class-imbalanced stream evaluations, buffer coverage and allocation diagnostics, a uniform-quota DMR ablation, and bandwidth-sensitivity results.

We hope these revisions address the reviewers' concerns, and further strengthen the clarity and empirical support of our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.