acceptodds
Under review as a conference paper at ICLR 2027

VAULT: Trustworthy Finite Memory for Contaminated Continual Learning

Abstract

In contaminated continual learning, replay can turn transient labeling errors into persistent supervision. A finite memory must therefore determine which observations are reliable enough to influence future tasks. We introduce VAULT, a framework that separates participation in current-task learning from admission to long-term memory. VAULT constructs hierarchical evidence in a bootstrapped semantic space, combining granular support and stacked zentropy to assess candidate reliability. Classifier and prototype agreement then certify candidates for storage, while conflict-aware replay preserves the knowledge supported by the resulting memory. Experiments on CIFAR-100 show that VAULT achieves higher final accuracy than the compared replay baselines at the smallest evaluated memory budget across task partitions, together with compact, high-purity storage. Ablations and sample-level analyses connect this advantage to the complementary roles of evidence-based admission and replay preservation. Together, the results show how evidence-based admission builds reliable historical supervision within finite replay memory.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Contaminated Continual Learning, Granular-Ball Computing, Zentropy

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jZVfOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the problem that provides the long-time wrong supervision through persistent replay in contaminated continual learning. The core idea is to split the current-task participation and persistent memory admission. The proposed method VAULT routes the sample by calculating the hierarchical evidence and stacked zentropy, and certificate the sample that will be saved into persistent memory via classifier and validation-prototype agreement. Then, it preserves the knowledge of certified memory from feature/logit/relation preservation and gradient projection.

Critical strengths

S1. Nice Problem setting to split the participation and persistent admission. The sample that is worth to train the current task and the sample that is reliable to replay iteratively in the future works are not equivalent. VAULT addresses explicitly those difference.

S2. Empirical results are promising in small-memory regime. VAULT shows the FAA 59.28% and memory purity 98.66% under 5 tasks, 50% contamination, and cap 500 constraint on CIFAR-100.

S3. Evaluation of memory quality is well-aligned with research purpose. The paper analyzes not only nominal capacity but also actual occupancy, purity, and effective clean capacity. It validates the motivation that acquiring the reliable replay supervision is more important than naively filling in the memory.

Critical weaknesses

W1. Contribution of validation-prototype supervision. VAULT leverages labeled validation prototype on not only checkpoint selection, but also memory certification. Hence, it is uncertain that how much portion of the entire improvement can get the admissions mechanism.

W2. Empty-memory case of HE/SZ ablation hides the effectiveness of core mechanism. Empty replay memory (i.e., Acutal=0) can be occurred under w/o HE, w/o SZ, or etc, thereby those FAA drop is caused from two things; (1) Hierarchical evidence or the effect of changing stacked zentropy. (2) The diminished effect of persistent replay. The current results show that the corresponding component needs to generate the memory under the previous threshold setup, but it is hard to validate the better trust estimator under the identical replay budget.

W3. Lack of contribution attribution compared to framework complexity. VAULT combines with lots of things, but those might be reasonable. The paper should split the effectiveness of each component to show the necessity, but current ablation studies are not enough. In particular, comparison by changing the admission criterion is needed while keeping the identical backbone, bootstrap, validation access, and replay objectiveness.

W4. Threshold selection and sensitivity. In routing and certification, there are several thresholds, and those can affect to memory occupancy and purity directly. However, those analyses are not included in this manuscript.

W5. Clarification of scope. All the experiments are conducted on the Task incremental learning setup, but in the initial part of the paper including abstract and introduction, there is no mention about task incremental learning. Hence, I expect that this paper can cover the class incremental learning while reading introduction section. Moreover, I'm curious that why the proposed method did not experiment under the class incremental learning. To the best of my understanding, the proposed method can compare with the other baselines (in fact those baselines comes from class incremental learning setup) under the CIL setup.

W6. Buffer size. The benefit of VAULT is focused on cap 500. As the cap increases, the performance of VAULT is saturated, while the performance of EARL beats VAULT. I think purity-occupancy trade-off make the limits to potential improvement as increasing the buffer. It would be nice approach when the authors discuss more deeply those kind of topics in the manuscript.

W7. Real noise scenario. There is a paper in terms of real noise CIFAR-100 that humans annotated [1]. Those result will validate the effectiveness of the proposed method can be worked in real scenario.

[1] Jiaheng Wei, et al. "Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations", ICLR 2022.

W8. Statistics. Table 1--3 shows only average performance, but in noisy environment, standard deviation is also very important metric to show the robustness of the method. Please report the standard deviation, and if the number is negligibly small, it validates the robustness.

W9. Fairness between the proposed method and baselines. Most of the baselines use the single backbone (RN50), but the proposed method uses two: semantic encoder and plastic encoder. Therefore, for fair comparison, the authors should match the parameter size.

Questions for authors

See the weaknesses. Here are the minor comments below.

Q1. Why are most of the figures and tables located in the middle of the page? Typically, most of the figures and tables in the paper should be located at the top of the paper.

Q2. In contamination procedure, there are two types to make noisy labels: (1) Symmetric noise (2) Asymmetric noise (Please see [1]). Could you show the results of your method under those environments?

Q3. I think Table 6 should be located in the main script, not Appendix. It would be good evidence to show the effectiveness of VAULT in various datasets (especially large-scale dataset). Moreover, please discuss the reason of low performance in WebVision.

[1] ihwan Bang, Hyunseo Koh, Seulki Park, Hwanjun Song, Jung-Woo Ha, and Jonghyun Choi. Online continual learning on a contaminated data stream with blurry task boundaries. CVPR 2022.

Reviewer_KTG9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VAULT, a replay-memory framework for contaminated continual learning that combines structural evidence, zentropy, classifier/prototype agreement, and conflict-aware replay, achieving strong performance on CIFAR-100.

Critical strengths
  1. Interesting and meaningful problem formulation. Contaminated continual learning is interesting and treating replay-memory construction as a reliability-based admission problem is well motivated for contaminated continual learning
Critical weaknesses
  1. The method contains too many tightly coupled components. The overall design feels more like a collection of existing techniques than a clearly isolated, novel contribution, making it difficult to identify the paper’s central methodological innovation.
  2. The experimental validation is limited. The authors should evaluate VAULT on longer task sequences and additional backbones, since the current experiments rely mainly on CIFAR-100 with ResNet-18 and do not establish generality. What's more, VAULT relies on uncertainty and prediction agreement for memory admission, the paper should report calibration metrics such as ECE.
Questions for authors
  1. Regarding the method design, what is the central methodological innovation of VAULT, and why are all of its components necessary?
  2. Regarding experimental validation, how does VAULT perform on longer task sequences and with different backbones or datasets?
Reviewer_hV9HOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies contaminated continual learning, where replay can repeatedly reuse mislabeled samples and make temporary errors persistent. VAULT separates current-task learning from long-term memory admission. It combines semantic evidence, classifier/prototype agreement, and conflict-aware replay to decide which samples should be stored and preserved. The method performs on CIFAR-100 and ImageNet100 under synthetic contamination, while the gains on WebVision are more limited.

Critical strengths
  1. The motivation is reasonable. In replay-based continual learning, a noisy sample can affect not only the current task but also future tasks once it enters memory. Separating temporary training use from persistent memory admission is therefore a useful design choice.

  2. The experiments cover several important factors, which help connect final accuracy to the samples that are actually stored.

Critical weaknesses
  1. The novelty could be positioned more precisely relative to prior noise-aware replay methods. Previous work already considers memory purity, noisy-sample filtering, prototype-based denoising, and replay preservation. VAULT's clearest contribution appears to be the explicit separation between current-task participation and persistent memory admission. The paper would benefit from making this distinction more explicit and from showing more directly what is gained by the admission formulation beyond existing noise-aware memory selection methods.

  2. The current ablations do not cleanly isolate the effects of the admission components. In Table 2, w/o HE and w/o HE+GP both obtain 45.86 FAA, while w/o SZ and w/o SZ+GP both obtain 40.22. Appendix G shows that these variants produce no certified replay memory, so the later replay mechanism cannot operate. In contrast, when CP is also removed and replay memory remains available, w/o HE+CP and w/o SZ+CP recover to 53.25 and 53.29 FAA. This suggests that the large drops attributed to HE and SZ partly reflect their interaction with the memory certification pipeline rather than their independent contribution. The reported gap between full VAULT and w/o GP is comparatively smaller (61.96 vs. 60.24 FAA).

  3. Several implementation details of the memory pipeline are underspecified(define the quality/coverage ranking used to select certified samples under the memory budget) . The task-end two-step consolidation is also unclear: Equation 11 defines the memory loss using Mt−1M_{t-1}, while Algorithm 1 performs the consolidation after MtM_t has been selected. Clarifying which memory is used and how the final anchors are ranked would improve reproducibility.

Questions for authors
  1. See Weaknesses.

  2. Labeled validation prototypes are used directly in memory certification, beyond their role in checkpoint selection. Since this supervision affects which samples enter persistent memory, the paper should clarify the assumptions on the validation labels and how the same validation information is available to the compared methods. An ablation without prototype-based certification would also help quantify its contribution.

1 comment

I thank the authors for their detailed responses and clarifications. I think that most of my main concerns are resolved. Therefore, I will increase my overall evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.