acceptodds
Under review as a conference paper at ICLR 2027

Historical Supervision Reactivation with Dynamic Heterogeneous Memory for Long-Horizon Class-Incremental Learning

Abstract

Rehearsal-free prompt-based methods adapt pretrained models to new classes by training a small set of prompt parameters, yet can still forget old classes during subsequent learning. We investigate whether a limited history of examples can alleviate this forgetting by restoring direct supervision. We introduce Historical Supervision Reactivation (HSR), which enables losses on stored images to update trainable prompts and the classifier. To sustain this training process within limited storage, we further introduce Dynamic Heterogeneous Memory (DHM). DHM protects historical images and reserves replay quotas to retain their update paths and opportunities for use, while managing the remaining examples through a classification-stability-guided lifecycle from FP32 images to FP16 images and compact features. Our controlled study evaluates 23 configurations across three seeds on CIFAR-100 and ImageNet-R, for 69 runs with ten tasks and three training epochs per task. In supervision-path comparisons, HSR improves mean final accuracy by 1.84 and 3.15 percentage points over image replay with historical prompt gradients blocked. Under the same memory caps, DHM improves accuracy by 1.99 and 1.01 points over a fixed heterogeneous buffer. Historical image supervision mitigates forgetting in prompt-based models, and DHM further improves accuracy and forgetting under a fixed memory budget.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
continual learning, class-incremental learning, prompt-based learning, rehearsal, replay, memory management, heterogeneous memory

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_AtgwOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper adds replay to a prompt-based class-incremental learner (DualPrompt on a frozen ViT-B/16). HSR feeds stored images back through the current prompts using the DER++ losses, so old examples can update both the prompts and the classifier. DHM is a memory manager under a fixed byte budget. It keeps one permanent full-precision image per class and reserves replay slots for these protected images. Other records gradually move from FP32 images to FP16 images to cached features, with the most “stable” examples converted first. On 10-task CIFAR-100 and ImageNet-R (3 seeds, 3 epochs per task), HSR beats the same replay with prompt gradients blocked by 1.84 and 3.15 points. DHM beats a fixed half-image, half-feature buffer by 1.99 and 1.01 points under the same budget. The full system reaches 89.33% and 70.11%, ahead of the PGP and CODA-Prompt results reported in the paper.

Critical strengths

The DETACH control is a clean way to test the main claim. Comparing replay where gradients reach the prompts with the same replay where they’re blocked (Table 3) is a direct test of whether historical images help by updating prompts. The gains (1.84 and 3.15 points, with lower forgetting) consistently point the same way on both datasets. The authors are also careful about what they claim: they say they aren’t the first to combine replay with prompts, and they don’t claim significance for small differences.

Critical weaknesses
  1. Some memory results don’t make sense, which makes me doubt the comparisons DHM relies on. Because the budget is set in bytes, storing images in FP16 should fit about twice as many images, and uint8 about four times as many, as FP32. Yet in Table 4 both do clearly worse than FP32: on CIFAR-100, 85.85% and 85.80% versus 88.48%; on ImageNet-R, 67.65% and 66.69% versus 69.44%. FP16 is close to lossless for images, so more of them shouldn’t hurt this much. The paper notes that “storage pipelines also differ in preprocessing,” which suggests these rows differ in more than storage format. That matters because DHM’s key step is moving images from FP32 to FP16. It also means FP32 is an unusual baseline: most replay methods store uint8 images, which would already give four times the capacity. Until this is explained, I can’t tell how much of DHM’s gain comes from the memory design and how much from pipeline differences. DHM’s own gain over plain FP32 replay is also small (0.85 and 0.67 points), and getting it takes many added parts: three storage states, a stability score, permanent anchors, replay quotas, reservations and an eviction fallback.
  2. The novelty is limited and the comparisons don’t place the method fairly. HSR is essentially DER++ applied to DualPrompt with gradients allowed into the prompts, and replay combined with prompts has been done before (e.g., DuPt, which the paper cites). The external comparisons don’t help settle how good the method is:
  • PGP and CODA-Prompt use no memory, while HSR+DHM does.
  • ER and DER++ fully fine-tune ViT-B/16 for 3 epochs and reach only 31–39% on CIFAR-100. That looks badly under-tuned, not like a real baseline.
  • The SigLIP + LoRA result is a single untuned setup.
  • CODA-Prompt reports noticeably higher ImageNet-R accuracy in its own paper than the 67% here, likely because of the 3-epoch schedule. Several more recent pretrained-model methods report higher accuracy on both datasets without any replay. The fair comparisons are missing: DualPrompt or CODA-Prompt with plain replay, an existing replay-plus-prompt method, and stronger recent baselines. The title also promises “long-horizon” learning, but every experiment uses only 10 tasks.
Questions for authors
  1. Why do FP16 and uint8 images, which fit two and four times as many images under the same budget, perform clearly worse than FP32? What exactly differs in their preprocessing, and do the results change if all formats use the same pipeline?
  2. How does DHM compare with simply storing uint8 images (the usual practice) under the same byte budget, with the same preprocessing as FP32?
  3. Can you compare against DualPrompt and CODA-Prompt with standard replay under the same budget, and against an existing replay-plus-prompt method such as DuPt?
  4. Why do ER and DER++ reach only 31–39% on CIFAR-100 with ViT-B/16? How were they tuned?
  5. DHM’s standard deviation is about 0.16 on CIFAR-100, while all its ablations are around 1.0–1.8. Is there a reason DHM is so much more stable across seeds? With only three seeds and gaps of 1–3 points, can you show the per-seed results?
  6. DHM reserves space using the known total number of classes. How would it work when that number isn’t known ahead of time?
Reviewer_G4dCOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposed to solve the continual learning problem in long-horizon class-incremental learning using a prompt based learning and reply buffer. It proposes Historical Supervision Reactivation (HSR), which replays stored images so losses from old classes can update both prompts and the classifier. To operate in a fixed memory budget the author proposed Dynamic Heterogeneous Memory (DHM). DHM retains protected full-precision image anchors for each class, reserves replay for them, and progressively converts other examples from FP32 images to FP16 images and cached features based on classification stability. The final cache features are only used to supervise the classifier. The experiment on the CIFAR and ImageNet-R dataset for the 10 tasks and three random seeds are conducted. HSR improves final accuracy over replay with historical prompt gradients blocked by 1.84 and 3.15 percentage points for the above datasets respectively. Adding DHM raises accuracy and outperforms a fixed heterogeneous buffer.

Critical strengths

The isolation of historical prompt supervision through a gradient blocking control is interesting, also it shows the accuracy gains on CIFAR-100 and ImageNet-R.

The proposed DHM combines protected full precision image anchors, reserved replay quotas, and stability guided conversion to lower precision images or features, preserving historical supervision under strict fixed storage constraints.

Paper is evaluated over controlled ablations, matched memory comparisons, three random seeds, and two datasets, also the author reports checkpoint specifications, class orders, optimization settings, and explicit limitations to support reproducibility and interpretation.

Critical weaknesses

The paper leverages the prompt based approach and proposed HSR to combine the prompt based approach with the replay buffer to solve the continual learning problem. However, the key contributions mostly exists; DuPt already combines rehearsal with dual-prompt continual learning and feature-level knowledge transfer. Thus, the central concept of updating a prompt-based learner using historical examples is not new. The key loss Eq. 2-5 is largely explored in DER++ combines replayed historical images, supervised cross-entropy, and logit-matching losses. The key backbone is adapted from the work DualPrompt. I find most of the components are established and paper combines these again with the minor changes. The key distinctive contribution appears to be the controlled comparison between enabling and blocking historical prompt gradients, which I do not find a significant contribution. I request the authors to justify the key claims.

DHM combines three representation states (FP32 images, FP16 images, and cached features), a stability-guided conversion policy, permanent image anchors, and replay quotas. Several individual ideas are already established in [a.b.c,d]. Also,Table 4 shows that the advantage of DHM over plain FP32-image HSR is modest; the major performance gain comes from using the historical supervision, whereas the key distinctive contribution DHM provides a substantially smaller improvement.

The result shown in Table 2 is insufficient to show the improvement over the modern pretrained-model continual-learning baselines because of the following reason: (1) PGP and CODA-Prompt are rehearsal-free, whereas HSR + DHM explicitly stores and replays old images. (2) ER and DER++ fully fine-tune a ViT-B/16 with approximately 85.88 million trainable parameters in the CIFAR-100 experiments, while HSR uses a frozen backbone. (3) The paper does not experimentally compare with DuPt, although it is especially relevant to the main claimed contribution. DuPt is the closest paper to the proposed model and the author has not compared the same paper. The baseline should have: a matched-backbone and matched-budget comparison covering DualPrompt + DER++, DuPt, fixed-feature classifiers, and adaptive-buffer variants. Training compute should also be reported.

The title emphasizes long-horizon learning, but the paper evaluates only ten sequential tasks, with three training epochs per task. This does not justify the long long-horizon learning.

There are recent generative prompt based approaches [e] that show better results without using the replay samples, these should be also included in the baselines and if you apply the replay over these approaches what are the improvements?

The result reported for the CODA-Prompt does not match the reported result in the original paper, why is this the result. I find the author manipulated results and shows the wrong baselines results.

Reference:

[a] RMM: Reinforced Memory Management for Class-Incremental Learning. Advances in Neural Information Processing Systems (NeurIPS-21)

[b] Memory Replay with Data Compression for Continual Learning. International Conference on Learning Representations (ICLR-22)

[c] Class-Incremental Exemplar Compression for Class-Incremental Learning (CVPR-23)

[d] Autoencoder-Based Hybrid Replay for Class-Incremental Learning (ICML-25)

[e] Convolutional Prompting meets Language Models for Continual Learning (CVPR-24)

Questions for authors

Please refer to the weakness section

1 comment

No response are received from the author, I will keep my rating.

Thanks

Reviewer_vxweOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Historical Supervision Reactivation (HSR) mechanism, applying past classification results and logit-based loss to DualPrompt architecture. It also introduces Dynamic Heterogeneous Memory (DHM) that combines stored images, replay, and stability-guided conversion image to feature space under a limited budget. Both HSR and DHM are heavily rooted in past continual learning mechanisms and can be seen as simple extensions to prompt-based architecture of already proposed approaches.

Critical strengths

The obtained improvements are interesting, raging from 1.84 to 3.15 on CIFAR-100 and ImageNet-R. I like the byte-based accounting, three-seed based results, and explicit acknowledgment of comparison limitations - that reads like an honest statement from the Authors.

Critical weaknesses

I find this paper to have 4 critical limitations that prohibit it from being published in its current state:

W1: Limited novelty. HSR uses well-known DER++ loses and applies them to prompt-based architecture. Restoring gradients through replayed images is a very standard approach in replay-based continual learning. There exist prior works, such as DuPt [1] that already combines rehearsal and prompts. Therefore, the main contributions lies in DHM - but there is lack of experimental results / ablation that shows how much DHM contributes to the overall results, as compared with other compression strategies.

W2: Memory comparison do not sufficiently isolate the proposed mechanism. Looking at Table 4, we can see that it changes admission rules and preprocessing alongside the representation learning/management itself. A truly convincing control study would preserve the same anchors, admission / eviction rules, preprocessing, and replay sampling without adding any variation on conversion policy. I am also missing any explanation on the substantial degradation of FP16 replay.

W3: Lack of examination of long-term efficacy and limited evaluation horizon. Current experiments use only T=10 setting, which is not up to current SOTA standards. Additionally, only two datasets and one backbone are being examined, with three epochs per task. This is substantially below current experimental standards. I would need to see longer tasks sequences, multiple memory budgets, sensitivity to training duration, etc. Permanent per-class anchors are definitely affecting the memory usage with class count growth, and the advance reservation assumes we have an oracle knowledge on the final class count - very impractical and limiting assumption.

W4: Limited reproducibility and comparison with SOTA. Table 2 shows a mic of different architectures, trainable parameters, and replay sources. There are missing details, like non-anchor eviction policy. SOTA methods for prompt-based continual learning are missing. Figure 4 seems to diagnose early admission rule and not actually the final proposed method.

[1] Shengqin Jiang, Daolong Zhang, Fengna Cheng, Xiaobo Lu, Qingshan Liu: DuPt: Rehearsal-based continual learning with dual prompts. Neural Networks 187: 107306 (2025)

Questions for authors

I have the following questions to the Authors:

Q1: What is the actual novelty in HSR beyond applying DER++ based loses and how your method compares to DuPt under matched resources?

Q2: Does DHM still outperforms image replay when preprocessing, anchor protection, admission and eviction rules, and replayed samples are completely identical? Because under such assumption only the representation conversion policy would change truly proving the usefulness of DHM.

Q3: Why do FP16 and uint8 image buffers perform significantly weaker when compared to FP32, while actually storing more samples under the same byte budget? Can you separate the precision effect from preprocessing / sampling differences?

Q4: How does DHM handle longer task sequences, different datasets, and and smaller memory budgets, especially when the final class count is unknown?

Q5: How does your method compare to recent SOTA (2025/2026) on prompt-based continual learning published in A* conferences? There are plethora of recent methods in this area.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.