- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper adds replay to a prompt-based class-incremental learner (DualPrompt on a frozen ViT-B/16). HSR feeds stored images back through the current prompts using the DER++ losses, so old examples can update both the prompts and the classifier. DHM is a memory manager under a fixed byte budget. It keeps one permanent full-precision image per class and reserves replay slots for these protected images. Other records gradually move from FP32 images to FP16 images to cached features, with the most “stable” examples converted first. On 10-task CIFAR-100 and ImageNet-R (3 seeds, 3 epochs per task), HSR beats the same replay with prompt gradients blocked by 1.84 and 3.15 points. DHM beats a fixed half-image, half-feature buffer by 1.99 and 1.01 points under the same budget. The full system reaches 89.33% and 70.11%, ahead of the PGP and CODA-Prompt results reported in the paper.
Critical strengths
The DETACH control is a clean way to test the main claim. Comparing replay where gradients reach the prompts with the same replay where they’re blocked (Table 3) is a direct test of whether historical images help by updating prompts. The gains (1.84 and 3.15 points, with lower forgetting) consistently point the same way on both datasets. The authors are also careful about what they claim: they say they aren’t the first to combine replay with prompts, and they don’t claim significance for small differences.
Critical weaknesses
- Some memory results don’t make sense, which makes me doubt the comparisons DHM relies on. Because the budget is set in bytes, storing images in FP16 should fit about twice as many images, and uint8 about four times as many, as FP32. Yet in Table 4 both do clearly worse than FP32: on CIFAR-100, 85.85% and 85.80% versus 88.48%; on ImageNet-R, 67.65% and 66.69% versus 69.44%. FP16 is close to lossless for images, so more of them shouldn’t hurt this much. The paper notes that “storage pipelines also differ in preprocessing,” which suggests these rows differ in more than storage format. That matters because DHM’s key step is moving images from FP32 to FP16. It also means FP32 is an unusual baseline: most replay methods store uint8 images, which would already give four times the capacity. Until this is explained, I can’t tell how much of DHM’s gain comes from the memory design and how much from pipeline differences. DHM’s own gain over plain FP32 replay is also small (0.85 and 0.67 points), and getting it takes many added parts: three storage states, a stability score, permanent anchors, replay quotas, reservations and an eviction fallback.
- The novelty is limited and the comparisons don’t place the method fairly. HSR is essentially DER++ applied to DualPrompt with gradients allowed into the prompts, and replay combined with prompts has been done before (e.g., DuPt, which the paper cites). The external comparisons don’t help settle how good the method is:
- PGP and CODA-Prompt use no memory, while HSR+DHM does.
- ER and DER++ fully fine-tune ViT-B/16 for 3 epochs and reach only 31–39% on CIFAR-100. That looks badly under-tuned, not like a real baseline.
- The SigLIP + LoRA result is a single untuned setup.
- CODA-Prompt reports noticeably higher ImageNet-R accuracy in its own paper than the 67% here, likely because of the 3-epoch schedule. Several more recent pretrained-model methods report higher accuracy on both datasets without any replay. The fair comparisons are missing: DualPrompt or CODA-Prompt with plain replay, an existing replay-plus-prompt method, and stronger recent baselines. The title also promises “long-horizon” learning, but every experiment uses only 10 tasks.
Questions for authors
- Why do FP16 and uint8 images, which fit two and four times as many images under the same budget, perform clearly worse than FP32? What exactly differs in their preprocessing, and do the results change if all formats use the same pipeline?
- How does DHM compare with simply storing uint8 images (the usual practice) under the same byte budget, with the same preprocessing as FP32?
- Can you compare against DualPrompt and CODA-Prompt with standard replay under the same budget, and against an existing replay-plus-prompt method such as DuPt?
- Why do ER and DER++ reach only 31–39% on CIFAR-100 with ViT-B/16? How were they tuned?
- DHM’s standard deviation is about 0.16 on CIFAR-100, while all its ablations are around 1.0–1.8. Is there a reason DHM is so much more stable across seeds? With only three seeds and gaps of 1–3 points, can you show the per-seed results?
- DHM reserves space using the known total number of classes. How would it work when that number isn’t known ahead of time?
No response are received from the author, I will keep my rating.
Thanks