- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
L2C selects prompts at three depths of a frozen ViT. A transition history updated by Oja's rule contributes a selection prior and determines which prompts are frozen or have their keys re-initialized. The method combines centered queries, prompt reuse, and a fixed-size pool, with results on CIFAR-100, ImageNet-R, and ImageNet-A.
Critical strengths
Selecting and protecting individual prompts across depths is a reasonable way to support reuse without allocating an entire new module per task. Table 1 reports consistently favorable average accuracy and forgetting across nine settings and five class-order seeds. The ablations include alternatives to Oja's rule and separate the history prior from similarity-based selection. The reported parameter count and throughput are also useful.
Critical weaknesses
The contribution of the Hebbian history is not fully separated from that of generic popularity-based protection. Proposition 1 and the discussion show that in-degree closely tracks selection frequency, while the prior is relatively stable across inputs. A matched baseline using per-prompt usage counts for freezing and a popularity bias for selection would test whether storing cross-stage pathways is necessary. Table 2(b) changes the transition update rule, which does not answer this specific question. The main baseline comparison uses a common five-epoch schedule, which need not give comparable optimization quality or computation across methods. The discussion refers to longer-schedule comparisons with HiDe-Prompt, MOS, and TUNA, but also changes L2C to six pools of twenty prompts. A comparison that keeps L2C's configuration fixed and reports both computation and accuracy would clarify the advantage.
Questions for authors
Does L2C outperform a matched usage-count freezing policy with an independent per-stage popularity prior? How often do old examples change their selected prompts after key recycling and subsequent training?
2 comments
Comment 2 of 2
W3 (fixed configuration under a longer schedule)
Thank you for the suggestion; the experiment was run as requested. The default configuration (three pools of ten prompts) was trained for 20 epochs without any change, and parameters and training time are reported for every method. With its configuration fixed, L2C keeps its advantage at 5 epochs.
10-task ImageNet-R at 5 and 20 epochs per task (five seeds). Parameters are counted after the last task; time is the training time of one run.
| Method | Trainable parameters | 5 epochs: Avg / Fgt | Time (min) | 20 epochs: Avg / Fgt | Time (min) |
|---|---|---|---|---|---|
| HiDe-Prompt | 937,160 | 68.75 / 4.12 | 21.1 | 74.22 / 12.88 | 77.1 |
| MOS | 3,197,000 | 69.88 / 3.89 | 19.3 | 83.07 / 5.04 | 40.9 |
| TUNA | 3,639,560 | 70.15 / 3.55 | 19.6 | 85.91 / 6.20 | 44.1 |
| L2C (default) | 292,040 | 79.24 / 3.56 | 18.0 | 79.53 / 5.61 | 39.1 |
| L2C-Large | 706,760 | 77.15 / 6.94 | 20.1 | 86.81 / 2.81 | 46.8 |
-
Default configuration. It has converged at 5 epochs. At 5 epochs it exceeds all three methods in Avg, and it exceeds HiDe-Prompt trained for 20 epochs.
-
MOS and TUNA at 20 epochs. Both exceed the default configuration, but with 11 and 12 times its trainable parameters: they add an adapter per task, and the longer schedule trains this growing capacity.
-
A larger configuration under the longer schedule. L2C-Large (six pools of 20 prompts with transition matrices between all pairs of pools; 0.71M parameters, less than one quarter of those of MOS and TUNA and independent of the number of tasks) attains the highest Avg and the lowest Fgt at 20 epochs. This is consistent with Tables 12 and 17, which are measured at 5 epochs: under that budget, larger pools and more stages do not raise Avg (L2C-Large at 5 epochs: 77.15, below the default 79.24), whereas the longer schedule gives the additional prompts enough updates to be trained.
Conclusion. With its configuration fixed, L2C is the best at 5 epochs and does not need a longer schedule. The lead of MOS and TUNA over the default configuration at 20 epochs reflects the capacity that a longer schedule can train rather than a weakness of the method, since it disappears with L2C-Large. The default row and the columns of parameters and time are added to Table 10 of the supplementary material.
Q1 (usage-count baseline)
Yes. Against this baseline (“Usage count” in W1), L2C is 1.70 and 5.13 points higher in Avg on 20-task ImageNet-R and ImageNet-A, with lower Fgt and higher on both.
Q2 (selection changes of old examples)
This was measured directly. Old examples change their prompts rarely where the history applies, and almost never because of key recycling.
Test inputs of each task whose selected set after a later task differs from the set selected right after their own task (20 tasks; averaged over all later tasks and five seeds). "Re-initialized" is the share of the changes in which a re-initialized prompt enters the set.
| ImageNet-R: changed | Re-initialized | ImageNet-A: changed | Re-initialized | |
|---|---|---|---|---|
| L2C, shallow | 17.1% | – | 29.8% | – |
| L2C, middle | 10.6% | 1.0% | 22.2% | 7.1% |
| L2C, deep | 16.1% | 0.6% | 23.7% | 3.1% |
| Single pool (M = 10, K = 5) | 61.8% | – | 79.1% | – |
-
Subsequent training. At the middle and deep stages, to which the history applies, the selection is maintained in 84–89% of the cases on ImageNet-R and 76–78% on ImageNet-A; a single pool of the same size maintains it in 21–38%.
-
Key recycling. A re-initialized key is drawn from the orthogonal complement of the hub keys and starts at a neutral similarity to earlier queries (Proposition 3), so recycling leaves the selection of earlier inputs nearly unchanged.
Conclusion. Key recycling is not a source of forgetting. The table is added to Section D.14 of the supplementary material (Table 20).
Comment 1 of 2
Presentation
Apologies for the presentation, which made the method difficult to follow. The revision opens Section 3 with a three-step summary of the procedure (computing the queries from the frozen backbone; selecting prompts with the selection history; freezing and re-initializing prompts at task boundaries) and adds a table of notation to the supplementary material (Table 24).
W1 (Hebbian history versus popularity)
Thank you for this suggestion, which isolates the contribution of the Hebbian history. The requested baseline was run, together with a stricter one. Both remove the transition matrices and both are below L2C, which shows that storing the cross-stage pathways is necessary.
Variants of the history. In both baselines, the usage decides freezing and re-initialization and is added to the selection score with the same weight λ.
Results on the 20-task sequences (mean ± standard deviation over five seeds; Table 18 of the revision).
L2C is better than both baselines in every metric on both benchmarks.
The second baseline differs from L2C only in that its prior is not conditioned on the preceding stage, and the margin remains. The gain comes from the cross-stage transitions, not from recording the usage.
Conclusion. Popularity-based protection alone does not reproduce the results of L2C, so the contribution of the Hebbian history is separated from that of popularity. The results are added to Section 4.3 and to Section D.12 of the supplementary material.
W2 (five-epoch schedule)
The common schedule gives every method the same computation, and the comparison does not rely on it alone, since the methods that need longer schedules are also evaluated at 20 epochs.
Protocol. The common 5-epoch schedule follows continual learning under a computational budget (Prabhu et al., 2023; Seo et al., 2025): every method receives the same number of iterations per task.
Computation. It is comparable under this schedule: the training times on 10-task ImageNet-R lie within 18.0–21.1 minutes (table in W3), and the throughput of L2C lies between those of CODA-Prompt and L2P (Table 15 of the supplementary material).
Optimization quality. L2C has converged at 5 epochs, and the methods designed for longer schedules are also compared at 20 epochs (W3).