- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies forgetting when a pretrained LLM is adapted to new data and only a few past examples (“anchors”) are still available. The proposed CPLUS employs the idea of Collaborative Parameter Learning adding self-probes and mask smoothing. Probes are never training targets, so they can only shrink proposed updates. The novelty of this work is limited. The coordinate-wise sign mask comes from CPL, ICL-generated self-probes from SSR , and EMA smoothing is standard approach. The contribution is the combination, plus the finding that probes work better as gradients than as replay targets .
Critical strengths
- Setting matches open-weight models released without their training data. The split of roles is clean: new data decide what is learned, and probes decide how much of each update is applied. Because probes are never targets, they cannot directly pull the model toward its own errors.
- Comparing methods at matched Acquisition avoids the usual ambiguity in stability-plasticity trade-offs, although Acquisition should be measured on held-out data (see weaknesses).
- The paper is well written and easy to follow, and figures are informative.
Critical weaknesses
-
Acquisition is training accuracy. It is computed on the adaptation (training) examples themselves (App. E.12), so it measures memorization and overfitting, not learning of the new task. It reaches 100% for every method, model, benchmark and budget, typically within 5–10 of the 25 epochs (Fig. 12), so it cannot tell the methods apart.
-
The adaptation set consists of items the base model gets wrong, i.e., the atypical tail of each benchmark. Learning on such items is expected to proceed largely by memorization (see: Feldman&Zhang, What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation, NeurIPS 2020).
-
Retention is measured on unseen items from the same benchmark. The sharp drop for SFT (≈41% on CommonsenseQA and ≈5% on MedQA for Qwen3-0.6B, at 100% Acquisition) is consistent with overfitting to the adaptation set, not only with forgetting in the continual-learning sense.
-
Retention includes the anchors (E.12). At A = 128 of 418 past examples, about 30% of the Retention set feeds directly into the preservation signal or the replay buffer. Memorizing the anchors therefore inflates Retention for anchor-based methods relative to SFT, and partly explains why Retention rises with A. Retention should be reported separately on anchor and non-anchor items.
-
The test in E.5 “disable early stopping, epoch selection, and Acquisition-based run filtering,” which implies the main grid uses all three. The criteria are never described. If epoch selection looks at Retention on the full past set, it uses data that the limited-access setting assumes is unavailable.
-
The rehearsal baselines are weak, which undermines the central claim. The claim that self-probes work better as gradients than as replay (Fig. 5) can only be as strong as the replay comparators. Important standard baselines are missing, as (i) Expansion/architectural methods. The paper discusses them (DER, FOSTER) but compares against none. Classic architecture-growing methods such as Progressive Neural Networks (Rusu et al., 2016) are ill-suited to this task-agnostic, within-benchmark setting. They need task identity for routing, and full columns do not scale to LLMs. Their parameter-efficient counterparts are standard, however: LoRA in the main grid (Biderman et al., TMLR 2024, report that LoRA forgets less) and a per-task adapter method in the sequential experiment, e.g., O-LoRA, InfLoRA or Progressive Prompts. (ii) regularization and weight interpolation. L2-SP or EWC are missing; the Fisher information could even be estimated on the self-probes. So is weight interpolation with the base model (WiSE-FT). (iii) A-GEM or OGD driven by the same blended gradient would separate the contribution of the probes from that of coordinate-wise masking.
-
Why only one seed is used for exoerimentsl evaluation? Reporting results for 9-11 seeds would dispel any doubts about seed hacking.
-
Replay hyperparameters should be tuned under the same development protocol as CPLUS. With only 4–8 anchors, the effect of anchor selection should also be reported, comparing random selection with herding or gradient-based selection (GSS; Aljundi et al., NeurIPS 2019) for both replay and CPLUS.
-
Some presented results are inconsistent, e.g., the identical setting (Qwen3-0.6B, CommonsenseQA, A = 8) gives a CPLUS Retention of ca.74 in Figs. 1a and 3, 79.96 in Table 1, ca.70 in Fig. 5, 71.05 in Table 6, and ca.67 in Figs. 8 and 12.
Questions for authors
-
How is the final checkpoint chosen in the main grid (early stopping, epoch selection, Acquisition-based run filtering)? Which data does each step use?
-
Can you report Acquisition on held-out new-task items and accuracy on a disjoint test split? Can you report Retention separately on anchor and non-anchor items? How does CPLUS compare with SFT using early stopping on a held-out validation split?
-
How much of Acquisition and Retention is memorization? For example, report the cross-fitted in-train vs out-of-train accuracy across seeds, accuracy under option permutation or paraphrase, results with random labels, and a contamination check on base-correct items. Full influence-based memorization scores (Feldman & Zhang, 2020) are expensive, but cheaper evidence (proxy) is available.
-
How does CPLUS compare with logit-based replay on the same anchors and probes (DER++, and X-DER in the sequential setting), with LAMOL, with self-distillation fine-tuning, and with ER-ACE? How were the replay ratio and loss weight tuned?
-
How does CPLUS compare with LoRA, L2-SP/EWC and WiSE-FT, and with A-GEM or OGD using the same blended gradient? In the sequential experiment, how does it compare with per-task adapters (O-LoRA, InfLoRA, Progressive Prompts)? Can the mask be applied on top of LoRA?
-
How are anchors selected? Does a non-random selection rule (herding, GSS) change the results for replay or for CPLUS?
-
How sensitive is CPLUS to N? Why use N = 512 when N = 32 (Fig. 7) appears to give comparable gains? What are the total wall-clock and memory costs, including probe generation?
4 comments
Response to Reviewer jUC7 (2/3)
Response to Reviewer jUC7 (2/3)
(Continued)
4. Replay baselines
"How were the replay ratio and loss weight tuned?"
In Fig 5, we fixed the replay ratio and gave replay and new data equal loss weight (1.0).
Below, we report our quick experiment on qwen3-0.6B /CSQA (same as fig 5) to further study the effect of replay weights . The table reports Retention (%). The 1.0 column gives the original Fig. 5 results.
Probe replay
| Anchors | 0.2 | 0.4 | 0.6 | 0.8 | 1.0 (Fig. 5) |
|---|---|---|---|---|---|
| 4 | 46.31 | 50.38 | 53.49 | 56.04 | 58.13 |
| 8 | 47.77 | 52.85 | 56.83 | 60.14 | 62.84 |
| 16 | 48.22 | 53.72 | 58.13 | 61.72 | 64.59 |
| 32 | 49.24 | 55.46 | 60.45 | 64.49 | 67.70 |
Anchor-and-probe replay
| Anchors | 0.2 | 0.4 | 0.6 | 0.8 | 1.0 (Fig. 5) |
|---|---|---|---|---|---|
| 4 | 45.89 | 48.62 | 50.83 | 52.32 | 53.43 |
| 8 | 48.74 | 53.45 | 58.35 | 59.19 | 60.85 |
| 16 | 49.13 | 54.29 | 59.67 | 61.16 | 62.18 |
| 32 | 49.64 | 55.10 | 59.22 | 62.63 | 63.80 |
For both baselines, the replay weights showed quite monotonic patters, the original weight of 1.0 giving the highest Retention. Still, CPLUS outperformed them across all settings. We will update the manuscript to include these results.
5. Other update rules and adapters
"How does CPLUS compare with LoRA, L2-SP/EWC and WiSE-FT, and with A-GEM or OGD using the same blended gradient?"
Tables 6 and Figure 7 compare CPLUS with controls for update magnitude, mask structure, and gradient projection.
Table R2. Published update-rule comparisons on Qwen3-0.6B/CSQA at eight anchors. Table 6 reports absolute Retention at full Acquisition. Figure 7 reports gain over SFT in percent points and uses 32 probes for CPLUS.
| Method | Retention in Table 6 (%) | Gain over SFT in Figure 7 (points) |
|---|---|---|
| SFT with matched update magnitude | 45.45 | — |
| CPLUS with shuffled mask | 47.13 | — |
| A-GEM | — | +7.50 |
| OGD-GTL | — | +3.92 |
| CPLUS | 71.05 | +35.76 |
CPLUS substantially exceeds both the matched-magnitude SFT control and the shuffled-mask control. Shuffling preserves the mask coefficients while changing their assignment to parameters, showing that the mask's parameter alignment matters.
Figure 7 compares CPLUS with A-GEM and OGD-GTL using anchor gradients for the projection baselines. CPLUS improves Retention over SFT by 35.76 points, compared with 7.50 points for A-GEM and 3.92 points for OGD-GTL.
6. Anchor selection
"How are anchors selected?"
Anchors are selected uniformly at random without replacement from initially correct questions. Within each seed, all methods shared the same anchors.
7. Probe count and cost
"Why use N = 512 when N = 32 (Fig. 7) appears to give comparable gains?"
We used N = 512 as the fixed probe budget for Fig. 3. FIg. 7 shows that N = 32 already improves Retention at eight anchors across all three Qwen3 sizes, showing that CPLUS is effective with substantially fewer probes.
Response to Reviewer jUC7 (3/3)
Response to Reviewer jUC7 (3/3)
(continued)
8. Seed counts and differences across figures
At eight CommonsenseQA anchors, Figures 1a and 3 report the same five-run mean of 74.16% for Qwen3-0.6B. Figure 5 reports 72.33% across three seeds. Within each seed, all three methods used the same 512 probes. Mixed replay and CPLUS also used the same eight anchors. Tables 1 and 6 report separate single-run results of 79.96% and 71.05%, respectively.
Thank you for flagging Figures 8 and 12. Both track forgetting epoch by epoch. We identified two errors. The anchor (A=8) limit also restricted the probe gradient to 8 of the intended 512 self-probes, resulting in the 66.46% scores. We corrected both figures again (66.46 -> 71.05) and the full 512-probe CPLUS measurements from Table 6. CPLUS Retention fell to 30.14% after epoch 1 and recovered to 71.05% by epoch 25.
For clarity, we will update the manuscript with both corrected fig 8 and 12 (marked with a blue box), state the exact training settings and seed counts in the captions, and release the per-epoch values shown before and after correction.
We will shortly update the manuscript with the added results and hightlight the added results in blue
he rebuttal strengthens the evidence but my concerns remain
Thank you for your detailed response and the additional experiments. The new results from the test set have dispelled some of my doubts. Specifying a fixed checkpoint explains the model choice. Table R1 shows that the increase is not an artifact of including reference points in retention: retention without reference points is 71.46% for CPL and 49.51%, and test retention is 73.48% for CPL and 51.01%. It is also evident that, in this setup, the mask does not limit generalization to a new task: the learning rate on the test set is 63.45%, compared to 62.86–64.69% for the other methods.
My main reservations, however, remain unchanged: Reference values for repetitions. The main claim—that self-tests are more useful as gradients than as data for reproduction—is still being tested exclusively in comparison with reproduction using hard labels. The new study covers repetition weights in the range of 0.2–1.0. Retention increases monotonically up to 1.0, the highest value tested; therefore, before concluding that 1.0 is the optimal value, weights above 1.0 should be investigated. Logit-based repetition was not evaluated for the same reference points and probes (DER++/X-DER or KL distillation to the base model). Table R1 also shows that reproducing 8 anchor points helps non-anchor points to a lesser extent than the standard SFT method (47.80% compared to 49.51%), which is consistent with the overfitting of simple reproduction to a small buffer and confirms the need to use stronger reference models for reproductions. The main claim—that in-house probes are more useful as gradients than as data for reconstruction—is still being verified exclusively in comparison with hard-label reconstruction. The new study covers reconstruction weights in the range of 0.2–1.0. Retention increases monotonically up to 1.0, the highest value tested; therefore, weights above 1.0 must be examined before concluding that 1.0 is the optimal value. Logit-based reconstruction for the same anchor points and probes (DER++/X-DER or KL distillation to the baseline model) was not evaluated. Table R1 also shows that reconstructing 8 anchor points helps non-anchor points to a lesser extent than the standard SFT method (47.80% compared to 49.51%), which is consistent with naive reconstruction overfitting to a small buffer and confirms the need for stronger repetition-based base models. The LoRA, L2-SP/EWC, and WiSE-FT models were not included. The results for the A-GEM and OGD-GTL models shown in Fig. 7 use only anchor point gradients. A projection-based foundation model is still needed, one that uses the same anchor point gradient and probes to separate the probe’s contribution from that of the coordinate mask.
Single runs of the same configuration yield results of 79.96% (Table 1) and 71.05% (Table 6), and the SFT index reaches 49.28% in a new run with an initial value of 0, compared to approximately 41% in Fig. 1a, 3, and 5. The spread of this value makes single-grain results (e.g., +1.25 pp for EMA smoothing) impossible to interpret, so multi-grain versions of Tables 1, 5, and 6 are still needed. The error that limited the probe gradient to 8 out of 512 probes in Figs. 8 and 12 also requires confirmation that no other reported result used the coding affected by this error. The response mentioned two errors but described only one.
The choice of N = 512 remains unjustified, and its cost has not been specified. The runs affected by the error (8 probes, 66.46%) and Fig. 7 (32 probes) suggest that even a few probes account for most of the gain. Excluding probes above the N threshold, at a cost that includes probe generation, would help clarify this.
The new study includes one model, one benchmark test, one anchor budget, and, it appears, one seed. For A = 128, retention outside the anchor is reported only for CPLUS, although the anchor-inclusion concern applies to all anchor-based methods. Some minor comments Validation-selected SFT was chosen by validation Acquisition only. Overall validation accuracy would be the standard criterion. The response cites 51.45% held-out Acquisition, which does not appear in Table R1. The A = 128 counts sum to 414, not 418, in past examples. Please clarify what “Acquisition-based run filtering” in App. E.5 refers to: were any runs discarded? Overall, the rebuttal strengthens the evidence that CPLUS preserves behavior beyond the anchors. However, it does not test the central gradients-versus-replay claim against strong rehearsal baselines, and the concerns about variance remain. I therefore keep my score. Presenting the results of a logit model-based repetition on the same probes, along with ablations from multiple starting points, may lead me to reconsider my final assessment.
Response to Reviewer jUC7 (1/3)
Response to Reviewer jUC7 (1/3)
We thank you for your feedback. Below, we have answered to each of your concerns.
1. Model selection
We adopted CPL’s protocol (25-epoch) so that all methods were evaluated after the same number of training epochs. We reported the final checkpoint, but the score after each epoch were logged. In our A=8 CSQA run, CPLUS also had the highest Retention at epochs 15, 20, and 25 (please see below), with all checkpoints at 100% training Acquisition.
Retention on the initially correct CSQA questions, at 8 anchors (Qwen3-0.6B, seed 0).
2. Held-out learning and Retention outside the anchors
We tested four baselines on 1,102 unseen CSQA questions and separately scored Retention on the original anchors and non-anchors (this complements Fig 3):
Table R1. Qwen3-0.6B with 8 anchors For validation-selected SFT, epoch 4 was the earliest checkpoint attaining the highest validation Acquisition.
CPLUS generated all 512 probes from the same 8 anchors available to CPL. Its Retention gain goes beyond those anchors: 71.46% non-anchor Retention, compared with 49.51% for CPL, and 73.48% Retention on the disjoint test set, compared with 51.01% for CPL.
On the held-out test set, CPLUS achieves similar Acquisition to CPL (63.45% vs. 63.32%) while improving Retention by 22.47 pp and overall accuracy by 9.1 pp. Validation-selected SFT (as requested) achieves 52.27% test acc., compared with 67.48% for CPLUS.
At the larger budget highlighted by the reviewer, a separate CPLUS run retained all 128 anchors and 221/286 non-anchors. Retention was 84.30% including anchors and 77.27% excluding anchors.
3. Memorization
At epoch 25, all four methods reach 100% Acquisition, while Acquisition on unseen questions ranges from 62.86% ~ 64.69%. The held-out test set separates training-set fit from generalization. CPLUS achieves 63.45% held-out Acquisition while preserving 71.46% of non-anchor items and 73.48% of initially correct test items, showing preservation beyond the (potentialy memorized) reference anchors.
At epoch 25, all four methods corrected every training question (100% Acq.). On unseen questions that the base model initially answered incorrectly, they corrected 51.45–64.69% in the matched evaluation (Tab. R1). This shows that learning transferred to unseen (not memorized) questions. CPLUS also retained 293/410 (71.46%) initially correct non-anchor answers and 327/445 (73.48%) initially correct test answers, showing that the effect extended beyond the eight anchors.
We also reported a overlap audit of 1,536 generated prompts vs. 1,096 evaluation prompts in app. C.2: only one matched exactly.