acceptodds
Under review as a conference paper at ICLR 2027

Continual Learning via Self-Probe Gradients

Abstract

Adapting pretrained models to new data can cause catastrophic forgetting of previously learned behavior. When only a few past samples remain, they give continual learning methods sparse and narrow evidence about what to preserve. We show that language models can expand this evidence through self-probing, in which the frozen model generates new inputs from the retained samples and records its own predictions on them. Unlike prior work that replays such data as training examples, our method, CPLUS uses self-probe and past-sample gradients to scale down parameter updates that conflict with prior behavior. Experiments with five language models on four benchmarks show three results. First, the same probes preserve more prior behavior as gradient signals than as replay data. Second, CPLUS learns the new data while consistently reducing forgetting more than existing baselines, especially when past data are scarce, and this protection extends to benchmarks not used for training. Third, we observe that CPLUS also becomes more effective as models grow: within the Qwen3 model family, it recovers an increasing share of the forgetting caused by standard fine-tuning.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
continual learning, large language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jUC7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies forgetting when a pretrained LLM is adapted to new data and only a few past examples (“anchors”) are still available. The proposed CPLUS employs the idea of Collaborative Parameter Learning adding self-probes and mask smoothing. Probes are never training targets, so they can only shrink proposed updates. The novelty of this work is limited. The coordinate-wise sign mask comes from CPL, ICL-generated self-probes from SSR , and EMA smoothing is standard approach. The contribution is the combination, plus the finding that probes work better as gradients than as replay targets .

Critical strengths
  1. Setting matches open-weight models released without their training data. The split of roles is clean: new data decide what is learned, and probes decide how much of each update is applied. Because probes are never targets, they cannot directly pull the model toward its own errors.
  2. Comparing methods at matched Acquisition avoids the usual ambiguity in stability-plasticity trade-offs, although Acquisition should be measured on held-out data (see weaknesses).
  3. The paper is well written and easy to follow, and figures are informative.
Critical weaknesses
  1. Acquisition is training accuracy. It is computed on the adaptation (training) examples themselves (App. E.12), so it measures memorization and overfitting, not learning of the new task. It reaches 100% for every method, model, benchmark and budget, typically within 5–10 of the 25 epochs (Fig. 12), so it cannot tell the methods apart.

  2. The adaptation set consists of items the base model gets wrong, i.e., the atypical tail of each benchmark. Learning on such items is expected to proceed largely by memorization (see: Feldman&Zhang, What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation, NeurIPS 2020).

  3. Retention is measured on unseen items from the same benchmark. The sharp drop for SFT (≈41% on CommonsenseQA and ≈5% on MedQA for Qwen3-0.6B, at 100% Acquisition) is consistent with overfitting to the adaptation set, not only with forgetting in the continual-learning sense.

  4. Retention includes the anchors (E.12). At A = 128 of 418 past examples, about 30% of the Retention set feeds directly into the preservation signal or the replay buffer. Memorizing the anchors therefore inflates Retention for anchor-based methods relative to SFT, and partly explains why Retention rises with A. Retention should be reported separately on anchor and non-anchor items.

  5. The test in E.5 “disable early stopping, epoch selection, and Acquisition-based run filtering,” which implies the main grid uses all three. The criteria are never described. If epoch selection looks at Retention on the full past set, it uses data that the limited-access setting assumes is unavailable.

  6. The rehearsal baselines are weak, which undermines the central claim. The claim that self-probes work better as gradients than as replay (Fig. 5) can only be as strong as the replay comparators. Important standard baselines are missing, as (i) Expansion/architectural methods. The paper discusses them (DER, FOSTER) but compares against none. Classic architecture-growing methods such as Progressive Neural Networks (Rusu et al., 2016) are ill-suited to this task-agnostic, within-benchmark setting. They need task identity for routing, and full columns do not scale to LLMs. Their parameter-efficient counterparts are standard, however: LoRA in the main grid (Biderman et al., TMLR 2024, report that LoRA forgets less) and a per-task adapter method in the sequential experiment, e.g., O-LoRA, InfLoRA or Progressive Prompts. (ii) regularization and weight interpolation. L2-SP or EWC are missing; the Fisher information could even be estimated on the self-probes. So is weight interpolation with the base model (WiSE-FT). (iii) A-GEM or OGD driven by the same blended gradient would separate the contribution of the probes from that of coordinate-wise masking.

  7. Why only one seed is used for exoerimentsl evaluation? Reporting results for 9-11 seeds would dispel any doubts about seed hacking.

  8. Replay hyperparameters should be tuned under the same development protocol as CPLUS. With only 4–8 anchors, the effect of anchor selection should also be reported, comparing random selection with herding or gradient-based selection (GSS; Aljundi et al., NeurIPS 2019) for both replay and CPLUS.

  9. Some presented results are inconsistent, e.g., the identical setting (Qwen3-0.6B, CommonsenseQA, A = 8) gives a CPLUS Retention of ca.74 in Figs. 1a and 3, 79.96 in Table 1, ca.70 in Fig. 5, 71.05 in Table 6, and ca.67 in Figs. 8 and 12.

Questions for authors
  1. How is the final checkpoint chosen in the main grid (early stopping, epoch selection, Acquisition-based run filtering)? Which data does each step use?

  2. Can you report Acquisition on held-out new-task items and accuracy on a disjoint test split? Can you report Retention separately on anchor and non-anchor items? How does CPLUS compare with SFT using early stopping on a held-out validation split?

  3. How much of Acquisition and Retention is memorization? For example, report the cross-fitted in-train vs out-of-train accuracy across seeds, accuracy under option permutation or paraphrase, results with random labels, and a contamination check on base-correct items. Full influence-based memorization scores (Feldman & Zhang, 2020) are expensive, but cheaper evidence (proxy) is available.

  4. How does CPLUS compare with logit-based replay on the same anchors and probes (DER++, and X-DER in the sequential setting), with LAMOL, with self-distillation fine-tuning, and with ER-ACE? How were the replay ratio and loss weight tuned?

  5. How does CPLUS compare with LoRA, L2-SP/EWC and WiSE-FT, and with A-GEM or OGD using the same blended gradient? In the sequential experiment, how does it compare with per-task adapters (O-LoRA, InfLoRA, Progressive Prompts)? Can the mask be applied on top of LoRA?

  6. How are anchors selected? Does a non-random selection rule (herding, GSS) change the results for replay or for CPLUS?

  7. How sensitive is CPLUS to N? Why use N = 512 when N = 32 (Fig. 7) appears to give comparable gains? What are the total wall-clock and memory costs, including probe generation?

4 comments
AuthorsOpenReview ↗

Response to Reviewer jUC7 (1/3)

Response to Reviewer jUC7 (1/3)

We thank you for your feedback. Below, we have answered to each of your concerns.

1. Model selection

"How is the final checkpoint chosen in the main grid..?"

We adopted CPL’s protocol (25-epoch) so that all methods were evaluated after the same number of training epochs. We reported the final checkpoint, but the score after each epoch were logged. In our A=8 CSQA run, CPLUS also had the highest Retention at epochs 15, 20, and 25 (please see below), with all checkpoints at 100% training Acquisition.

Retention on the initially correct CSQA questions, at 8 anchors (Qwen3-0.6B, seed 0).

MethodEpoch 15Epoch 20Epoch 25
SFT49.52%49.52%49.28%
Replay48.09%48.33%48.80%
CPL49.76%50.24%50.48%
CPLUS73.21%72.49%72.01%

2. Held-out learning and Retention outside the anchors

"Can you report Acquisition on held-out new-task items and accuracy on a disjoint test split? Can you report Retention separately on anchor and non-anchor items? How does CPLUS compare with SFT using early stopping on a held-out validation split?"

We tested four baselines on 1,102 unseen CSQA questions and separately scored Retention on the original anchors and non-anchors (this complements Fig 3):

Table R1. Qwen3-0.6B with 8 anchors For validation-selected SFT, epoch 4 was the earliest checkpoint attaining the highest validation Acquisition.

MethodEpochTraining AcquisitionAnchors retainedNon-anchor RetentionTest AcquisitionTest RetentionTest accuracy
Base0—————40.38
SFT251003/849.5162.8652.1358.53
SFT /validation-selected496.312/835.1263.7735.2852.27
Replay251008/847.8064.6948.3158.08
CPL251008/849.5163.3251.0158.35
CPLUS251008/871.4663.4573.4867.48

CPLUS generated all 512 probes from the same 8 anchors available to CPL. Its Retention gain goes beyond those anchors: 71.46% non-anchor Retention, compared with 49.51% for CPL, and 73.48% Retention on the disjoint test set, compared with 51.01% for CPL.

On the held-out test set, CPLUS achieves similar Acquisition to CPL (63.45% vs. 63.32%) while improving Retention by 22.47 pp and overall accuracy by 9.1 pp. Validation-selected SFT (as requested) achieves 52.27% test acc., compared with 67.48% for CPLUS.

At the larger budget highlighted by the reviewer, a separate CPLUS run retained all 128 anchors and 221/286 non-anchors. Retention was 84.30% including anchors and 77.27% excluding anchors.

3. Memorization

"How much of Acquisition and Retention is memorization?"

At epoch 25, all four methods reach 100% Acquisition, while Acquisition on unseen questions ranges from 62.86% ~ 64.69%. The held-out test set separates training-set fit from generalization. CPLUS achieves 63.45% held-out Acquisition while preserving 71.46% of non-anchor items and 73.48% of initially correct test items, showing preservation beyond the (potentialy memorized) reference anchors.

At epoch 25, all four methods corrected every training question (100% Acq.). On unseen questions that the base model initially answered incorrectly, they corrected 51.45–64.69% in the matched evaluation (Tab. R1). This shows that learning transferred to unseen (not memorized) questions. CPLUS also retained 293/410 (71.46%) initially correct non-anchor answers and 327/445 (73.48%) initially correct test answers, showing that the effect extended beyond the eight anchors.

We also reported a overlap audit of 1,536 generated prompts vs. 1,096 evaluation prompts in app. C.2: only one matched exactly.

AuthorsOpenReview ↗

Response to Reviewer jUC7 (2/3)

Response to Reviewer jUC7 (2/3)

(Continued)

4. Replay baselines

"How were the replay ratio and loss weight tuned?"

In Fig 5, we fixed the replay ratio and gave replay and new data equal loss weight (1.0).

Below, we report our quick experiment on qwen3-0.6B /CSQA (same as fig 5) to further study the effect of replay weights . The table reports Retention (%). The 1.0 column gives the original Fig. 5 results.

Probe replay

Anchors0.20.40.60.81.0 (Fig. 5)
446.3150.3853.4956.0458.13
847.7752.8556.8360.1462.84
1648.2253.7258.1361.7264.59
3249.2455.4660.4564.4967.70

Anchor-and-probe replay

Anchors0.20.40.60.81.0 (Fig. 5)
445.8948.6250.8352.3253.43
848.7453.4558.3559.1960.85
1649.1354.2959.6761.1662.18
3249.6455.1059.2262.6363.80

For both baselines, the replay weights showed quite monotonic patters, the original weight of 1.0 giving the highest Retention. Still, CPLUS outperformed them across all settings. We will update the manuscript to include these results.

5. Other update rules and adapters

"How does CPLUS compare with LoRA, L2-SP/EWC and WiSE-FT, and with A-GEM or OGD using the same blended gradient?"

Tables 6 and Figure 7 compare CPLUS with controls for update magnitude, mask structure, and gradient projection.

Table R2. Published update-rule comparisons on Qwen3-0.6B/CSQA at eight anchors. Table 6 reports absolute Retention at full Acquisition. Figure 7 reports gain over SFT in percent points and uses 32 probes for CPLUS.

MethodRetention in Table 6 (%)Gain over SFT in Figure 7 (points)
SFT with matched update magnitude45.45—
CPLUS with shuffled mask47.13—
A-GEM—+7.50
OGD-GTL—+3.92
CPLUS71.05+35.76

CPLUS substantially exceeds both the matched-magnitude SFT control and the shuffled-mask control. Shuffling preserves the mask coefficients while changing their assignment to parameters, showing that the mask's parameter alignment matters.

Figure 7 compares CPLUS with A-GEM and OGD-GTL using anchor gradients for the projection baselines. CPLUS improves Retention over SFT by 35.76 points, compared with 7.50 points for A-GEM and 3.92 points for OGD-GTL.

6. Anchor selection

"How are anchors selected?"

Anchors are selected uniformly at random without replacement from initially correct questions. Within each seed, all methods shared the same anchors.

7. Probe count and cost

"Why use N = 512 when N = 32 (Fig. 7) appears to give comparable gains?"

We used N = 512 as the fixed probe budget for Fig. 3. FIg. 7 shows that N = 32 already improves Retention at eight anchors across all three Qwen3 sizes, showing that CPLUS is effective with substantially fewer probes.

AuthorsOpenReview ↗

Response to Reviewer jUC7 (3/3)

Response to Reviewer jUC7 (3/3)

(continued)

8. Seed counts and differences across figures

At eight CommonsenseQA anchors, Figures 1a and 3 report the same five-run mean of 74.16% for Qwen3-0.6B. Figure 5 reports 72.33% across three seeds. Within each seed, all three methods used the same 512 probes. Mixed replay and CPLUS also used the same eight anchors. Tables 1 and 6 report separate single-run results of 79.96% and 71.05%, respectively.

Thank you for flagging Figures 8 and 12. Both track forgetting epoch by epoch. We identified two errors. The anchor (A=8) limit also restricted the probe gradient to 8 of the intended 512 self-probes, resulting in the 66.46% scores. We corrected both figures again (66.46 -> 71.05) and the full 512-probe CPLUS measurements from Table 6. CPLUS Retention fell to 30.14% after epoch 1 and recovered to 71.05% by epoch 25.

For clarity, we will update the manuscript with both corrected fig 8 and 12 (marked with a blue box), state the exact training settings and seed counts in the captions, and release the per-epoch values shown before and after correction.


We will shortly update the manuscript with the added results and hightlight the added results in blue

Reviewer_jUC7OpenReview ↗

he rebuttal strengthens the evidence but my concerns remain

Thank you for your detailed response and the additional experiments. The new results from the test set have dispelled some of my doubts. Specifying a fixed checkpoint explains the model choice. Table R1 shows that the increase is not an artifact of including reference points in retention: retention without reference points is 71.46% for CPL and 49.51%, and test retention is 73.48% for CPL and 51.01%. It is also evident that, in this setup, the mask does not limit generalization to a new task: the learning rate on the test set is 63.45%, compared to 62.86–64.69% for the other methods.

My main reservations, however, remain unchanged: Reference values for repetitions. The main claim—that self-tests are more useful as gradients than as data for reproduction—is still being tested exclusively in comparison with reproduction using hard labels. The new study covers repetition weights in the range of 0.2–1.0. Retention increases monotonically up to 1.0, the highest value tested; therefore, before concluding that 1.0 is the optimal value, weights above 1.0 should be investigated. Logit-based repetition was not evaluated for the same reference points and probes (DER++/X-DER or KL distillation to the base model). Table R1 also shows that reproducing 8 anchor points helps non-anchor points to a lesser extent than the standard SFT method (47.80% compared to 49.51%), which is consistent with the overfitting of simple reproduction to a small buffer and confirms the need to use stronger reference models for reproductions. The main claim—that in-house probes are more useful as gradients than as data for reconstruction—is still being verified exclusively in comparison with hard-label reconstruction. The new study covers reconstruction weights in the range of 0.2–1.0. Retention increases monotonically up to 1.0, the highest value tested; therefore, weights above 1.0 must be examined before concluding that 1.0 is the optimal value. Logit-based reconstruction for the same anchor points and probes (DER++/X-DER or KL distillation to the baseline model) was not evaluated. Table R1 also shows that reconstructing 8 anchor points helps non-anchor points to a lesser extent than the standard SFT method (47.80% compared to 49.51%), which is consistent with naive reconstruction overfitting to a small buffer and confirms the need for stronger repetition-based base models. The LoRA, L2-SP/EWC, and WiSE-FT models were not included. The results for the A-GEM and OGD-GTL models shown in Fig. 7 use only anchor point gradients. A projection-based foundation model is still needed, one that uses the same anchor point gradient and probes to separate the probe’s contribution from that of the coordinate mask.

Single runs of the same configuration yield results of 79.96% (Table 1) and 71.05% (Table 6), and the SFT index reaches 49.28% in a new run with an initial value of 0, compared to approximately 41% in Fig. 1a, 3, and 5. The spread of this value makes single-grain results (e.g., +1.25 pp for EMA smoothing) impossible to interpret, so multi-grain versions of Tables 1, 5, and 6 are still needed. The error that limited the probe gradient to 8 out of 512 probes in Figs. 8 and 12 also requires confirmation that no other reported result used the coding affected by this error. The response mentioned two errors but described only one.

The choice of N = 512 remains unjustified, and its cost has not been specified. The runs affected by the error (8 probes, 66.46%) and Fig. 7 (32 probes) suggest that even a few probes account for most of the gain. Excluding probes above the N threshold, at a cost that includes probe generation, would help clarify this.

The new study includes one model, one benchmark test, one anchor budget, and, it appears, one seed. For A = 128, retention outside the anchor is reported only for CPLUS, although the anchor-inclusion concern applies to all anchor-based methods. Some minor comments Validation-selected SFT was chosen by validation Acquisition only. Overall validation accuracy would be the standard criterion. The response cites 51.45% held-out Acquisition, which does not appear in Table R1. The A = 128 counts sum to 414, not 418, in past examples. Please clarify what “Acquisition-based run filtering” in App. E.5 refers to: were any runs discarded? Overall, the rebuttal strengthens the evidence that CPLUS preserves behavior beyond the anchors. However, it does not test the central gradients-versus-replay claim against strong rehearsal baselines, and the concerns about variance remain. I therefore keep my score. Presenting the results of a logit model-based repetition on the same probes, along with ablations from multiple starting points, may lead me to reconsider my final assessment.

Reviewer_5jaROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CPLUS addresses catastrophic forgetting in continual learning with limited historical data by introducing self-probe and past-sample gradients to scale down parameter updates that conflict with prior behavior. Experiments with five language models on four benchmarks show three results.

Critical strengths
  1. The motivation is well justified. Through the two sets of experiments in Figure 1, the authors identify two challenges faced by existing continual learning methods when access to historical data is limited, motivating the proposed Self-Probing approach.
  2. CPLUS demonstrates clear advantages when only a very small number of historical samples are available. It more effectively preserves the samples that the pretrained model could originally answer correctly than previous methods, while still fitting all new training samples.
Critical weaknesses
  1. The novelty of the paper is limited, as it mainly combines three existing methods: self-probes, gradient blending, and smooth update masking.
  2. CPLUS builds on CPL, which appears to be available as a preprint rather than a peer-reviewed publication. While this does not invalidate the proposed method, it makes a careful assessment of the inherited assumptions and baseline implementation important.
Questions for authors
  1. As noted in Weakness 1, the rationale for each component and the complementary relationships among the three components require further clarification.
  2. As noted in Weakness 2, the evaluation metrics used in CPLUS (Retention and Acquisition) are not widely established in the field. I encourage the authors to evaluate their method using well-established metrics commonly adopted in continual learning.
  3. The rationale for assigning a fixed weight of 0.5 in the gradient blending strategy needs further clarification. When the Self-Probe gradient has a much larger than the anchor gradient, the results may be dominated by the generated samples, and vice versa.
2 comments
AuthorsOpenReview ↗

Response to Reviewer 5jaR

Response to Reviewer 5jaR

Thank you for recognizing CPLUS’s strength in a realistic setting. Below, we respond to each of your concerns.

1. What does CPLUS contribute?

"The novelty of the paper is limited, as it mainly combines three existing methods..."

Our main finding is that the same generated data is more useful for continual learning (CL) as gradients rather than as exemplar replay data.

  • Fig. 5 tests this on Qwen3-0.6B CSQA. With just 8 anchors, Retention was 62.84% when replaying the generated data, 60.85% when replaying anchors and generated data together, and 72.33% with CPLUS. CPLUS outperforms both replay methods (and all others baselines) at all four anchor budgets (see Fig. 3)

2. What does CPLUS inherit from CPL?

"CPLUS builds on CPL, which appears to be available as a preprint .. While this does not invalidate the proposed method, it makes a careful assessment of the inherited assumptions and baseline implementation important."

CPL suggests a simple & cost effective way to achieve 'zero-forgetting' but with ** full access to past data**. Empirically, its effectiveness drops significantly as fewer past data are available (Figure 1a).

CPLUS addresses this limitation by using gradients from data generated by the frozen model to guide the update.

At a matched protocol (e.g., epochs, the same real data, learning rate, etc.), CPLUS outperforms all compared baselines including CPL, Replay, etc, especially at limited past-data scenarios. e.g., at four CSQA anchors, Retention was 41.55% for CPL and 74.12% for CPLUS, both at full Acquisition (Figure 3). We also find a scaling trend where larger models benefit more from the self-probe gradients (Figure 10)

3. Why are the three components useful together?

"As noted in Weakness 1, the rationale for each component and the complementary relationships among the three components require further clarification."

Self-probing supplies generated data beyond the few real past data. Gradient blending lets both sources guide each update. Smoothing makes the decision to update a parameter less sensitive to a single training step (makes CPLUS so much stable under limited data). In our ablation (Tab. 1), Retention was 55.13% with anchor gradients alone, 71.49% with gradients from generated data alone, 78.71% with both and no smoothing, and 79.96% with full CPLUS at matched 100% new-task acquisition.

4. How does CPLUS perform on standard CL metrics?

"I encourage the authors to evaluate their method using well-established metrics commonly adopted in continual learning."

fig. 6 and tab. 7 report Average Task Accuracy over three seeds. CPLUS led at every stage after the shared CSQA checkpoint, reaching 83.94% after ARC-C, 80.86% after MMLU, and 78.87% after MedQA. Replay, the strongest final baseline, reached 69.73%. CPLUS also finished with the highest accuracy on each earlier task (tab. 8). Mean backward transfer, computed from Figure 6, was −6.55 points for CPLUS versus −13.26 for replay.

We reported Retention and Acquisition because a model can learn new answers while losing previously correct ones. Overall accuracy can hide that loss. Table 4 gives a concrete example. Changing which eight examples served as anchors raised overall CSQA accuracy from 40.15% to 45.35%, while Retention fell from 83.49% to 80.86%.

5. Why use equal weights if one gradient can be larger?

"The rationale for assigning a fixed weight of 0.5 in the gradient blending strategy needs further clarification..."

We average each source separately, so the amount of generated data does not determine its weight. we agree that when the two gradients point in opposite directions for a parameter, the larger one determines the decision. We chose 0.5 because equal weighting gave the highest Retention among the tested weights at all all tested settings (Fig. 11). We also tested whether scaling the two gradients to the same overall size helped: in the ablation, Retention was 79.94% with scaling and 79.96% without it, both at full Acquisition (Table 1), hence did not justify the added complexity.

Thank you for the additional experiments and clarifications. While I appreciate the authors' efforts, my main concern regarding novelty remains. The additional results demonstrate the effectiveness of combining existing components, but do not sufficiently establish the methodological originality of CPLUS or its conceptual distinction from existing gradient-based continual learning approaches. The justification for the fixed gradient blending weight of 0.5 also remains somewhat limited. I acknowledge the improved experimental evidence. I will take other reviewers' opinions into consideration and tentatively maintain my original rating.

Reviewer_WqQbOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper considers continual learning of language models in the setting where the number of samples from the old data is limited. First, the authors propose to generate new pseudo-data from the old data available, by taking advantage of the language models’ in-context and instruction following capabilities (provide 2 data points to the model and generate a new input, which is then labelled by the model).

Next, given new data, at the beginning of each training epoch, the authors compute a reference gradient over the old data (gradient is computed on true old data samples and on pseudo-data separately, and then averaged). When the optimizer proposes a new parameter update, the method computes a binary mask over each parameter, based on whether the proposed update and the reference partial derivative point in different directions. This mask is then combined with masks computed on previous batches (exponential average), resulting in a real-valued mask with values between [0,1]. The real-valued mask is multiplied by the proposed updates to produce the final updates that are applied to the model. This aims to reduce (or prevent) model updates in directions that would induce forgetting.

Critical strengths

The method is shown to outperform previous baselines across numerous different language models.

Critical weaknesses

The paper’s novelty appears to be severely limited. The paper builds on top of CPL. Since CPL’s performance deteriorates with data, the method generates pseudo data which is not a novel idea. From what I can tell, two things that are new to this method are: 1) combining the gradient of old data and pseudo data by averaging 2) using an exponential mask instead of a binary mask.

Questions for authors

q1: Since your method relies on model-labelled pseudo-data, is it fair to say that it is restricted to preserving capabilities that are already well approximated? E.g. if the model achieves only 20% accuracy on such dataset, the method would not be more effective at preserving performance than CPL.

2 comments
AuthorsOpenReview ↗

Response to Reviewer WqQb

Response to Reviewer WqQb

Thank you for recognizing the contributions of our work and for your feedback! Below, we have addressed your concerns.

1. Contribution beyond CPL

"..two things that are new to this method are: 1) combining the gradient of old data and pseudo data by averaging 2) using an exponential mask instead of a binary mask.."

We agree that CPL showed how gradients from past data can identify conflicting parameter updates. Our contribution is to show that when real past data is limited, a model’s own generations (probes) can be used better as gradient signals than as training data for continual learning. We believe this is a very common scenario for users of released models, who have limited access to the original training data (lines 32–37).

We'd like to clarify that this is not just 'model-labeled', but fully synthetic (both question and answer are generated), which is significant given that synthetic data had shown clear limitations in continual learning.

Empirically, we compare Self-Probe Replay (training on these generated data) vs. Our CPLUS (using generated data's gradients for selective updates) across several experiments.

  • fig. 3 compares these methods at the same anchor budget. With just 4 anchors, Qwen3-0.6B achieved 41.55% Retention with CPL, 58.03% with Self-Probe Replay, and 74.12% with CPLUS. CPLUS exceeded Self-Probe Replay in all 24 Qwen3 settings with four or eight anchors.
  • Fig. 5 also tests how the probes are used. Given the same 8 anchors and the same probes, replaying both yielded 60.85% Retention, while using their gradients to guide updates yielded 72.33%. Both methods corrected all adaptation examples.

Regarding ema masks:

  • The ablation in Tab. 1 shows the role of smoothing. Here, Retention was 55.13% with anchor gradients alone, 78.71% with anchor and probe gradients w/o smoothing, and 79.96% with the full CPLUS. Most of the gain was already present before smoothing was added.

2. Low base-model accuracy

"..is it fair to say that it is restricted to preserving capabilities that are already well approximated? E.g. if the model achieves only 20% accuracy on such dataset, the method would not be more effective at preserving performance than CPL."

Even at 20% base acc., (only 20/100 initially correct): CPLUS can still improve on CPL if it preserves more of those 20 answers during training. The self-probes record the base model’s behavior, and labeled new data provide the supervision for correcting errors.

Fig. 3 exactly gives an low-accuracy scenario that you've mentioned. On MedQA, Qwen3-0.6B initially answered only about 25% of the questions correctly in the initial evaluation. Across fig. 3, CPLUS retained more previously correct answers than CPL at just four anchors (18.96% vs. 11.37%) and eight anchors (31.75% vs. 27.14%). Both methods reached 100% Acquisition (learning the 75% initially wrong samples). We believe this alone shows CPLUS's strength over CPL even when the base model answers most questions incorrectly.

Thank you for your response. I acknowledge that the empirical results are informative and point to the promise of such an approach. At the same time, due to the, what appears to be, very limited methodological novelty of the paper, I cannot recommend acceptance to ICLR. Therefore, I will raise my score from 1 to 2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.