acceptodds
Under review as a conference paper at ICLR 2027

Learning What to Preserve: Parameter Attribution for Continual Learning in Large Language Models

Abstract

Large language models (LLMs) often suffer from catastrophic forgetting in continual learning: after learning a subsequent new task, they perform worse on earlier tasks. Existing replay and regularization based methods constrain model updates to mitigate forgetting, but may not reliably identify parameters that should be preserved or adapted. Common importance measures, such as Fisher information and gradient-based signals, capture local sensitivity rather than parameter contributions to model predictions. We propose a parameter-level attribution framework that traces output logits to individual parameters, enabling a mechanistic analysis of how the locations of important parameters overlap across tasks. We train LLMs independently on different tasks, analyze the resulting parameter importance patterns, and find that dissimilar tasks show lower overlap in LLMs layers. Motivated by this finding, we estimate parameter importance within each layer and selectively constrain updates to parameters important for previous tasks, while allowing less important parameters to adapt to new tasks. Experiments under both full fine-tuning and LoRA fine-tuning show that our method consistently outperforms all baselines across task settings on Llama and Qwen models, improving performance by at least 2 percentage points over replay-based methods and 3 percentage points over parameter importance based methods.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Continual Learning, Large Language Models, Parameter Attribution, Layer-wise Relevance Propagation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_PndwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AGFT is a continual fine-tuning method for LLMs. It first fine-tunes a separate copy of the model on each task and attributes the predicted logit to individual parameters. During sequential training, it then scales down gradients on parameters that were important to earlier tasks. It is evaluated on different continual-learning benchmarks and on Llama and Qwen backbones of different sizes.

Critical strengths
  1. The evaluation is broad, covering more than one benchmark, backbones of different families and sizes, both full fine-tuning and LoRA, and baselines from several kinds of continual-learning methods.
  2. The attribution scores pass a clear sanity check: masking the top-ranked parameters hurts much more than masking random ones.
Critical weaknesses
  1. The gains rest on a single run. All experiments use one fixed seed, with no variance reported. On TRACE, the lead over the strongest replay baseline is under a point for both fully fine-tuned models.
  2. The paper calls the starting point a pretrained or base LLM, but one backbone is instruction-tuned while the others appear to be base models. This choice is not discussed, although an instruction-tuned model may forget differently from a base one. The smaller models also use full fine-tuning and the larger ones LoRA, so the effect of model size cannot be told apart from that of the fine-tuning mode.
Questions for authors
  1. Can you report mean and variance over several seeds?
  2. Which backbones are base models and which are instruction-tuned, and why are they mixed?
  3. How does AGFT do with LoRA on the smaller models, or with full fine-tuning on a larger one?
4 comments
AuthorsOpenReview ↗

Response to Reviewer

Thank you for taking the time to review our paper!

Q1: The paper calls the starting point a pretrained or base LLM, but one backbone is instruction-tuned while the others appear to be base models. Which backbones are base models and which are instruction-tuned, and why are they mixed?

A1: Regarding model initialization, we use four backbones: Llama-3.2-3B-Instruct, Qwen2.5-3B, Llama-2-7B, and Qwen2.5-7B. Among them, only Llama-3.2-3B-Instruct is instruction-tuned, while the other three are base models. Our intention is to evaluate AGFT across different model families and initialization settings. Using heterogeneous initialization settings is also not without precedent in recent continual learning research. For example, [1] evaluates Llama-3.2-3B (base) on Merge and Llama-3.2-3B-Instruct on TRACE. These studies demonstrate the relevance of evaluating continual learning methods across different model initialization settings.

[1] Merge to Remember: Sharpness-Aware Isotropic Merging for Continual Learning. Forty-third International Conference on Machine Learning.

Q2: The smaller models also use full fine-tuning and the larger ones LoRA. How does AGFT do with LoRA on the smaller models, or with full fine-tuning on a larger one?

A2: Our experiments were conducted on NVIDIA A6000 GPUs with 48 GB of memory. Under our experimental settings, full-parameter fine-tuning of 7B models resulted in CUDA out-of-memory errors. We conducted additional experiments using LoRA fine-tuning on two models, Llama-3.2-3B-Instruct and Qwen2.5-3B on concept-1k dataset. As shown in Table 1, AGFT outperforms all competing baselines. Specifically, compared with the strongest baseline, DER++, AGFT achieves improvements of 2.082 and 1.773 points on the two models, respectively. These additional results demonstrate that AGFT remains effective when LoRA fine-tuning is applied to smaller models

Table 1. Performance Comparison of Different Methods with LoRA Fine-Tuning on Concept-1K.

MethodLlama-3.2-3B-InstructQwen2.5-3b
AGFT20.37123.657
SEQ17.55720.460
Freeze17.34720.262
EWC18.03519.0972
SNIP17.455721.342
wanda17.19620.317
Replay18.196621.358
DER++18.288621.884

Q3: The gains rest on a single run. All experiments use one fixed seed, with no variance reported. On TRACE, the lead over the strongest replay baseline is under a point for both fully fine-tuned models. Can you report mean and variance over several seeds?

A3: Due to limited computational resources and the short rebuttal timeline, we conducted additional multi-seed experiments on Concept-1K across all four model configurations and report the corresponding mean and standard deviation. However, running multiple seeds on TRACE is considerably more expensive, as training each method on this benchmark requires substantially more time. Therefore, we were unable to complete multi-seed experiments for TRACE within the rebuttal period.

Table 2. Results on Concept-1K (Mean ± Std over Two Seeds).

MethodQwen2.5-3B (Full)Qwen2.5-7B (LoRA)Llama-3.2-3B (Full)Llama-2-7B (LoRA)
AGFT (Ours)28.66 ± 0.5324.36 ± 0.1725.88 ± 0.6422.33 ± 1.94
SEQ21.37 ± 0.1321.91 ± 1.8720.06 ± 0.2319.60 ± 2.14
Freeze21.21 ± 0.0421.60 ± 0.1519.43 ± 0.7517.85 ± 0.44
EWC18.36 ± 0.0819.17 ± 0.3420.56 ± 0.9919.40 ± 0.53
SNIP25.71 ± 0.5523.17 ± 0.1123.09 ± 0.4219.12 ± 0.12
Wanda21.83 ± 0.2823.23 ± 0.0721.71 ± 0.2820.44 ± 1.76
Replay21.41 ± 0.0023.54 ± 0.0419.98 ± 0.3320.31 ± 1.51
DER++22.12 ± 0.4323.53 ± 0.1423.84 ± 0.0920.48 ± 3.03

As shown in Table 2, AGFT achieves the highest average performance across all four model configurations. Compared with the strongest baseline in each configuration, AGFT improves the average score by 2.95, 0.82, 2.04, and 1.86 points on Qwen2.5-3B, Qwen2.5-7B, Llama-3.2-3B, and Llama-2-7B, respectively, yielding an average improvement of 1.92 points across all configurations. These results suggest that AGFT maintains its performance advantage across different model families and fine-tuning settings.

Regarding TRACE, it is important to note that Replay and AGFT operate under different data access settings. Replay stores and reuses training data from previous tasks, whereas AGFT does not require access to any historical training samples. AGFT achieves better performance than Replay, even though the performance margin is small for the full fine-tuning models. This demonstrates that AGFT can effectively preserve previously learned knowledge without relying on storing or replaying historical task data, highlighting its practical advantage in continual learning scenarios where access to previous-task data is limited or prohibited.

Thank you for the response.

  1. The two-seed results on Concept-1K help, and AGFT stays ahead in all four settings. TRACE is still a single run, and I understand the cost issue you mention.
  2. The answer on base and instruction-tuned backbones is clear, and the memory limit explains why the 7B models use LoRA. Model size and fine-tuning mode are still tied together, though, so their effects cannot be separated.
  3. Protecting important parameters is already common in continual learning, for example EWC, and Child-Tuning (Xu et al., EMNLP 2021) uses a similar gradient mask in standard fine-tuning. The main new part of AGFT is the attribution score, so I see the novelty as limited. For these reasons I keep my score.
AuthorsOpenReview ↗

Reply to Reviewer (1)

Q1: Protecting important parameters is already common in continual learning, for example EWC, and Child-Tuning (Xu et al., EMNLP 2021) uses a similar gradient mask in standard fine-tuning. The main new part of AGFT is the attribution score, so I see the novelty as limited.

A1: Thanks for your reply. We would like to clarify that the novelty of AGFT does not lie in gradient masking itself, but in how parameter importance is defined, estimated, analyzed, and exploited for continual learning in LLMs.

(1) Difference from EWC [1]. EWC estimates parameter importance using the diagonal Fisher Information Matrix (FIM). It measures the local sensitivity of the predictive distribution to parameter perturbations and uses a quadratic penalty to discourage changes to important parameters. In contrast, AGFT quantifies parameter importance from a fundamentally different perspective: the direct contribution of individual parameters to next-token predictions. Specifically, we extend Layer-wise Relevance Propagation (LRP) to Transformer architectures and derive parameter-level relevance propagation rules for both self-attention and FFN modules. Thus, AGFT identifies which individual parameters contribute to task-specific predictions, rather than relying on gradient-based sensitivity as a proxy for importance.

Moreover, estimating Fisher importance introduces additional computational and memory overhead, particularly for LLMs with large parameter spaces and long input sequences. In our experiments, we encountered CUDA out-of-memory (OOM) errors when estimating Fisher information on long-sequence examples from the TRACE dataset. This limitation highlights an important scalability challenge of Fisher-based importance estimation for continual learning in LLMs.

More importantly, on concept-1k dataset, as reported in Table 2 of our previous response, AGFT consistently outperforms EWC across all four model configurations, achieving improvements of 10.30, 5.19, 5.32, and 2.93 percentage points, respectively, with an average improvement of 5.94 percentage points across the four settings. These results demonstrate the effectiveness of our attribution-based parameter importance estimation compared with Fisher-based importance estimation.

(2) Difference from Child-Tuning. Child-Tuning employs binary gradient masks to select a subset of parameters for updating, whereas AGFT derives continuous, element-wise gradient masks from attribution-based parameter importance scores. However, gradient masking itself is not the primary methodological novelty of our work. Our key novelty lies in identifying task-relevant parameters through LRP-based attribution and leveraging their importance to guide continual learning.

To further disentangle the effects of gradient modulation from those of parameter importance estimation, we conducted additional controlled experiments in response to Reviewer xd5j. we conducted two additional controlled experiments:

  • 1 Random Importance: We randomly generate parameter importance scores and apply the same normalization, gradient mask construction, and parameter update procedure as AGFT.

  • 2 Shuffled Importance: We shuffle AGFT's estimated importance scores across model parameters, disrupting their correspondence with specific parameter locations, while retaining the same score distribution. We then apply the same gradient modulation procedure as AGFT.

All other experimental settings remain identical to AGFT. These comparisons allow us to examine whether the performance gains result from accurately identifying important parameters or merely from applying gradient masking.

Table 3. Controlled comparison on Concept-1K.

MethodQwen2.5-3B (Full)Qwen2.5-7B (LoRA)Llama-3.2-3B (Full)Llama-2-7B (LoRA)
AGFT (Ours)28.29224.47626.33623.707
Random24.70021.32820.87216.666
Shuffle27.65322.73823.91122.371

Table 4. Controlled comparison on TRACE.

MethodQwen2.5-3B (Full)Qwen2.5-7B (LoRA)Llama-3.2-3B (Full)Llama-2-7B (LoRA)
AGFT (Ours)54.92254.74554.88047.775
Random53.02052.77849.43139.661
Shuffle53.47754.12751.93946.445

Results and Analysis. Tables 3 and 4 show that AGFT outperforms Random and Shuffled Importance on all four models across both benchmarks. On Concept-1K, AGFT averages 25.703, exceeding Random and Shuffled Importance by 4.811 and 1.535 points, respectively. On TRACE, AGFT averages 53.081, with gains of 4.358 and 1.584 points, respectively. These results suggest that AGFT's gains from parameter importance scores, not gradient masking alone.

AuthorsOpenReview ↗

Reply to Reviewer (2)

(3) Further clarification of AGFT's novelty. We would like to emphasize three contributions that go beyond applying an existing gradient mask:

Parameter-level attribution for LLMs. We derive LRP-based relevance propagation rules for Transformer architectures, covering both self-attention and FFN modules, to quantify the contribution of individual parameters to next-token predictions.

Analysis of task-dependent parameter overlap. We investigate the overlap of important parameters across different tasks and Transformer layers, revealing how parameter importance is shared among related tasks and differs across dissimilar tasks.

Attribution-guided continual learning. We leverage the estimated parameter-level contributions to characterize knowledge acquired from previous tasks and construct historical importance priors. These priors enable selective protection of parameters critical to previously learned behaviors while preserving the flexibility of less important parameters for new-task adaptation.

In summary, AGFT makes a contribution by introducing Transformer-specific parameter-level attribution, uncovering task-dependent parameter importance patterns, and leveraging these attribution scores to guide knowledge preservation in continual LLM fine-tuning. Our additional controlled experiments further demonstrate that the performance gains cannot be attributed merely to gradient masking, but critically depend on accurately identifying task-important parameter locations through our attribution mechanism.

Reviewer_xd5jOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies catastrophic forgetting in continual learning for large language models and proposes attribution-guided fine-tuning (AGFT). This paper uses layer-wise relevance propagation to trace predicted next-token logits to individual parameters and analyze how important parameter locations overlap across tasks. Motivated by the finding that dissimilar tasks show lower overlap across LLM layers, AGFT selectively constrains updates to parameters important for previous tasks while allowing less important parameters to adapt. Experiments on Concept-1K and TRACE with Llama and Qwen models report consistent improvements over the evaluated baselines under both full and LoRA fine-tuning.

Critical strengths
  • This paper provides a mathematically explicit parameter-level attribution framework. The authors derive relevance propagation rules for FFN and MHA modules, extend the framework to LoRA, and establish relevance conservation properties for the proposed rules. These derivations provide a structured basis for analyzing task-specific parameter importance across Transformer layers.
  • The experimental results are encouraging across multiple settings. AGFT consistently outperforms the evaluated baselines on Concept-1K and TRACE across four models from the Llama and Qwen families, covering both full and LoRA fine-tuning. These results suggest that the proposed approach has potential across different model families and continual learning tasks.
Critical weaknesses
  • The connection between parameter attribution and reduced forgetting is insufficiently established. Although the experiments report consistent improvements, they lack controlled comparisons that isolate the contribution of the proposed attribution scores and a clear analysis of forgetting across tasks. This makes it difficult to determine whether the gains result from the proposed parameter-importance estimates.

  • The methodological novelty is limited relative to prior work. ECQx [1] already uses LRP for weight-level relevance, RNF [2] employs LRP-based importance to preserve previously learned knowledge, and HAT [3] uses cumulative task masks to modulate gradients. Although the paper provides detailed theoretical derivations, its core methodological ideas are not entirely new.

[1] ECQx: Explainability-Driven Quantization for Low-Bit and Sparse DNNs.

[2] Explain to Not Forget: Defending Against Catastrophic Forgetting with XAI.

[3] Overcoming Catastrophic Forgetting with Hard Attention to the Task.

Questions for authors
  • Relevance conservation does not establish that the identified parameters are critical for preventing forgetting. Can the authors provide controlled comparisons to verify that the performance gains come from the proposed parameter importance scores rather than from gradient masking alone?
  • The experiments mainly report overall continual learning performance. Can the authors provide per-task results across training stages or explicit forgetting metrics to demonstrate that AGFT improves the retention of previously learned tasks?
  • Are the important parameter locations identified in independently fine-tuned single-task models consistent with those identified in the continual learning model?
7 comments
AuthorsOpenReview ↗

Response to Reviewer (1)

Thank you for taking the time to review our paper!

Q1: The experiments mainly report overall continual learning performance. Can the authors provide per-task results across training stages or explicit forgetting metrics to demonstrate that AGFT improves the retention of previously learned tasks?

A1: We thank the reviewer for this valuable suggestion. To explicitly evaluate the retention of previously learned knowledge, we additionally compute the per-task forgetting scores and average forgetting across all seven previously learned tasks. We adopt a non-negative forgetting measure defined as: Fi(T)=max⁡(0,max⁡t∈i,…,T−1At,i−AT,i)F_i^{(T)}=\max\left(0,\max_{t\in{i,\ldots,T-1}}A_{t,i}-A_{T,i}\right), where At,iA_{t,i} denotes the model's performance on task ii after training on task tt. This metric measures the performance degradation from the best previously achieved performance to the final performance.

Table1: Average Forgetting across Four Model Configurations on trace dataset (Lower is Better).

MethodQwen2.5-3BQwen2.5-7BLlama-3.2-3BLlama-2-7B
SEQ8.7369.68615.1599.122
Replay7.7778.4429.4118.903
Freeze7.97411.4539.33610.483
SNIP7.93014.7048.99610.800
Wanda6.1609.1429.13210.169
DER++5.6269.1326.8399.028
AGFT (Ours)5.5204.6516.1916.787

Table2: Average Forgetting across Four Model Configurations on concept dataset (Lower is Better).

MethodQwen2.5-3BQwen2.5-7BLlama-3.2-3BLlama-2-7B
SEQ9.34311.8446.4658.610
Replay9.1689.3408.9038.104
Freeze9.43011.0977.57312.183
EWC12.54813.7236.81511.038
SNIP5.6919.5613.68910.211
Wanda9.0399.6324.9406.980
DER++8.1129.0112.3008.120
AGFT (Ours)3.6518.0561.9586.260

Results and Analysis. Tables 1 and 2 report the average forgetting scores on TRACE and Concept, respectively, across four model configurations. Lower values indicate better knowledge retention. As shown in the tables, AGFT consistently achieves the lowest average forgetting across all four model configurations on both benchmarks, demonstrating its effectiveness in mitigating catastrophic forgetting.

Q2: They lack controlled comparisons that isolate the contribution of the proposed attribution scores and a clear analysis of forgetting across tasks. This makes it difficult to determine whether the gains result from the proposed parameter-importance estimates. Can the authors provide controlled comparisons to verify that the performance gains come from the proposed parameter importance scores rather than from gradient masking alone?

A2: To isolate the contribution of our proposed parameter importance scores from the effect of gradient masking itself, we conducted two additional controlled experiments:

    1. Random Importance: We randomly generate parameter importance scores and follow the same procedure as AGFT to normalize these scores, construct gradient masks, and modulate parameter updates during continual learning.
    1. Shuffled Importance: We shuffle the parameter importance scores estimated by AGFT across model parameters, thereby disrupting the correspondence between importance scores and their associated parameters. We then construct gradient masks and perform continual learning using the same procedure as AGFT.

For both experiments, we keep the remaining experimental settings identical to AGFT. Therefore, these comparisons allow us to examine whether the performance gains originate from the parameter-specific importance estimated by our attribution method, rather than simply from applying gradient masking.

Table 3. Controlled comparison on Concept-1K.

MethodQwen2.5-3B (Full)Qwen2.5-7B (LoRA)Llama-3.2-3B (Full)Llama-2-7B (LoRA)
AGFT (Ours)28.29224.47626.33623.707
Random24.70021.32820.87216.666
Shuffle27.65322.73823.91122.371

Table 4. Controlled comparison on TRACE.

MethodQwen2.5-3B (Full)Qwen2.5-7B (LoRA)Llama-3.2-3B (Full)Llama-2-7B (LoRA)
AGFT (Ours)54.92254.74554.88047.775
Random53.02052.77849.43139.661
Shuffle53.47754.12751.93946.445

Results and Analysis. Tables 3 and 4 show that AGFT outperforms Random and Shuffled Importance on all four models across both benchmarks. On Concept-1K, AGFT averages 25.703, exceeding Random and Shuffled Importance by 4.811 and 1.535 points, respectively. On TRACE, AGFT averages 53.081, with gains of 4.358 and 1.584 points, respectively. These results suggest that AGFT's gains from parameter importance scores, not gradient masking alone.

AuthorsOpenReview ↗

Response to Reviewer (2)

Q3: The methodological novelty is limited relative to prior work. ECQx [1] already uses LRP for weight-level relevance, RNF [2] employs LRP-based importance to preserve previously learned knowledge, and HAT [3] uses cumulative task masks to modulate gradients. Although the paper provides detailed theoretical derivations, its core methodological ideas are not entirely new.

A3: We thank the reviewer for pointing out these related works. However, AGFT differs fundamentally from ECQx [1], RNF [2], and HAT [3] in terms of the attribution granularity, model architectures, and knowledge preservation mechanisms.

(1) Difference from ECQx [1]. ECQx applies LRP to estimate the importance of individual weights in linear layers and evaluates its method on conventional DNN architectures, such as MLPs, VGG, and ResNet. However, it does not address Transformer self-attention or estimate the importance of its query (Q), key (K), and value (V) projection parameters. In contrast, our work extends LRP attribution to LLMs, deriving attribution rules for both FFN and self-attention modules, including the Q, K, V, and O projection parameters.

(2) Difference from RNF [2]. Although RNF also employs LRP in continual learning, its importance estimation operates at the hidden neuron level rather than the individual parameter level. Specifically, RNF evaluates hidden-unit relevance, iteratively prunes less important neurons and their associated connections, and protects the remaining important neurons by freezing them or reducing their learning rates. Thus, RNF preserves previous knowledge through neuron-level pruning and freezing, rather than identifying the contributions of individual model parameters. In contrast, our method directly estimates parameter-level relevance in both the self-attention and FFN modules of LLMs and uses these scores to selectively constrain parameter updates. Furthermore, RNF was not designed or evaluated for LLMs or Transformer architectures.

(3) Difference from HAT [3]. HAT learns task-specific attention embeddings for neurons in each layer through backpropagation. Specifically, it constructs gradient masks for connections based on the learned attention values of neurons in two adjacent layers (i.e., the current and preceding layers). These attention embeddings are optimized during training with an additional regularization loss, making the resulting masks optimization-learned. In contrast, AGFT directly derives gradient masks from LRP-based parameter importance scores, without introducing additional learnable mask embeddings or mask regularization. Thus, our masks are attribution-derived and operate directly at the individual parameter level, rather than being learned from neuron-level attention vectors.

Our contributions can be summarized in three aspects:

(1) Parameter-level attribution for LLMs. We derive LRP-based relevance propagation rules for Transformer architectures, covering both self-attention and FFN modules, to quantify the contribution of individual parameters to next-token predictions.

(2) Analysis of task dependent parameter overlap. We investigate the overlap of important parameters across different tasks and Transformer layers, revealing how parameter importance is shared among related tasks and differs across dissimilar tasks.

(3) Attribution-guided continual learning. We leverage the estimated parameter importance as a signal to construct historical gradient masks, selectively constraining updates to parameters important for previous tasks while allowing less important parameters to adapt to new tasks.

Q4: Are the important parameter locations identified in independently fine-tuned models consistent with those identified in the continual learning model?

A4: We compare the top-10% important parameter locations in independently fine-tuned and continual-learning for Qwen2.5-3B models on TRACE dataset. Table 5 shows layer-wise and overall overlap.

Table 5. Top-10% Parameter Location Overlap on TRACE (Qwen2.5-3B, %) between independently fine-tuned and continual learning.

TaskLayer 0Layer 10Layer 20Layer 30Output Layer (35)Average (All Layers)
Task 033.9138.9061.8358.7751.4152.17
Task 149.8045.5852.0362.1650.6455.12
Task 265.2064.0556.9063.0652.8662.82
Task 353.6155.1153.4655.5247.5253.18
Task 458.7655.0155.8056.4043.3755.79
Task 559.8244.7044.1259.3346.1254.59
Task 668.2546.0342.6058.9244.2756.49

As shown in Table 5, the overlap remains above 52% for every task. This high overlap can be attributed to our gradient masking mechanism, which converts the importance scores estimated from independently fine-tuned models into masks that constrain updates to parameters important for previously learned tasks.

Reviewer_xd5jOpenReview ↗

Response to the Authors' Rebuttal

Thank you for the additional experiments. I have two remaining questions.

  1. Shuffled Importance outperforms SNIP in six of the eight settings (Tables 1 and 2 in the paper, Tables 3 and 4 in your response). Does this suggest that the gains come mainly from the reduced update magnitude induced by masking, rather than from the specific locations identified by the attribution method?

  2. Could you clarify how the parameter location overlap in Table 5 is calculated? My understanding is that Task 0 is trained without a mask, so its training procedure should match independent fine-tuning. Why is its overlap only 52.17%?

Reviewer_1WLQOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AGFT, a continual learning framework that estimates parameter relevance from predicted next-token logits in independently trained task models. It combines historical relevance scores into gradient gates intended to preserve earlier capabilities during sequential training. Experiments on two benchmarks and four Llama and Qwen backbones cover full and LoRA tuning, reporting the best aggregate scores in all eight configurations.

Critical strengths
  1. The method is clearly presented. Lemma 4.1 defines how relevance is distributed among activations, weights, and biases, with an extension to LoRA factors in Appendix A.4. Figures 1 and 10 further show that removing highly ranked parameters causes greater performance degradation than random removal, providing a direct check of the importance scores.

  2. The experiments demonstrate the effectiveness of AGFT. Tables 1 and 2 report the best aggregate scores across eight configurations on Llama and Qwen models, with an average gain of 2.077 points over DER++. The evaluations cover both full and LoRA tuning on two benchmarks with diverse tasks.

Critical weaknesses
  1. The attribution rules do not fully specify the evaluated FFN structure. Proposition 4.2 models an FFN with two weight matrices, yet Figures 6 and 13 analyze an additional gate projection without deriving its attribution.

  2. The mapping between task indices and datasets is unclear. Section 1 describes TRACE tasks 0 and 1 as stance classification and tasks 5 and 6 as numerical reasoning, while Table 3 lists only one stance task. This ambiguity weakens the interpretation that related tasks share important parameters.

  3. The comparisons do not isolate the benefit of the attribution method. Algorithm 1 adds independent model training and filtering of correct generations before importance estimation, but does not establish whether SNIP and Wanda receive the same settings. This leaves the source and computational cost of the gains in Tables 1 and 2 unresolved.

Questions for authors
  1. What relevance rules were implemented for the FFN gate projection and the split between the two inputs to g=LN(H+f)g=\mathrm{LN}(H+f)?

  2. Which datasets correspond to TRACE indices 0 through 6 in the overlap figures, and do these match the continual learning task stream?

  3. Under identical task models, attribution samples, normalization, and update rules, how does AGFT compare with SNIP and Wanda?

6 comments
AuthorsOpenReview ↗

Response to Reviewer (1)

Thank you for taking the time to review our paper!

Q1: The attribution rules do not fully specify the evaluated FFN structure. Proposition 4.2 models an FFN with two weight matrices, yet Figures 6 and 13 analyze an additional gate projection without deriving its attribution. What relevance rules were implemented for the FFN gate projection.

A1: We thank the reviewer for pointing out this ambiguity. In Proposition 4.2, we adopt a simplified two-matrix FFN formulation, where W1W_1 denotes the input projection and W2W_2 denotes the output projection. However, actual FFN architectures may differ across model families. Specifically, Llama-3.2-Instruct-3B adopts a gated MLP (SwiGLU) architecture with three projection matrices: gate_proj, up_proj, and down_proj. In this architecture, down_proj plays the role of W2W_2 in our simplified notation, while gate_proj and up_proj constitute two parallel input-side projections, with gate_proj serving the gating role analogous to W1W_1.

In this architecture, we first propagate the output relevance through down_proj (corresponding to W2W_2) using Lemma 4.1. Since the outputs of gate_proj and up_proj are combined through element-wise multiplication in the SwiGLU architecture, we equally distribute this relevance between the two branches. We then independently apply Lemma 4.1 to the two input-side linear projections to compute the corresponding relevance scores for gate_proj and up_proj.

Q2: What relevance rules were implemented for the split between the two inputs g=LN(H+f)g=LN(H+f)?

A2: We treat LayerNorm as an identity mapping for relevance propagation, meaning that the relevance is passed backward through LayerNorm without redistribution, i.e., R(g(l))=R(H(l)+f(l))R(g^{(l)})=R(H^{(l)}+f^{(l)}). After propagating through LayerNorm, we apply the standard LRP rule (Eq. 4) to the residual addition H(l)+f(l)H^{(l)}+f^{(l)} to distribute the relevance between its two inputs: $R(H^{(l)}) = \frac{H^{(l)}}{H^{(l)}+f^{(l)}} \odot R(g^{(l)})$ and $R(f^{(l)}) = \frac{f^{(l)}}{H^{(l)}+f^{(l)}} \odot R(g^{(l)})$.

Q3: The mapping between task indices and datasets is unclear. Section 1 describes TRACE tasks 0 and 1 as stance classification and tasks 5 and 6 as numerical reasoning, while Table 3 lists only one stance task. This ambiguity weakens the interpretation that related tasks share important parameters. Which datasets correspond to TRACE indices 0 through 6 in the overlap figures, and do these match the continual learning task stream?

A3: We thank the reviewer for identifying this inconsistency. We apologize for the unclear task indexing and dataset description in the manuscript. Specifically, the overlap figures use zero-based indexing (Tasks 0–6) for the seven datasets selected from the original eight-task TRACE benchmark, following their original task order. The correspondence is as follows:

  • Task 0 C-STANCE: Chinese social-media stance detection, which identifies the stance (e.g., favor, against, or neutral) toward a given target.
  • Task 1 FOMC: Financial-domain stance classification, which determines whether monetary policy statements reflect a hawkish, dovish, or neutral stance.
  • Task 2 MeetingBank: Long-context meeting summarization, which generates concise summaries of meeting transcripts and their key decisions.
  • Task 3 PY150: Python code completion, which requires the model to generate subsequent code given a partial Python program.
  • Task 4 ScienceQA: Science question answering, which evaluates the model's ability to answer science-related questions with supporting reasoning.
  • Task 5 NumGLUE-cm: Commonsense numerical reasoning, which involves solving arithmetic problems presented in everyday contexts.
  • Task 6 NumGLUE-ds: Domain-specific numerical reasoning, which requires arithmetic reasoning grounded in domain-specific and grade-school scientific knowledge.

Consistency with the Continual Learning Task Stream. These seven datasets follow exactly the same sequential order in our continual learning task stream as in the overlap analysis. Therefore, the task indices in the overlap figures directly correspond to the task order used in continual learning.

Clarification and Manuscript Revision. The confusion arises because Table 3 and Appendix A.7 do not consistently reflect the seven datasets used in these analyses: C-STANCE was inadvertently omitted. We acknowledge this presentation error and have corrected it in the revised manuscript. Specifically, we have updated Table 3 and revised the dataset descriptions to ensure consistency with the task indexing used in the overlap figures and the continual learning task stream.

AuthorsOpenReview ↗

Response to Reviewer (2)

Q4: The comparisons do not isolate the benefit of the attribution method. Algorithm 1 adds independent model training and filtering of correct generations before importance estimation, but does not establish whether SNIP and Wanda receive the same settings. This leaves the source and computational cost of the gains in Tables 1 and 2 unresolved. Under identical task models, attribution samples, normalization, and update rules, how does AGFT compare with SNIP and Wanda?

A4: We thank the reviewer for raising this important concern. To address it, we conducted additional controlled experiments comparing AGFT with SNIP and Wanda under identical experimental settings.

Clarification of the Original Experimental Settings. In our original experiments, AGFT and the importance-based baselines (SNIP and Wanda) use the same number of correctly generated samples, the same importance score normalization and task-level aggregation procedures, and the same gradient update strategy. Specifically, for SNIP and Wanda, importance estimation is performed online during continual learning: after completing training on the current task, we evaluate the resulting continual-learning model on the corresponding task's training set, retain the correctly generated samples, compute the method-specific importance scores, and then normalize and aggregate these scores to guide subsequent task updates.

Additional Controlled Experiments. We conducted additional experiments in which all three methods use exactly the same independently fine-tuned task-specific models, the same set of correctly generated samples, and the same number of samples for importance estimation. We also ensure that all methods follow identical importance-score normalization, task-level aggregation, and gradient update procedures, including the same historical importance aggregation and gradient modulation rules described in our paper. The only difference lies in the importance estimation criterion: AGFT uses our LRP-based parameter attribution, whereas SNIP and Wanda use their respective importance estimation criteria.

Results. The results are presented in the two tables above. AGFT consistently outperforms both SNIP and Wanda across all four model configurations on both Concept-1K and TRACE.

Table 1. Comparison on Concept-1K.

MethodLlama3.2-3B (Full)Llama2-7B (LoRA)Qwen2.5-3B (Full)Qwen2.5-7B (LoRA)
AGFT26.33623.70728.29224.476
SNIP22.44119.31227.62422.802
Wanda22.60419.22927.46621.934

Table 2. Controlled comparison on TRACE.

MethodLlama3.2-3B (Full)Llama2-7B (LoRA)Qwen2.5-3B (Full)Qwen2.5-7B (LoRA)
AGFT54.88047.77554.92254.745
SNIP52.99746.73053.65253.897
Wanda50.33446.88754.02753.763

On Concept-1K datasts, AGFT outperforms SNIP and Wanda by 4.145 and 4.105 points on Llama models, respectively, and by 1.171 and 1.684 points on Qwen models. Averaged across all four model configurations, AGFT achieves improvements of 2.658 points over SNIP and 2.895 points over Wanda.

Similarly, on TRACE, AGFT outperforms SNIP and Wanda by 1.464 and 2.717 points on Llama models, respectively, and by 1.059 and 0.939 points on Qwen models. Averaged across all four model configurations, AGFT achieves improvements of 1.262 points over SNIP and 1.828 points over Wanda.

These experiments demonstrate that the performance gains arise from the effectiveness of our parameter-level attribution-based importance estimation.

Thanks for the author's detailed reply, in which most of my concerns have been addressed, but one question remains. The author states in their reply, "In Proposition 4.2, we adopt a simplified two-matrix FFN formulation," but "actual FFN architectures may differ across model families." Does this mean that the applicability of Proposition 4.2 is narrowed?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.