- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates how the functional organization of attention heads changes during post-training and uses these observations to propose Function-Targeted Adaptation (FTA).
First, the authors use probing and gradient-based attribution to identify attention heads associated with specific capabilities, including mathematical reasoning, harmlessness, and knowledge unlearning. By tracking these heads across checkpoints of one-epoch fine-tuning runs on Llama-3.1-8B, they find that many of the most important heads remain among the top-ranked heads, although their importance scores change during training. They also observe that heads associated with other capabilities remain relatively stable.
Based on these findings, the authors propose FTA, which identifies capability-relevant attention heads in the base model and restricts fine-tuning to their query, key, value, and output projections (~1% of the model parameters). The method is evaluated on four models (Llama-3.1-8B, Llama-3.2-3B, OLMo-2-7B, and OLMo-2-13B) for mathematical reasoning, harmlessness, and honesty under SFT and DPO, as well as knowledge unlearning on TOFU. It is compared against full fine-tuning, LoRA, P2D, and a random-head control. The results suggest that FTA can achieve comparable or better target-task performance than the baselines while better preserving performance on ten non-target benchmarks.
Critical strengths
-
The general idea of the paper is interesting, especially the connection between understanding how models change during post-training and leverating that information to guide adaptation.
-
The authors evaluate FTA across different model families, model sizes, capabilities, and training objectives. They consider not only improvements on the target task but also the preservation of non-target capabilities, evaluating performance across ten additional benchmarks.
-
I found the comparison with randomly selected attention heads insightful. The results suggest that the benefits of FTA come not only from restricting the number of updated parameters but also from selecting which heads to update. The comparison with parameter-matched LoRA is also useful in this regard. Although I have some concerns about the reliability of the experimental comparisons, I think these ablations provide some evidence supporting the main idea of the paper.
Critical weaknesses
-
The paper appears to report only one training run per configuration. While the authors analyze variability across sampled generations (Table 14), this does not capture variability across independent training runs. Reporting results across multiple seeds, at least for the main comparisons, would help establish how robust the improvements are.
-
Probing and gradient-based attribution can identify heads associated with a capability, but do not establish a causal role. Ablating the selected heads could help validate this interpretation. Additionally, it is unclear whether the observed stability of top-k heads reflects the strengthening of existing functional structures or simply the limited changes expected from one epoch of fine-tuning.
-
The paper would benefit from comparing the proposed probing-based selection with other ways of identifying parameters relevant to a task. For example, could a simpler gradient-based importance measure computed directly from the adaptation objective achieve similar results? The authors already use gradient-based attribution without a probe for unlearning (Appendix A.1), making this comparison particularly relevant.
-
I would have liked a clearer discussion of how the findings in Section 3 differ from previous observations, particularly given that the post-training plasticity analysis is presented as one of the main contributions. It would also be interesting to discuss how these findings relate to Park et al. (2025), which reports newly recruited attention heads during post-training.
-
The authors primarily measure forgetting using ROUGE-L on the forget set, but a low ROUGE-L score does not necessarily indicate that the underlying knowledge has been removed. Although the authors acknowledge this limitation, additional unlearning metrics would help support the conclusions about knowledge suppression.
Questions for authors
-
Could the authors clarify the discrepancies between the reported results? For example, the target average for LoRA (r=8) on Llama-3.1-8B under mathematical reasoning SFT is 78.4 in Table 1, but 44.2 in Tables 5 and 17. Similarly, the LoRA results for Llama-3.2-3B in Table 8 repeat the individual target benchmark scores reported for Llama-3.1-8B in Table 1.
-
Several baselines perform considerably worse than the original model after fine-tuning. For example, on Llama-3.1-8B under mathematical reasoning SFT, the reported results for full fine-tuning, LoRA, and P2D are all below the base model (80.4). There are also cases in the alignment experiments where full fine-tuning causes very large drops in non-target performance. Could the authors provide more details about the hyperparameter search spaces and selection criteria used for each baseline? This would help clarify whether the results reflect limitations of the adaptation methods or differences in optimization stability.
-
Could the authors clarify how the elbow criterion is used to determine the number of selected heads, k? Although Figure 4 provides the ranked importance curves, the procedure is not described in enough detail to reproduce the selection. It would also help to report the selected values for each model and capability in a table. Additionally, which value was used for harmlessness: k=116, as reported in Section 3.2 and Appendix A.3, or k=107, as reported in Tables 6 and 7?
-
Does the efficiency comparison in Table 5 include the full cost of head selection, including activation extraction, probe training, and attribution? If not, could the authors report these costs to provide a more complete picture of FTA’s efficiency?
6 comments
Next
Q. Causal validation of capability-associated heads and uncertainty about whether top-k head stability persists beyond one epoch of fine-tuning.
To assess the causal relevance of the identified heads, we conducted additional inference-time ablation experiments on Llama-3.1-8B for mathematical reasoning. Specifically, we perform head ablation by scaling each selected attention head's output by a small factor (), comparing the effects of ablating the top-k capability-associated heads against an equal number of randomly selected heads. As shown in the table below, ablating the identified heads causes substantial performance degradation across all four benchmarks, whereas ablating random heads results in considerably smaller declines. These results provide additional causal evidence for the functional involvement of the selected heads.
Table 3. Performance comparison between masking the top- heads and random heads on Llama-3.1-8B for math reasoning.
| Method | GSM8K | SVAMP | ASDiv | GSM+ | Math Avg. |
|---|---|---|---|---|---|
| Base | 81.90 | 88.20 | 84.00 | 67.50 | 80.40 |
| Random- | 74.98 | 80.80 | 80.36 | 57.72 | 73.46 |
| Top- | 55.65 | 59.20 | 74.08 | 41.85 | 57.69 |
To further investigate the stability of top-k heads, we extended our post-training plasticity analysis to three epochs (300% training progress) of full-parameter fine-tuning. As shown in Figure 4 of Rebuttal PDF, head stability persists beyond a single epoch, while changes in head importance remain concentrated among initially important heads (AUROC 0.73), indicating continued adaptation of existing functional structures. Furthermore, the relatively low cross-domain overlap suggests that these changes remain capability-specific. These observations are consistent with our original one-epoch findings, demonstrating that the identified patterns of functional plasticity persist over extended fine-tuning rather than being limited to a single epoch.
Q. Comparison of probing-based head selection with direct gradient-based importance measures.
Our current experiments already include P2D, a task-sensitive head selection method that identifies important attention heads by measuring weight-distribution shifts following a lightweight proxy fine-tuning process. To further address this concern, we conducted an additional comparison with direct gradient-based head selection.
The results show that FTA outperforms direct gradient-based selection by 13.2 points on target tasks (81.0% vs. 67.8%) and 5.0 points on non-target tasks (68.3% vs. 63.3%). These results further support the effectiveness of probing-based capability localization for selective adaptation.
Table 4. Comparison between probing-based head selection and direct gradient-based head selection on Llama-3.1-8B for math reasoning.
| Benchmark | Ours | Direct Gradient | Δ |
|---|---|---|---|
| Target | |||
| GSM8K | 82.8 | 71.7 | −11.1 |
| SVAMP | 89.4 | 72.6 | −16.8 |
| ASDiv | 83.4 | 71.4 | −12.0 |
| GSM+ | 68.4 | 55.5 | −12.9 |
| Avg | 81.0 | 67.8 | −13.2 |
| Non-Target | |||
| ARC-C | 81.9 | 79.4 | −2.5 |
| LogiQA2 | 56.1 | 47.7 | −8.4 |
| HumanEval+ | 63.4 | 64.6 | +1.2 |
| CRUX | 30.0 | 24.6 | −5.4 |
| MMLU | 63.2 | 59.4 | −3.8 |
| CSQA | 76.5 | 75.6 | −0.9 |
| OBQA | 83.2 | 82.4 | −0.8 |
| SIQA | 73.5 | 69.2 | −4.3 |
| SQuAD | 71.1 | 48.6 | −22.5 |
| BoolQ | 84.0 | 81.1 | −2.9 |
| Avg | 68.3 | 63.3 | −5.0 |
Next
Q. Distinction from prior studies on post-training plasticity, particularly Park et al. (2025).
Several prior studies have investigated how LLM components, particularly attention heads, evolve during fine-tuning.
Zhao et al. [1] employ gradient-based attribution to analyze attention-head activation patterns, showing that SFT selectively activates task-specific heads. Wang et al. [2] investigate fine-tuning through circuit analysis and find that functional circuits retain high node similarity despite substantial changes in their connections.
Park et al. [3] examine post-training dynamics through circuit analysis, focusing on emergent reasoning heads and their evolution under different training paradigms. A key methodological distinction lies in how head involvement is measured: Park et al. identify emergent heads based on discrete membership in sparse circuits, whereas we track continuous probe-based importance scores across training checkpoints. Under the former criterion, a head that already contributes to the baseline model but is not selected into its sparse circuit may be classified as emergent once it enters the post-training circuit.
Importantly, Park et al. also report that approximately two-thirds of baseline attention-head nodes remain active after distillation (Section 4.1), while GRPO primarily reuses existing computational pathways. These observations are consistent with our findings on the persistence of pre-existing functional heads. Our analysis further characterizes non-uniform changes in head importance and cross-capability stability, revealing how capability-associated functional structures evolve throughout post-training.
Overall, we systematically characterize head-level functional plasticity, and translate these insights into FTA for selective parameter updates. We will clarify these connections and distinctions in the revised manuscript.
[1] Zhao, Y., et al. Analyzing the rapid generalization of SFT via the perspective of attention head activation patterns.
[2] Wang, X., et al. Towards understanding fine-tuning mechanisms of LLMs via circuit analysis.
[3] Park, Y., et al. Thinking sparks!: Emergent attention heads in reasoning models during post training.
Q. More unlearning metrics beyond ROUGE-L.
To provide a more comprehensive evaluation, we follow the standard TOFU evaluation protocol in [1] and report additional metrics, including answer probability and Truth Ratio, across the Forget Set, Retain Set, Real Authors, and World Facts.
As shown in the following table, FTA achieves a near-zero forget-set answer probability and the highest forget-set Truth Ratio, indicating strong suppression of target knowledge. Meanwhile, FTA achieves the highest retain-set ROUGE-L score and answer probability among all methods, along with superior performance on Real Authors and World Facts, demonstrating better preservation of unrelated knowledge. These complementary metrics provide stronger evidence for FTA's effectiveness in balancing targeted knowledge suppression and knowledge retention.
Table 5. More results on TOFU.
| Method | Forget Prob.↓ | Forget Truth Ratio↑ | Retain Prob.↑ | Retain Truth Ratio↑ | Retain ROUGE-L↑ | Real Authors Prob.↑ | Real Authors Truth Ratio↑ | World Facts Prob.↑ | World Facts Truth Ratio↑ |
|---|---|---|---|---|---|---|---|---|---|
| Full | 2.5×10⁻⁶⁷±3.5×10⁻⁶⁷ | 34.7±10.6 | 21.5±30.4 | 32.0±9.2 | 23.4±32.7 | 37.7±19.0 | 39.9±23.5 | 40.5±20.5 | 42.5±24.3 |
| LoRA | 3.4±0.7 | 37.9±2.5 | 15.5±4.2 | 48.4±0.0 | 37.5±1.4 | 41.4±2.4 | 41.0±3.1 | 51.6±0.8 | 54.8±0.1 |
| P2D | 17.7±1.8 | 44.9±0.6 | 55.2±0.5 | 50.0±0.9 | 51.5±3.9 | 47.8±5.8 | 50.4±7.5 | 54.8±4.5 | 57.8±4.7 |
| Ours | 1.3×10⁻⁴±1.8×10⁻⁴ | 52.7±3.4 | 71.5±0.5 | 50.0±1.6 | 70.2±2.3 | 52.9±0.7 | 59.2±1.5 | 57.5±2.3 | 62.6±2.4 |
[1] Maini, P., et al. Tofu: A task of fictitious unlearning for llms.
Q. Could the authors clarify the discrepancies in the reported results, for LoRA () under Math SFT on Llama-3.1-8B and Llama-3.2-3B?
Thank you for identifying these inconsistencies. For Llama-3.1-8B under Math SFT, the LoRA (r=8) results are 81.3 (GSM8K), 84.9 (SVAMP), 81.5 (ASDiv), and 65.8 (GSM+), yielding an average of 78.4, as reported in Table 1.
The corrected results for Llama-3.2-3B LoRA (r=8) are presented in the table below. FTA achieves the highest performance across all four benchmarks, with an average accuracy of 72.2%, outperforming full fine-tuning and LoRA. We will correct these reporting errors in Appendix and ensure consistency across all tables in the revised manuscript.
Table 6. Corrected Math SFT results on Llama-3.2-3B, including the updated LoRA (r=8) results.
| Method | GSM8K | SVAMP | ASDiv | GSM+ | Avg. |
|---|---|---|---|---|---|
| Base | 53.9 | 64.4 | 65.0 | 51.0 | 58.6 |
| Full FT | 65.0 | 83.0 | 80.1 | 47.5 | 68.9 |
| LoRA (r=8) | 44.1 | 72.6 | 73.5 | 41.3 | 57.9 |
| FTA (Ours) | 69.1 | 84.2 | 80.3 | 55.3 | 72.2 |
Response to Reviewer
Dear Reviewer,
We deeply appreciate the time and effort you have dedicated to reviewing our work. Below, we address the questions you raised.
Q. Reporting results across multiple seeds, at least for the main comparisons, would help establish how robust the improvements are.
Following the reviewer's suggestion, we conducted three independent training runs with different random seeds for the main experimental comparisons, reporting the mean and standard deviation of target and non-target performance.
The results in Table 3 show that FTA achieves the best or tied-best mean target performance in 9 out of 13 settings with significant (p < 0.05) or marginally significant (p < 0.1) improvements in several comparisons. In particular, FTA consistently outperforms the baselines on mathematical reasoning and achieves strong results on honesty-related tasks, although some settings exhibit greater variability.
For non-target capabilities, FTA demonstrates significantly better preservation on Llama-3.1-8B Math SFT than all baselines (p < 0.01), while maintaining broadly comparable performance to P2D in several other settings, with some trade-offs.
Overall, these additional analyses strengthen the empirical evidence for FTA's robustness across training runs. We will incorporate these results and clarifications into the revised manuscript.
Table 1. Target and non-target performance (mean±std over three seeds).
Table 2. Statistical analysis of Ours vs. others. bold: p < 0.1