acceptodds
Under review as a conference paper at ICLR 2027

DHAFT: Dynamic Hierarchy-Aware Fine-Tuning for Efficient LLM Reasoning

Abstract

Large language models (LLMs) have achieved remarkable performance on complex reasoning tasks, but efficiently adapting them to diverse reasoning scenarios remains challenging. Soft prompt tuning provides a parameter‑efficient adaptation strategy with a small number of trainable prompt vectors. However, existing approaches overlook the evolving nature of internal representations across model depth, making prompt effectiveness sensitive to different representation stages. To address this issue, we propose DHAFT, a hierarchy‑aware soft prompt tuning framework that dynamically identifies hierarchical representation stages from internal representations and performs stage‑specific prompt injection at representative layers. By aligning prompt placement with representation stages, DHAFT enables more effective prompt utilization and reduces potential cross‑stage interference. Furthermore, we introduce PSMI, a prompt‑stage misalignment index that quantifies the mismatch between prompt influence and representation stages. Extensive experiments across diverse LLM backbones and reasoning benchmarks demonstrate that DHAFT achieves competitive or superior performance compared with representative PEFT methods while maintaining minimal parameter overhead.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Parameter-Efficient Fine-Tuning, Soft Prompt Tuning, Large Language Models, Complex Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1BYmOpenReview ↗
Rating
2
Confidence
2
Informativeness
3(+1)
Presentation
2
Soundness
2
Read review
Paper summary

The authors present DHAFT, a soft prompt tuning method for frozen LLMs. DHAFT uses linear CKA to compare layers based on 256 unlabeled training inputs. Then, agglomerative clustering is used to group the layers into G "stages". G is determined by silhouette score with bootstrap. At the beginning of each stage, a trainable soft prompt is inserted; at the end of each stage, the prompt is removed. This ensures that the sequence length remains constant at n+m.

The paper also introduces PSMI, an index derived from layer-wise gradient-times-attention saliency. PSMI quantifies both the amount of saliency mass that extends beyond the stage where a prompt is inserted and the variability of saliency within the stage.

The authors evaluate DHAFT on eight commonsense reasoning tasks using four different LLaMA-family models and five math-related tasks on Llama-2-7B, Llama-3-8B and Qwen3-8B. The gains on commonsense reasoning are minimal, the gains on math-related tasks are more substantial, primarily due to improvements on GSM8K.

Critical strengths
  • The proposed approach is relatively straightforward and computationally efficient. Only a small number of soft prompts are trained, and there is no additional per-sample inference cost other than the extra prompt tokens.

  • Using CKA rather than cosine distance to compare layers is logical, and the authors provide a clear explanation of the invariance property in Appendix A.5.

  • Removing the prompt at the end of each stage is a clean way to limit the sequence length. Algorithm 2 make the forward pass easy to understand.

  • The experiments involve multiple backbones and two task families, and seed variance is reported for the math results.

Critical weaknesses
  • The saliency score is not specific to the prompt. Equation (1) sums the gradient-times-attention term over the entire attention matrix and then averages over all token positions. It does not restrict the calculation to attention involving prompt tokens. The method calculates flows between specific token groups. Therefore, I^l represents overall attention saliency at layer l, not "prompt influence". PSMI mainly reflects stage size. The histogram in Figure 7 indicates that most saliency values fall between 0.70 and 0.82, so after normalization, q is nearly uniform. For nearly-uniform q, the leakage term is approximately 1 - |G_i|/L, as stated by the authors in property (v). Therefore, the comparison of shallow/middle/deep in Figure 2(f) is primarily a comparison of how large the stage containing each insertion layer is.
  • The relationship between PSMI and accuracy shown in Figure 2(g) is based on four points. These four configurations also vary in aspects other than PSMI. Therefore, there is insufficient evidence to conclude that higher PSMI corresponds to lower accuracy. Furthermore, PSMI is never reported for DHAFT, so the paper does not demonstrate that DHAFT actually reduces misalignment. As presented, the PSMI contribution is unrelated to the method.
  • The math results appear excessively high, and the training data is not specified. Llama-2-7B at 66.1% on GSM8K is comparable to what MetaMath-7B reports with full fine-tuning on approximately 395K augmented examples.
  • All baselines listed in Table 1 are taken from previous publications, whereas DHAFT is evaluated within the authors' own pipeline.
Questions for authors
  • Figure 7 uses 100 of the 500 probing examples. Why?
  • The seeds also affect the calibration sample. Did the partition or G vary across seeds?
  • Typos: "Propogation" in Figure 1(c), and "Eq. equation 1" in several locations.
6 comments
AuthorsOpenReview ↗

Response to Reviewer 1BYm [1/4]

Dear Reviewer 1BYm,

We sincerely appreciate your insightful and constructive feedback, which has helped us refine the PSMI formulation, strengthen its empirical validation, and clarify the fairness and reproducibility of our evaluation. Below, we provide detailed point-by-point responses together with the corresponding revisions, with new experimental results highlighted in red and methodological clarifications highlighted in blue in the revised manuscript.


W1.1:

The saliency score is not specific to the prompt. Equation (1) sums the gradient-times-attention term over the entire attention matrix and then averages over all token positions. It does not restrict the calculation to attention involving prompt tokens. The method calculates flows between specific token groups. Therefore, I^l represents overall attention saliency at layer l, not "prompt influence". PSMI mainly reflects stage size. The histogram in Figure 7 indicates that most saliency values fall between 0.70 and 0.82, so after normalization, q is nearly uniform. For nearly-uniform q, the leakage term is approximately 1 - |G_i|/L, as stated by the authors in property (v). Therefore, the comparison of shallow/middle/deep in Figure 2(f) is primarily a comparison of how large the stage containing each insertion layer is.

Response: We thank the reviewer for this important observation. We agree that the original saliency formulation includes non-prompt attention interactions, while stage size and injection depth may introduce structural bias into PSMI. Following your suggestion, we introduce two refinements to address these concerns.

(1) Prompt-Specific Saliency. We restrict gradient-weighted attention to task-token queries and the corresponding prompt-token keys, rather than aggregating over the entire attention matrix:

Iiℓ=1H∣T∣∣Pi∣∑h=1H∑t∈T∑p∈Pi∣Atph,ℓ∂L∂Atph,ℓ∣.I_i^\ell = \frac{1}{H|\mathcal{T}||\mathcal{P}_i|} \sum_{h=1}^{H} \sum_{t\in\mathcal{T}} \sum_{p\in\mathcal{P}_i} \left| A_{tp}^{h,\ell} \frac{\partial\mathcal{L}}{\partial A_{tp}^{h,\ell}} \right|.

Here, T\mathcal{T} and Pi\mathcal{P}_i denote the task-token and prompt-token positions, respectively. This formulation excludes task-to-task attention and explicitly measures the direct attention saliency associated with the target prompt.

(2) Stage-Size and Injection-Depth Correction. To account for structural bias in cross-stage leakage, we introduce a uniform-saliency reference over the layers accessible to each prompt under the unmasked Retain policy:

CSLobs(i)=∑ℓ∈Ri∖GiIiℓ∑ℓ∈RiIiℓ\mathrm{CSL}_{\mathrm{obs}}(i) = \frac{\sum_{\ell\in R_i\setminus G_i} I_i^\ell}{\sum_{\ell\in R_i} I_i^\ell}

CSLnull(i)=1−∣Gi∩Ri∣∣Ri∣\mathrm{CSL}_{\mathrm{null}}(i) = 1-\frac{|G_i\cap R_i|}{|R_i|}

CSLcorrected(i)=CSLobs(i)−CSLnull(i)\mathrm{CSL}_{\mathrm{corrected}}(i) = \mathrm{CSL}_{\mathrm{obs}}(i) - \mathrm{CSL}_{\mathrm{null}}(i)

Here, GiG_i denotes the assigned stage and RiR_i denotes the layers accessible to the target prompt under the unmasked Retain policy. The corrected metric is undefined when the total prompt-specific saliency is zero.

Injection LayerAssigned StageAccessible LayersUniform LeakageCorrected Leakage
Shallow (1)1–9320.7190
Middle (10)10–22230.4350
Deep (23)23–321000

As shown above, the corrected leakage is zero across all three injection depths under uniform saliency. This confirms that the expected contribution of stage size and injection depth is removed by construction under the uniform reference, rather than being interpreted as prompt-specific misalignment. We therefore treat the PSMI–accuracy relationship as descriptive evidence and further examine direct cross-stage prompt access through the fixed-checkpoint intervention in W1.2.

The revised prompt-specific saliency formulation in Section 3.2 and the stage-size correction in Appendix A.4 (Table 8) are highlighted in blue and red, respectively, with the corresponding explanations highlighted in blue.

AuthorsOpenReview ↗

Response to Reviewer 1BYm [2/4]

W1.2: Relationship Between PSMI and Reasoning Accuracy

The relationship between PSMI and accuracy shown in Figure 2(g) is based on four points. These four configurations also vary in aspects other than PSMI. Therefore, there is insufficient evidence to conclude that higher PSMI corresponds to lower accuracy. Furthermore, PSMI is never reported for DHAFT, so the paper does not demonstrate that DHAFT actually reduces misalignment. As presented, the PSMI contribution is unrelated to the method.

Response: We thank the reviewer for highlighting this important limitation. We agree that the original four-point PSMI–accuracy analysis does not isolate the effect of cross-stage prompt influence. We also recognize the need to directly evaluate PSMI for DHAFT.

Following your suggestion, we conduct a controlled attention intervention on the same Llama-3-8B Retain checkpoint, fixing model parameters, stage boundaries, prompt insertion locations, and evaluation settings while progressively restricting direct attention access to earlier prompts. We evaluate prompt-specific PSMI on 500 GSM8K probing examples and separately report independently trained DHAFT.

Prompt ConfigurationPSMI ↓GSM8K ↑
Retain-All0.47572.8
Retain + Mask 25%0.44273.0
Retain + Mask 50%0.36772.9
Retain + Mask 75%0.25573.2
Retain + Mask All0.00973.5
DHAFT (Remove)0.01174.0

As shown above, prompt-specific PSMI consistently decreases as direct cross-stage access is restricted, while GSM8K accuracy generally improves. Within the same Retain checkpoint, fully masking earlier prompts improves accuracy from 72.8% to 73.5% without retraining. Independently trained DHAFT achieves 74.0% accuracy with a substantially lower PSMI of 0.011, compared with 0.475 for Retain-All.

These results provide complementary evidence that restricting direct access to earlier prompts can improve reasoning performance. We accordingly interpret PSMI as a diagnostic measure rather than treating its correlation with accuracy as causal evidence.

The additional PSMI analysis in Section 4.3 (Table 7) is highlighted in red, with the corresponding discussion highlighted in blue. The revised PSMI formulation in Section 3.2 is highlighted in blue.


W1.3:

The math results appear excessively high, and the training data is not specified. Llama-2-7B at 66.1% on GSM8K is comparable to what MetaMath-7B reports with full fine-tuning on approximately 395K augmented examples.

Response: We appreciate the reviewer's concern regarding training data transparency and evaluation reproducibility. Following your suggestion, we clarify the training data, decoding settings, and answer extraction protocol underlying our mathematical reasoning results.

For GSM8K [1], DHAFT uses the official 7,473 training examples and 1,319 test examples, optimizing only stage-specific soft prompts while keeping the pretrained Llama-2-7B backbone frozen. The 256 calibration examples for CKA-based stage discovery are sampled exclusively from the training split.

Evaluation uses greedy decoding with a maximum of 512 new tokens. We extract the final numerical answer after removing numerical separators and trailing punctuation, counting invalid or unparseable outputs as incorrect. DHAFT is evaluated across three independent training seeds.

MethodParams (%)GSM8K ↑
LoRA0.8354.1
DoRA0.8455.4
LoReFT0.0358.9
DHAFT (Ours)0.02566.1 ± 0.28

DHAFT achieves 66.1 ± 0.28% GSM8K accuracy, exceeding the published LoReFT result by 7.2 percentage points with approximately 0.025% trainable parameters. The LoRA, DoRA, and LoReFT results are taken from their respective published evaluations.

For context, MetaMath-7B [2] reports 66.5% using approximately 395K augmented mathematical examples with full fine-tuning, whereas DHAFT uses only the original 7,473 GSM8K training examples for supervised adaptation with a frozen backbone. The training data, decoding settings, and answer extraction protocol in Sections 4.1–4.2 and Appendix A.6.2 are clarified in blue.


References

[1] Cobbe et al. Training Verifiers to Solve Math Word Problems. 2021.

[2] MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models. ICLR 2024.

AuthorsOpenReview ↗

Response to Reviewer 1BYm [3/4]

W1.4:

All baselines listed in Table 1 are taken from previous publications, whereas DHAFT is evaluated within the authors' own pipeline.

Response: We appreciate the reviewer's concern regarding the fairness of baseline comparisons. We agree that results obtained under different implementation and evaluation settings should be distinguished from strictly controlled comparisons.

For the original LLaMA-backbone results in Table 1, we retain the published baseline scores and clarify their provenance in Appendix A.6.1. DHAFT follows the established commonsense reasoning protocol of LLM-Adapters [1] and DoRA [2], using the combined training data of eight benchmarks and their corresponding task-specific test splits. We summarize the Llama-3-8B comparison below.

MethodParams (%)Avg. Acc. ↑
LoRA0.7080.8
DoRA0.7185.2
LoReFT0.0386.6
HiRA (r=32)0.7086.7
DHAFT (Ours)0.02186.9

DHAFT achieves 86.9% average accuracy with only 0.021% trainable parameters, exceeding the published LoReFT and HiRA (r=32) results by 0.3 and 0.2 percentage points, respectively. Given the differences in evaluation pipelines, we interpret these comparisons as evidence of parameter efficiency and competitiveness rather than attributing the small accuracy differences exclusively to DHAFT.

Additional Qwen3-8B Evaluation. To provide further empirical evidence beyond the previously reported LLaMA baselines, we extend our evaluation to eight commonsense reasoning benchmarks on Qwen3-8B, including the unadapted backbone and multiple adaptation baselines. We adopt the official Qwen3 chat template, Non-Thinking Mode, and candidate answer log-likelihood scoring, with baseline provenance explicitly distinguished in the revised manuscript. The results are summarized below.

MethodParams (%) ↓BoolQPIQASIQAHellaS.WinoG.ARC-eARC-cOBQAAvg. ↑
Pretrained—86.676.783.357.268.083.355.531.467.75
Prefix-Tuning0.021680.684.580.090.379.589.675.975.682.00
LoRA0.023485.187.882.493.484.592.079.681.885.83
LoReFT0.021686.489.383.794.785.992.881.483.187.16
HiRA (r=16)0.358886.389.183.794.885.892.981.883.587.24
HiRA (r=32)0.715087.189.283.995.086.092.982.683.387.50
DHAFT (Ours)0.020486.789.784.195.186.293.382.183.987.64

As shown above, DHAFT achieves 87.64% average accuracy with only 0.0204% trainable parameters, outperforming the unadapted backbone, LoRA, and LoReFT by 19.89, 1.81, and 0.48 percentage points, respectively. It also achieves a slightly higher average accuracy than HiRA (r=32), while using substantially fewer trainable parameters. These results provide additional evidence of DHAFT's competitive performance and parameter efficiency on a modern backbone.

The baseline provenance and evaluation protocol in Appendix A.6.1 are highlighted in blue. The Llama-3-8B comparison in Appendix A.6.2 (Table 9) and the additional Qwen3-8B results in Section 4.2 (Table 1) are highlighted in red, with evaluation configurations in Appendix A.7 (Table 13) clarified in blue.


References

[1] LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models. EMNLP 2023.

[2] DoRA: Weight-Decomposed Low-Rank Adaptation. ICML 2024.

Reviewer_g9BbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes DHAFT, a parameter-efficient fine-tuning method that uses cross-layer representation similarity to determine where soft prompts should be inserted. The authors use Centered Kernel Alignment (CKA) and hierarchical clustering to divide Transformer layers into representation stages. Each stage receives a separate trainable prompt, which is removed before entering the next stage, while the pretrained backbone remains frozen.

The paper also introduces the Prompt-Stage Misalignment Index (PSMI) to measure how prompt influence aligns with representation stages. Experiments on commonsense and mathematical reasoning benchmarks show improvements over several PEFT baselines with relatively few trainable parameters.

Critical strengths
  1. The proposed method is relatively simple and provides an interesting approach to selecting prompt injection positions based on internal model representations.
  2. The reported performance improvements are promising, especially on mathematical reasoning tasks. The gains over LoReFT on GSM8K are substantial, and the comparison with uniform partitioning provides some evidence that CKA-based layer selection is useful.
  3. The method requires relatively few trainable parameters, and the stage partition is computed offline without introducing additional per-instance clustering at inference time.
Critical weaknesses
  1. The motivation for stage-specific prompt removal is not sufficiently justified.

As I understand it, the paper's main argument is: Different Transformer layers have different representations. Therefore, prompts propagated across representation stages may become misaligned and cause interference. To avoid this interference, prompts should be removed at stage boundaries.

I think the key issue is the connection between these steps. Different representations do not necessarily mean that earlier prompts are no longer useful. The PSMI analysis shows a correlation between misalignment and accuracy across four injection strategies, but does not establish that cross-stage prompt influence actually harms reasoning. Without stronger evidence for this connection, I find the main intuition behind the proposed method unclear.

  1. Insufficient ablations.

The experiments show that CKA-based partitioning performs better than several alternative partitioning strategies. However, they do not directly compare retaining and removing prompts at the same injection positions. Such a comparison seems necessary to establish the benefit of prompt removal.

Questions for authors

Can the authors provide a controlled comparison of prompt retention and removal at identical injection positions and parameter budgets, examining both downstream prompt influence and reasoning accuracy, to test whether cross-stage prompt influence is harmful and whether removal mitigates it?

6 comments
AuthorsOpenReview ↗

Response to Reviewer g9Bb [1/2]

Dear Reviewer g9Bb,

We sincerely thank you for your thoughtful review and constructive suggestions. Your feedback has helped us strengthen the motivation for stage-boundary prompt removal and provide more rigorous experimental evidence. Below, we provide point-by-point responses and describe the corresponding manuscript revisions, with new experimental results highlighted in red and methodological clarifications highlighted in blue.


W1:

The motivation for stage-specific prompt removal is not sufficiently justified.

As I understand it, the paper's main argument is: Different Transformer layers have different representations. Therefore, prompts propagated across representation stages may become misaligned and cause interference. To avoid this interference, prompts should be removed at stage boundaries.

I think the key issue is the connection between these steps. Different representations do not necessarily mean that earlier prompts are no longer useful. The PSMI analysis shows a correlation between misalignment and accuracy across four injection strategies, but does not establish that cross-stage prompt influence actually harms reasoning. Without stronger evidence for this connection, I find the main intuition behind the proposed method unclear.

Response: We appreciate the reviewer's thoughtful observation regarding the motivation for stage-boundary prompt removal. We agree that representational dissimilarity across stages does not necessarily imply that earlier prompts become harmful. To directly examine this relationship, we conduct a controlled attention intervention using the same Llama-3-8B CKA + Retain checkpoint, fixing model parameters, stage boundaries, prompt insertion locations, sequence length, and task-token position IDs while varying only direct attention access to earlier prompt keys and values.

Prompt InterventionGSM8K ↑Task NLL ↓
Retain-All72.81.13
Retain + Mask Old Prompts73.51.11
DHAFT (Remove)74.01.09

As shown above, restricting direct cross-stage prompt access within the same checkpoint improves GSM8K accuracy by 0.7 points and reduces task NLL from 1.13 to 1.11. Independently trained DHAFT further achieves 74.0% accuracy and 1.09 NLL. These results provide direct experimental evidence that continued access to earlier prompts can impair reasoning performance, supporting the motivation for stage-boundary prompt removal.

The intervention results in Section 4.3 (Table 6) are highlighted in red, with the corresponding analysis highlighted in blue. The prompt-specific saliency formulation in Section 3.2 is also clarified in blue.


W2:

Insufficient ablations.

The experiments show that CKA-based partitioning performs better than several alternative partitioning strategies. However, they do not directly compare retaining and removing prompts at the same injection positions. Such a comparison seems necessary to establish the benefit of prompt removal.

Response: We thank the reviewer for highlighting the importance of isolating the contribution of boundary removal. We agree that this requires controlling the stage partition, prompt insertion locations, and trainable parameters.

Following your suggestion, we conduct a 2 × 2 controlled ablation on Llama-3-8B, comparing Retain and Remove policies under Uniform and CKA-based stage partitions. Within each partition, we fix stage boundaries, prompt insertion locations, trainable parameters, training settings, and task-token position IDs, varying only the retention policy. Each configuration is trained independently under the same protocol.

Stage PartitionPrompt PolicyGSM8K ↑MATH ↑AQuA ↑Attention FLOPs (G) ↓
UniformRetain64.733.138.4325.13
UniformRemove65.833.939.0220.15
DHAFT PartitionRetain72.840.643.8331.51
DHAFT PartitionRemove74.041.944.7220.15

As shown above, boundary removal improves GSM8K/MATH/AQuA by 1.2/1.3/0.9 points under CKA-based partitioning, while reducing prefill attention FLOPs by 33.59%. Uniform partitioning yields corresponding gains of 1.1/0.8/0.6 points, with a 32.29% FLOPs reduction. Under the same Remove policy, CKA-based partitioning further outperforms Uniform partitioning by 8.2/8.0/5.7 points. These controlled comparisons demonstrate the complementary benefits of stage-boundary prompt removal and representation-guided stage discovery, with consistent accuracy gains and reduced prefill attention computation.

The controlled ablation results in Section 4.3 (Table 5) are highlighted in red, with the corresponding analysis highlighted in blue.

AuthorsOpenReview ↗

Response to Reviewer g9Bb [2/2]

Q1:

Can the authors provide a controlled comparison of prompt retention and removal at identical injection positions and parameter budgets, examining both downstream prompt influence and reasoning accuracy, to test whether cross-stage prompt influence is harmful and whether removal mitigates it?

Response: We thank the reviewer for this important suggestion. Following your suggestion, we conduct two complementary controlled experiments on Llama-3-8B to isolate the effects of stage-boundary prompt removal on reasoning performance and direct cross-stage prompt access.

(1) Retain-versus-Remove Ablation. We compare both policies under Uniform and CKA-based stage partitions, fixing stage boundaries, prompt insertion locations, trainable parameters, training settings, and task-token position IDs within each partition. Each configuration is trained independently under the same protocol.

Stage PartitionPrompt PolicyGSM8K ↑MATH ↑AQuA ↑Attention FLOPs (G) ↓
UniformRetain64.733.138.4325.13
UniformRemove65.833.939.0220.15
DHAFT PartitionRetain72.840.643.8331.51
DHAFT PartitionRemove74.041.944.7220.15

As shown above, boundary removal improves GSM8K/MATH/AQuA by 1.2/1.3/0.9 points under CKA-based partitioning, while reducing prefill attention FLOPs by 33.59%. Consistent improvements under Uniform partitioning further indicate that these benefits are not limited to CKA-based stage discovery.

(2) Same-Checkpoint Attention Intervention. To isolate direct cross-stage prompt access without retraining, we fix model weights, stage boundaries, prompt insertion locations, sequence length, and task-token position IDs, modifying only attention access to earlier prompt keys and values.

Prompt InterventionGSM8K ↑Task NLL ↓
Retain-All72.81.13
Retain + Mask Old Prompts73.51.11
DHAFT (Remove)74.01.09

As shown above, masking direct cross-stage prompt access within the same Retain checkpoint improves GSM8K accuracy by 0.7 points and reduces task NLL from 1.13 to 1.11, without changing model parameters. Independently trained DHAFT further achieves 74.0% accuracy and 1.09 NLL.

Together, these results demonstrate that boundary removal improves performance under controlled training conditions, while the same-checkpoint intervention provides additional evidence that restricting direct access to earlier prompts can mitigate interference without retraining.

The controlled ablations in Section 4.3 (Tables 5 and 6) are highlighted in red, with the corresponding analyses highlighted in blue. The mechanism clarification in Section 3.2 is also highlighted in blue.


We thank you again for reviewing our work. Please let us know if we have misunderstood any of your questions or if further clarification would be helpful.

Thank you for the detailed response and the added Retain/Remove ablation (Table 5). I have three remaining questions.

  1. Shallow injection vs. DHAFT. In Figure 2(g), the shallow setting reaches 74.0% on GSM8K, the same as DHAFT, whereas input-level Prompt Tuning gets only 65.5% in Table 3. What exactly is the shallow setting, and how does it differ from Prompt Tuning? If a single shallow prompt already reaches 74.0%, what does DHAFT’s multi-stage design add?

  2. Uniform vs. CKA partition. What are the exact stage boundaries of the Uniform partition in Table 5? With 32 layers and three stages, an equal split should put the boundaries near layers 11 and 22, close to the CKA boundaries (9, 22) in Table 14. Why does such a small difference cause an 8 point gap on GSM8K, when the one-layer boundary shifts across calibration seeds (Table 14) give a standard deviation of only 0.24 (Table 12)?

  3. Evaluation settings. The paper mixes results copied from prior work with results from the authors’ own pipeline, and the protocols are often unspecified. For example, every-layer prompts on Llama-3-8B GSM8K score 63.0 (Figure 2(g)), 45.7 (Table 2), and 69.2 (Table 16). How do these settings differ? Please also mark the source and evaluation setting of each baseline row.

Reviewer_oAYFOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

DHAFT adapts frozen language models by grouping layers into representation stages using CKA-based similarity and hierarchical clustering. It learns stage-specific soft prompts, inserts them at the first layer of each stage, and removes their token positions at stage boundaries. The partition is derived offline and reused during training and inference. The paper introduces a Prompt-Stage Misalignment Index (PSMI) and evaluates the method on commonsense and mathematical reasoning benchmarks.

Critical strengths

A potentially useful adaptation method. Organizing prompts around representation structure is reasonable. DHAFT improves GSM8K accuracy over LoReFT and average accuracy across the evaluated backbone configurations (Tables 1–2). Placement ablations and seed replications support its potential as a lightweight adaptation strategy, subject to clarification of parameter accounting.

Critical weaknesses
  1. The specific contribution is insufficiently established. Previous literature has explored related ideas (SPT). DHAFT's distinctive contribution lies primarily in representation-derived stage boundaries and constraints on direct prompt propagation. Under standard causal attention with fixed prompt positions, the prepended stage prompts induce input-independent keys and values within a stage, suggesting a structured parameterization of layer-wise prefixes. Its added benefit remains unclear. The placement and stage-count ablations do not isolate boundary removal with insertion locations and trainable parameters fixed (Tables 3–4). The PSMI–accuracy association does not establish that reduced cross-stage influence causes the gains (§3.2), and indirect prompt effects persist through task-token states. The evidence supports a promising heuristic more strongly than its proposed explanation.

  2. The configuration is unclear for assessing efficiency. The per-stage prompt definition (§3.3.2), stated total budget and listed lengths do not clearly reconcile with Llama-3-8B's reported trainable-parameter percentage under the described prompt-only implementation (Tables 1–2). Exact checkpoint and generation settings are also missing. Qwen3-8B appears in Table 2 but not Table 1, limiting cross-domain coverage without invalidating comparisons within either table.

Questions for authors
  1. Prompt budget and implementation. What stage count, per-stage prompt lengths, trainable components and complete parameter count were used for Llama-3-8B? How do they reconcile with the stated total budget, listed lengths, and the percentages in Tables 1–2? What denominator was used?

  2. Contribution of boundary removal. With stage partition, insertion locations, trainable parameters and task-token positional conventions fixed, how does boundary removal compare with retaining prompts? What are the respective accuracy effects and additional attention costs?

  3. Evaluation configuration and coverage. Which checkpoints, prompt/chat templates, decoding settings, generation budgets and Qwen3 thinking mode were used? How do unadapted checkpoints perform under the same settings, and why does backbone coverage differ between Tables 1 and 2?

7 comments
AuthorsOpenReview ↗

Response to Reviewer oAYF [1/6]

Dear Reviewer oAYF,

We sincerely thank you for your insightful and constructive feedback, which has helped us strengthen the experimental evidence and clarify the contributions of our work. Below, we provide point-by-point responses to your concerns, with new experimental results highlighted in red and methodological clarifications highlighted in blue in the revised manuscript.


W1:

The specific contribution is insufficiently established. Previous literature has explored related ideas (SPT). DHAFT's distinctive contribution lies primarily in representation-derived stage boundaries and constraints on direct prompt propagation. Under standard causal attention with fixed prompt positions, the prepended stage prompts induce input-independent keys and values within a stage, suggesting a structured parameterization of layer-wise prefixes. Its added benefit remains unclear. The placement and stage-count ablations do not isolate boundary removal with insertion locations and trainable parameters fixed (Tables 3–4). The PSMI–accuracy association does not establish that reduced cross-stage influence causes the gains (Section 3.2), and indirect prompt effects persist through task-token states. The evidence supports a promising heuristic more strongly than its proposed explanation.


W1.1: Comparison with Existing Prompt Adaptation Methods

Response: We appreciate the reviewer's thoughtful observation regarding the relationship between DHAFT and existing prompt adaptation methods. We acknowledge that, under causal attention with fixed prompt positions, stage prompts induce input-independent keys and values, and DHAFT can therefore be viewed as a structured parameterization of layer-wise prefixes. The key distinction lies in how prompts are organized and propagated. Unlike Layer-wise Prefix-Tuning [2], which learns independent per-layer prefixes, DHAFT propagates stage-specific prompts within each stage and removes them at stage boundaries. Unlike SPT [1], which learns injection locations through task supervision, DHAFT discovers stage boundaries using cross-layer similarity on 256 unlabeled examples.

Following your suggestion, we conduct a parameter-matched comparison on Llama-3-8B under identical training and evaluation protocols, with approximately 0.02% trainable parameters. Efficiency is measured on an RTX 4090 using BF16, batch size 1, 512 input tokens, and 256 generated tokens.

MethodParamsGSM8K ↑MATH ↑AQuA ↑Throughput ↑Memory (GB) ↓
Layer-wise Prefix0.02%69.2 ± 0.3538.6 ± 0.2842.1 ± 0.3943.216.64
SPT0.02%72.3 ± 0.3040.6 ± 0.2544.0 ± 0.2341.016.75
DHAFT (Ours)0.02%74.0 ± 0.2441.9 ± 0.1844.7 ± 0.2744.816.61

As shown above, DHAFT improves over SPT by 1.7/1.3/0.7 points on GSM8K/MATH/AQuA, with 9.27% higher throughput and 0.14 GB lower peak memory. It also outperforms Layer-wise Prefix by 4.8/3.3/2.6 points. These results support the combined effectiveness of DHAFT's structured stage-wise prompt adaptation under matched parameter budgets.

The comparison results in Appendix A.7.1 (Table 16) are highlighted in red, with the corresponding analysis highlighted in blue. The baseline provenance in Appendix A.6.1 and evaluation configurations in Appendix A.7 (Table 13) are further clarified in the revised manuscript.


References

[1] SPT: Learning to Selectively Insert Prompts for Better Prompt Tuning, EMNLP 2023.

[2] Prefix-Tuning: Optimizing Continuous Prompts for Generation, ACL 2021.

AuthorsOpenReview ↗

Response to Reviewer oAYF [2/6]

W1.2: Contribution of Stage-Boundary Prompt Removal

Response: We thank the reviewer for highlighting the importance of isolating the contribution of stage-boundary prompt removal. We agree that this requires controlling prompt insertion locations and trainable parameters. Following your suggestion, we conduct a 2 × 2 controlled ablation on Llama-3-8B, comparing Retain and Remove policies under Uniform and CKA-based stage partitions. Within each partition, we fix the backbone, stage boundaries, prompt insertion locations, trainable parameters, training settings, and task-token position IDs, varying only the retention policy. Each configuration is trained independently under the same protocol.

Stage PartitionPrompt PolicyGSM8K ↑MATH ↑AQuA ↑Attention FLOPs (G) ↓
UniformRetain64.733.138.4325.13
UniformRemove65.833.939.0220.15
DHAFT PartitionRetain72.840.643.8331.51
DHAFT PartitionRemove74.041.944.7220.15

As shown above, boundary removal improves GSM8K/MATH/AQuA by 1.2/1.3/0.9 points under CKA-based partitioning, while reducing prefill attention FLOPs by 33.59%. Uniform partitioning yields corresponding gains of 1.1/0.8/0.6 points, with a 32.29% FLOPs reduction. Under the same Remove policy, CKA-based partitioning further outperforms Uniform partitioning by 8.2/8.0/5.7 points. These controlled comparisons demonstrate the independent benefit of boundary removal at fixed insertion locations, while showing that representation-guided stage discovery provides additional accuracy gains.

The controlled ablation results in Section 4.3 (Table 5) are highlighted in red, with the corresponding analysis highlighted in blue.


W1.3: Analysis of PSMI and Cross-Stage Prompt Influence

Response: We appreciate the reviewer's concern regarding the causal interpretation of PSMI. We agree that PSMI–accuracy correlation alone does not establish causality. Following your suggestion, we conduct a controlled attention intervention using the Llama-3-8B CKA + Retain checkpoint from W1.2. We fix model parameters, stage partition, prompt insertion locations, sequence length, and task-token position IDs, modifying only direct attention access to earlier prompt keys and values in subsequent stages. We evaluate GSM8K accuracy and reference-sequence task NLL under the same protocol, with independently trained DHAFT included for comparison.

Prompt InterventionGSM8K ↑Task NLL ↓
Retain-All72.81.13
Retain + Mask Old Prompts73.51.11
DHAFT (Remove)74.01.09

As shown above, restricting direct cross-stage prompt access within the same checkpoint improves GSM8K accuracy by 0.7 points and reduces task NLL from 1.13 to 1.11. Independently trained DHAFT achieves 74.0% accuracy and 1.09 NLL. These results provide direct experimental evidence that continued access to earlier prompts can impair reasoning performance, supporting the motivation for stage-boundary prompt removal.

The intervention results in Section 4.3 (Table 6) are highlighted in red, with the corresponding analysis highlighted in blue. The prompt-specific saliency formulation in Section 3.2 is also clarified in blue.

AuthorsOpenReview ↗

Response to Reviewer oAYF [3/6]

W2:

The configuration is unclear for assessing efficiency. The per-stage prompt definition (Section 3.3.2), stated total budget and listed lengths do not clearly reconcile with Llama-3-8B's reported trainable-parameter percentage under the described prompt-only implementation (Tables 1–2). Exact checkpoint and generation settings are also missing. Qwen3-8B appears in Table 2 but not Table 1, limiting cross-domain coverage without invalidating comparisons within either table.


W2.1: Prompt Budget and Trainable Parameter Accounting

Response: We thank the reviewer for pointing out the ambiguity in prompt configuration and parameter accounting. We agree that precise parameter accounting is essential for assessing the efficiency and reproducibility of DHAFT.

Following your suggestion, we clarify that DHAFT optimizes only stage-specific soft prompts while keeping the backbone frozen. As defined in Section 3.3.2, each stage contains P(i)∈Rm×dP^{(i)}\in\mathbb{R}^{m\times d}, where mm denotes the per-stage prompt length and dd denotes the hidden dimension. For GG stages, the total prompt budget is M=GmM=Gm, yielding Ntrainable=GmdN_{\mathrm{trainable}}=Gmd trainable parameters. Prompt propagation introduces no additional trainable parameters.

TaskStagesPer-Stage LengthsTotal VectorsTrainable ParamsTotal ParamsRatio (%)
Commonsense3(136, 136, 136)4081,671,1688,031,932,4160.020807
Complex Reasoning3(136, 136, 136)4081,671,1688,031,932,4160.020807

As shown above, both reasoning settings use 408 prompt vectors, corresponding to 1,671,168 trainable parameters and 0.020807% of the complete model. This is consistent with the approximately 0.02% parameter budget reported in Tables 1–2. The denominator includes 8,030,261,248 frozen backbone parameters and 1,671,168 trainable prompt parameters.

The prompt configuration and parameter accounting in Appendix A.7 (Table 10) are highlighted in red, with the corresponding clarifications highlighted in blue.


W2.2: Exact Checkpoints and Generation Settings

Response: We thank the reviewer for highlighting the importance of reproducible evaluation settings. We agree that specifying exact backbone checkpoints, prompt templates, decoding configurations, and generation budgets is essential for ensuring fair and reproducible comparisons. Following your suggestion, we clarify the reference evaluation protocols for Commonsense and Complex Reasoning.

For Commonsense Reasoning, the LLM-Adapters and DoRA reference protocols use instruction–response generation with beam search and answer extraction. For Complex Reasoning, we specify autoregressive generation with benchmark-specific answer extraction. Qwen3-8B is evaluated using its official chat template, with Non-Thinking Mode for commonsense reasoning and Thinking Mode for complex reasoning. The configurations are summarized below.

BackboneCheckpoint IDPrompt TemplateThinking ModeDecoding SettingsMax New Tokens
LLaMA-7Byahma/llama-7b-hfInstruction–ResponseN/ABeam=4, do_sample=False32
LLaMA-13Bhuggyllama/llama-13bInstruction–ResponseN/ABeam=4, do_sample=False32
Llama-2-7B (Commonsense)meta-llama/Llama-2-7b-hfInstruction–ResponseN/ABeam=4, do_sample=False32
Llama-3-8B (Commonsense)meta-llama/Meta-Llama-3-8BInstruction–ResponseN/ABeam=4, do_sample=False32
Llama-2-7B (Complex)meta-llama/Llama-2-7b-hfNormalized QAN/AGreedy, do_sample=False512 / 1024
Llama-3-8B (Complex)meta-llama/Meta-Llama-3-8BNormalized QAN/AGreedy, do_sample=False512 / 1024
Qwen3-8B (Commonsense)Qwen/Qwen3-8BOfficial Chat TemplateDisabledChoice LLN/A
Qwen3-8B (Complex)Qwen/Qwen3-8BOfficial Chat TemplateEnabledSampling, T=0.6, Top-p=0.95, Top-k=204096 / 8192

We further distinguish published reference configurations, reproduced baselines, and our evaluations to clarify result provenance and comparability. Regarding backbone coverage, we have additionally included Qwen3-8B on the eight commonsense benchmarks in Table 1, complementing its complex reasoning results in Table 2. DHAFT achieves 87.64% average accuracy with 0.0204% trainable parameters on Qwen3-8B for commonsense reasoning. The corresponding results are discussed in W2.3.

The checkpoint and evaluation configurations in Appendix A.7 (Table 13) are highlighted in red, with the corresponding explanations highlighted in blue. Baseline provenance is further clarified in Appendix A.6.1.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.