acceptodds
Under review as a conference paper at ICLR 2027

Training-Induced Reasoning Directions: Uncovering Reusable Structure in Post-Training Reasoning

Abstract

Reasoning post-training substantially improves the problem-solving ability of large language models, yet the internal structure that supports these gains remains poorly understood. We introduce Training-Induced Reasoning Directions (TIRDs), fixed activation directions extracted from computational differences between models before and after post-training. First, TIRD isolates reusable capability-relevant information: injecting it into the Base model improves reasoning accuracy by up to 50.9 percentage points, while suppressing it substantially degrades the post-trained model. Second, TIRD reveals how this information supports reasoning during generation: it remains functionally important throughout the reasoning process and provides strong predictive support for key reasoning decisions, whose probabilities drop by over 50 percentage points on average under suppression. Third, we introduce AdaTIRD for difficulty-adaptive reasoning allocation. AdaTIRD reduces generation by up to 27.8% while preserving performance close to the trained model, without additional post-training. Together, these results reveal reusable functional structure in post-training reasoning capability that can be isolated, understood, and directly controlled at inference time. Our code is available at https://anonymous.4open.science/r/TIRD-A36Bhttps://anonymous.4open.science/r/TIRD-A36B

Primary area:
interpretability and explainable AI
Keywords:
Large Language Models, Post-Training, Representation Engineering, Inference-Time Control

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_uEorOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper extracts fixed activation directions from differences between pre- and post-training checkpoints on shared reasoning trajectories. Adding these directions to the starting model recovers substantial accuracy gains. Subtracting them from the trained model reduces performance. Experiments cover six checkpoint pairs involving RL and on-policy distillation, with additional transfer from mathematics to code. The authors also examine token-level effects and propose AdaTIRD, which uses supplied difficulty labels to vary suppression strength and reduce generation length.

Critical strengths

The recovery experiment is the strongest part of the paper. Adding the OPD-derived direction raises Qwen3-4B Base’s AIME24 accuracy from 18.3% to 69.2%, with no weight updates. Random-direction and subtraction controls help establish that the extracted orientation matters. I also find the mathematics-to-code transfer useful. It shows that the intervention is not simply recovering behavior specific to the extraction task.

Critical weaknesses

I am less convinced by the explanation of what TIRD contributes during reasoning. Response length changes substantially alongside accuracy. The experiments therefore leave open how much of the gain comes from additional computation. The token analysis has a separate limitation: positions are selected for large probability drops. Table 6 shows comparable drops for ordinary-language tokens, but provides no category frequencies over the full responses. This makes the interpretation of the selected examples difficult to assess.

For AdaTIRD, the missing comparison is fixed suppression at the same average token cost. The decoding baselines do not answer whether assigning different coefficients by difficulty actually helps. This matters more to my assessment than further comparisons with temperature schedules.

Questions for authors
  • Does Base+TIRD retain an accuracy advantage when actual generated-token expenditure is matched?

  • What fraction of each token category passes the probability-drop threshold in Section 3.3? Do category differences remain after matching the original target-token probabilities?

  • On MATH500, how does AdaTIRD compare with a fixed coefficient selected on held-out data, at similar mean output length? This would help establish the contribution of difficulty-dependent allocation

4 comments

We sincerely thank the reviewer for the thoughtful evaluation and constructive feedback. We particularly appreciate the insightful questions regarding the relationship between reasoning capability and generation cost, the token-level effects of TIRD, and the effectiveness of difficulty-adaptive suppression. To further investigate these questions, we conducted additional experiments and extended our analyses as detailed below.

1. Reasoning Accuracy Under Comparable Generation Costs

We thank the reviewer for this valuable question. To investigate whether the accuracy improvements achieved by TIRD arise simply from increased token expenditure, we constructed three alternative steering directions using RepE [1]: increasing response length, suppressing EOS, and encouraging reasoning behavior.

We applied these directions to the model and obtained the following results.

MethodAIME24AIME25
Base19.17% / 2,93120.00% / 3,113
Increasing response length36.67% / 10,98230.83% / 10,138
Suppressing EOS35.00% / 10,63331.67% / 9,535
Encouraging reasoning behavior25.00% / 6,51419.17% / 5,969
TIRD67.50% / 12,42358.33% / 13,993

As shown in the results, both the response-length direction and the EOS-suppression direction substantially increase generation length, reaching token expenditures close to that of TIRD. Although these directions improve accuracy to some extent, their improvements remain far smaller than those achieved by TIRD.

This demonstrates that TIRD enhances reasoning capability, rather than merely increasing reasoning effort.

As analyzed in Section 3 of our paper, what changes during model reasoning is the way the model thinks. This behavioral change resembles observations made during reinforcement learning, such as those reported for DeepSeek-R1-Zero [2]. As training progresses, the model generates longer responses and becomes more inclined to carefully check its reasoning rather than directly produce an answer.

Reasoning capability and reasoning effort are inherently related. Additional reasoning effort can indeed increase the probability of arriving at the correct answer, but genuinely improving the model's capability requires effective reasoning rather than merely generating more tokens.

This is precisely what TIRD enables. TIRD shifts the model's internal activations toward representations that favor reflective reasoning, thereby activating genuinely effective emergent reasoning capabilities, rather than simply using more tokens to guess the answer.

2. Token Category Frequencies and Probability-Matched Analysis

We appreciate the reviewer's insightful suggestion. To provide a more comprehensive understanding of the token-level effects of TIRD, we extended our analysis to token categories across the full responses and examined probability changes under different original Target-token probability ranges.

Using Qwen3-4B RL as the Target model, we first collected statistics for the different token categories.

CategoryToken-type–category groupsShare of all groupsMean drop > 0.3Fraction
Mathematical content3,65548.84%37210.18%
Reasoning control86911.61%11012.66%
Discourse structure4405.88%6314.32%
Formatting4566.09%337.24%
Ordinary language2,06327.57%22010.66%

To analyze token probability changes in greater detail across different initial probability ranges, we grouped tokens according to their original sampling probabilities under Target and calculated the following statistics.

Original Target probabilityOrdinary languageMathematical contentReasoning controlDiscourse structureFormatting
[0%, 10%)−0.00230.0043−0.00350.0017−0.0115
[10%, 20%)0.04380.04540.07030.0494−0.0173
[20%, 30%)0.10520.11010.10890.11060.0700
[30%, 40%)0.14770.13850.14540.17170.1888
[40%, 50%)0.15980.17320.17420.19550.1116
[50%, 60%)0.17300.19460.18070.19330.0886
[60%, 70%)0.18760.20550.23530.20340.1075
[70%, 80%)0.21470.18190.26690.17450.1141
[80%, 90%)0.21380.13960.20150.24400.1021
[90%, 100%]0.07690.06670.12780.07630.0443

One important consideration is that averaging probability changes inevitably obscures finer-grained information. Consequently, it is difficult to determine from these averaged statistics alone how suppressing TIRD in activation space specifically affects the model and its reasoning decisions. This is precisely why we focus on tokens with probability changes greater than 0.3 in Section 3.3 of our paper.

To further investigate this issue, we compared the changes in predictive entropy and KL divergence between token groups with mean probability drops greater than 0.3 and those below this threshold.

Token groups with mean probability drop > 0.3

CategoryGroupsMean probability decreaseMean entropy changeMean KL change
Mathematical content3720.42101.04660.7330
Reasoning control1100.40801.05140.6839
Discourse structure630.39211.01110.6407
Formatting330.41030.59830.7080
Ordinary language2200.41331.11420.6929

Token groups with mean probability drop ≤ 0.3

CategoryGroupsMean probability decreaseMean entropy changeMean KL change
Mathematical content3,2830.08870.48230.2330
Reasoning control7590.08860.72420.4292
Discourse structure3770.07790.57190.3646
Formatting4230.05070.21340.1849
Ordinary language1,8430.08740.66890.3327

These results explain why we focus our analysis on tokens with probability drops greater than 0.3. As shown in the tables, these positions exhibit substantially larger changes in mean predictive entropy and KL divergence than other positions. Such larger distributional changes suggest that these positions are more likely to play a causal role in altering the model's reasoning trajectory, allowing us to better understand how TIRD influences the reasoning process.

To further investigate the role of these tokens, we conducted an additional experiment using Qwen3-4B RL on AIME24. Specifically, we let the Target model generate responses through autoregressive sampling. Whenever the probability of a sampled token under Target exceeded its probability under Target−TIRD by more than 0.3, we replaced that token with the highest-probability token predicted by Target−TIRD at the same position. The results are shown below.

ConditionAccuracyMean Length
Target62.50%11,830.24
Target with Token Replacement36.67%9,281.90

Across all generated sequences, only 5.74% of tokens were replaced, yet this intervention caused a substantial 25.83 percentage-point drop in accuracy, from 62.50% to 36.67%. These results demonstrate that tokens whose probabilities are substantially suppressed by removing TIRD play an important role in the reasoning process. Replacing only a small fraction of these tokens with the most probable alternatives under Target−TIRD is sufficient to substantially degrade reasoning performance, providing further evidence of their functional importance in shaping the model's reasoning trajectories.

3. AdaTIRD vs. Fixed-Coefficient Suppression

We thank the reviewer for suggesting this important comparison. To demonstrate the effectiveness of difficulty-dependent coefficient allocation relative to fixed-coefficient designs, we compared AdaTIRD against fixed-coefficient suppression baselines.

We also introduced two additional experimental settings using pretrained difficulty classifiers, namely Jev API and the open-source JevK5, to evaluate AdaTIRD when ground-truth difficulty labels are unavailable.

MethodAccuracyMean tokensToken savings
Target95.50%3,382.180.00%
Fixed α = −0.294.20%2,960.0518.39%
Fixed α = −0.493.20%2,498.7730.85%
Fixed α = −0.691.25%2,001.3140.83%
Fixed α = −0.888.95%1,756.0448.08%
Fixed α = −183.65%1,522.7054.98%
Temperature95.75%3,416.46−1.01%
Length94.50%3,264.393.48%
Temperature + Length94.10%3,311.762.08%
AdaTIRD Standard95.30%2,921.3713.62%
AdaTIRD + Jev API94.15%2,529.0025.23%
AdaTIRD + JevK593.40%2,243.7333.66%

The results demonstrate that even without access to ground-truth difficulty labels, and relying only on existing difficulty classifiers without additional training, our method achieves a clear advantage in the trade-off between accuracy and reasoning length, demonstrating its practical applicability.

Furthermore, compared with the fixed α = −0.2 baseline, AdaTIRD Standard achieves higher accuracy at a comparable generation length. Specifically, AdaTIRD Standard achieves 95.30% accuracy with 2,921 tokens, whereas fixed α = −0.2 achieves 94.20% accuracy with 2,960 tokens.

Similarly, the Jev API and JevK5 settings also achieve higher accuracy than fixed-coefficient baselines with comparable generation lengths. These results demonstrate the value of our difficulty-dependent coefficient allocation strategy, showing that assigning intervention strengths according to problem difficulty provides advantages over applying a uniform coefficient to all problems.

We sincerely appreciate the reviewer's thoughtful feedback and valuable suggestions. We hope that these additional experiments and analyses further demonstrate the effectiveness of TIRD and the benefits of difficulty-adaptive reasoning allocation.

References

[1] Andy Zou et al. Representation Engineering: A Top-Down Approach to AI Transparency. 2023.

[2] DeepSeek-AI et al. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. 2025.

Reviewer_uEorOpenReview ↗

reply

Thanks for this feedback; I maintain my original scores, but I will update my confidence

Reviewer_3UX8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Training-Induced Reasoning Directions, obtained by averaging layer-wise activation-update differences between a base model and its post-trained counterpart on shared reasoning traces.

Critical strengths
  1. The paper recovers substantial post-training gains across RL- and OPD-trained models, while random and reversed-direction controls support directional specificity.

  2. The research topic is well-motivated, the proposed TIRD uses matched Base–Target computations on identical reasoning prefixes, reducing confounding from different generated trajectories.

Critical weaknesses
  1. The proposed TIRD injection increases response length, while suppression shortens it. The accuracy changes may partly result from allocating more or less test-time computation rather than adding or removing a distinct reasoning capability.

  2. The experiment is limited, the used key results only rely on AIME24/25, each containing only 30 problems, with no confidence intervals or repeated extraction/evaluation seeds reported.

  3. The supplied MATH500 levels are unavailable for typical inputs, evaluation with a learned difficulty predictor and stronger adaptive-computation baselines is needed.

Questions for authors
  1. How does TIRD compare with directions explicitly constructed to increase response length, suppress EOS, or encourage continued reasoning?

  2. Have the local alignment assumptions in the theoretical analysis been measured empirically, especially for the relatively large intervention coefficient α\alpha =1

5 comments

We sincerely thank the reviewer for the thoughtful review and constructive suggestions. We appreciate the opportunity to provide further experimental evidence regarding the effectiveness, statistical reliability, and theoretical analysis of TIRD. We have conducted additional experiments to address the concerns raised and present our detailed responses below.

1. Comparison with Directions for Increasing Generation Length

We thank the reviewer for this insightful suggestion. To investigate how TIRD compares with directions explicitly designed to increase generation length or encourage continued reasoning, we constructed three alternative steering directions using RepE [1]: increasing response length, suppressing EOS, and encouraging reasoning behavior.

We applied these directions to the model and obtained the following results.

MethodAIME24AIME25
Base19.17% / 2,93120.00% / 3,113
Increasing response length36.67% / 10,98230.83% / 10,138
Suppressing EOS35.00% / 10,63331.67% / 9,535
Encouraging reasoning behavior25.00% / 6,51419.17% / 5,969
TIRD67.50% / 12,42358.33% / 13,993

The intervention results demonstrate that TIRD does much more than simply increase the model's output length. Although these alternative directions can increase reasoning length and improve accuracy to some extent, their improvements remain far smaller than those achieved by TIRD. This shows that TIRD enhances reasoning capability, rather than merely increasing reasoning effort.

As analyzed in Section 3 of our paper, what changes during model reasoning is the way the model thinks. This behavioral change resembles observations made during reinforcement learning, such as those reported for DeepSeek-R1-Zero [2]. As training progresses, the model generates longer responses and becomes more inclined to carefully examine its reasoning rather than directly produce an answer.

Reasoning capability and reasoning effort are inherently related. Additional reasoning effort can indeed increase the probability of arriving at the correct answer, but genuinely improving the model's capability requires effective reasoning rather than merely spending more tokens.

This is precisely what TIRD enables. TIRD shifts the model's internal activations toward representations that favor reflective reasoning, thereby activating genuinely effective emergent reasoning capabilities, rather than simply using more tokens to guess the answer.

2. Statistical Reliability and Evaluation on a Larger Benchmark

We appreciate the reviewer's valuable suggestion to strengthen the statistical evaluation.

To address this concern, we conducted repeated experiments using three extraction datasets: DeepMath, MATH500, and DAPO-17K. For each dataset, we sampled 256 examples and repeated the extraction procedure five times using different random seeds. We calculated the standard deviation and 95% confidence interval of the experimental results.

Extraction Data & InterventionAIME24 Accuracy (%)AIME24 LengthAIME25 Accuracy (%)AIME25 Length
Base19.172,93120.003,113
Target62.5011,83056.6712,357
DeepMath Base + TIRD65.50 ± 2.79 [62.04, 68.96]12,489 ± 454 [11,925, 13,053]57.50 ± 2.12 [54.87, 60.13]13,825 ± 157 [13,630, 14,020]
DeepMath Target − TIRD20.50 ± 1.26 [18.94, 22.06]4,053 ± 224 [3,775, 4,331]20.50 ± 2.47 [17.43, 23.57]4,172 ± 221 [3,898, 4,446]
MATH500 Base + TIRD66.17 ± 2.17 [63.48, 68.86]13,902 ± 347 [13,471, 14,333]60.67 ± 1.24 [59.13, 62.21]15,522 ± 620 [14,752, 16,292]
MATH500 Target − TIRD12.67 ± 2.31 [9.80, 15.54]3,376 ± 529 [2,719, 4,033]13.00 ± 3.26 [8.95, 17.05]3,283 ± 158 [3,087, 3,479]
DAPO Base + TIRD64.17 ± 3.97 [59.24, 69.10]12,301 ± 563 [11,602, 13,000]55.83 ± 4.46 [50.29, 61.37]13,779 ± 674 [12,942, 14,616]
DAPO Target − TIRD17.17 ± 2.09 [14.57, 19.77]3,723 ± 189 [3,488, 3,958]17.17 ± 2.33 [14.28, 20.06]3,915 ± 199 [3,668, 4,162]

Furthermore, we conducted additional experiments on Omni-MATH-Rule [3], which contains 2,821 problems.

SettingAccuracy (%)Mean length (tokens)
Base37.342,082
Base + TIRD61.979,006
Target62.819,351
Target − TIRD33.673,245

The results demonstrate that TIRD provides stable control over the model's reasoning capability. The observed performance improvements are not merely accidental effects arising from evaluation on a small number of problems.

3. Additional Comparisons for AdaTIRD

We also appreciate the reviewer's suggestion to evaluate AdaTIRD with difficulty predictors and stronger baselines.

To evaluate our method when ground-truth problem difficulty is unavailable, we introduced two additional experimental settings using pretrained difficulty classifiers, namely Jev API and the open-source JevK5. We further compared these settings against fixed-coefficient suppression baselines.

MethodAccuracyMean tokensToken savings
Target95.50%3,382.180.00%
Fixed α = −0.294.20%2,960.0518.39%
Fixed α = −0.493.20%2,498.7730.85%
Fixed α = −0.691.25%2,001.3140.83%
Fixed α = −0.888.95%1,756.0448.08%
Fixed α = −183.65%1,522.7054.98%
Temperature95.75%3,416.46−1.01%
Length94.50%3,264.393.48%
Temperature + Length94.10%3,311.762.08%
AdaTIRD Standard95.30%2,921.3713.62%
AdaTIRD + Jev API94.15%2,529.0025.23%
AdaTIRD + JevK593.40%2,243.7333.66%

The results show that even without access to ground-truth difficulty labels, and relying only on existing difficulty classifiers without additional training, our method still achieves a clear advantage in the trade-off between accuracy and reasoning length. These results demonstrate the practical applicability of AdaTIRD.

4. Empirical Evaluation of the Local Alignment Assumptions

We thank the reviewer for raising this interesting theoretical question. To empirically examine the local alignment assumptions under the relatively large intervention coefficient α = 1, we conducted an additional analysis of the KL divergence between Base, Base+TIRD, and Target.

Specifically, we analyzed 30 responses from AIME24 and compared the KL divergence before and after injecting TIRD with α = 1.

Position subsetNumber of positionsKL before injectionKL after injectionChange
KL-decreasing positions162,0070.1100740.036877−0.073197
KL-increasing positions147,8710.0539740.184001+0.130026
All positions309,8780.0833040.107083+0.023780

The results show that different token positions exhibit different changes in KL divergence. In particular, 52.28% of token positions become more closely aligned with the Target distribution after TIRD injection.

We further examined the tokens associated with decreases in KL divergence.

TokenMean KL before injectionMean KL after injectionMean decrease
**\n2.0458980.5181181.527780
}\n1.0617410.2057520.855988
First1.1796230.3720970.807526
Wait0.9038790.2090170.694862
But0.6576450.1922930.465351
Alternatively0.6357060.2338360.401871
Therefore0.5519780.2186230.333355
Let0.4399390.2240960.215843
␠denote0.4270030.2497260.177276
␠check0.2849130.2036410.081273
␠coordinates0.2794850.2523770.027108

We also examined the tokens associated with increases in KL divergence.

TokenMean KL before injectionMean KL after injectionMean increase
␠go0.3104850.8055840.495099
␠giving0.1674350.6539270.486493
␠constraint0.1981490.6661760.468027
␠give0.0615030.5085150.447012
␠piece0.1319000.5454640.413564
␠still0.1180630.5289980.410935
␠powers0.0745590.4700870.395528
␠assignments0.0970990.4663200.369221
␠Compute0.0877740.4557880.368013
At0.2171870.5601930.343006

We observe that most of the representative tokens with decreased KL divergence are associated with reasoning transitions, whereas many tokens with increased KL divergence are associated with the progression of reasoning.

This indicates that TIRD injection makes the model's output behavior more closely resemble that of Target at important reasoning-related positions. These results demonstrate that, even with an intervention coefficient of α = 1, TIRD can ensure local alignment at key positions.

We sincerely appreciate the reviewer's constructive feedback, which has motivated these additional experiments and analyses. We hope that the new evidence further strengthens the empirical and theoretical understanding of TIRD.

References

[1] Andy Zou et al. Representation Engineering: A Top-Down Approach to AI Transparency. 2023.

[2] DeepSeek-AI et al. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. 2025.

[3] Bofei Gao et al. Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Models. ICLR, 2025.

Dear Reviewer 3UX8,

We are writing to ask whether our rebuttal has addressed your concerns. We would welcome the opportunity to discuss any remaining concerns.

Sincerely,

Authors of Submission 32136

Reviewer_XCndOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TIRD, which extracts reusable activation directions by averaging transformer block-update differences between Base and post-trained models (RL/OPD). The authors show that injecting TIRD into Base models substantially improves reasoning accuracy, while suppressing it in Target models degrades performance. They further investigate its effects across reasoning stages and on token-level predictions, and demonstrate cross-task transfer from mathematics to coding. Finally, the paper introduces AdaTIRD, which adjusts TIRD suppression based on problem difficulty to reduce generation length while largely preserving reasoning accuracy.

Critical strengths
  1. Extensive Experimental Analysis. The authors evaluate TIRD across multiple model scales and training methods (RL and OPD), with additional analyses of directional specificity, intervention positions, and token-level effects, providing evidence for its functional relevance.

  2. Potential for Inference-time Control. AdaTIRD demonstrates that manipulating the extracted directions can reduce generation length while largely preserving reasoning accuracy, suggesting a potential application for inference-time efficiency.

Critical weaknesses
  1. Limited Novelty. Extracting and leveraging activation differences has been explored in prior work, such as [1] and [2]. The paper offers limited new insights beyond these approaches.

  2. Insufficient Methodological Justification. The paper lacks comparisons with existing direction extraction methods, such as directly averaging hidden-state differences. The necessity and advantages of the proposed extraction strategy remain unclear.

  3. Reasoning Capability vs. Reasoning Effort. TIRD affects both accuracy and generation length, but these effects are not disentangled. It is unclear whether improvements reflect genuine reasoning capability or simply increased reasoning effort. Length-matched ablations are needed.

  4. Limited Practical Value and Generalization. Given the availability of the trained Target, the motivation for Base + TIRD is unclear. AdaTIRD relies on predefined difficulty labels and lacks fixed-coefficient and stronger inference-budget baselines. Generalization is only demonstrated from math to closely related coding tasks.

[1] Narrow Finetuning Leaves Clearly Readable Traces in Activation Differences, ICLR 2026.

[2] Steering Llama 2 via Contrastive Activation Addition, ACL 2024.

Questions for authors
  1. Extraction Method. Could the authors compare TIRD with existing approaches, particularly direct activation differences [1] and CAA [2]? These comparisons would help establish the necessity and advantages of the proposed block-update differences and mean pooling.

  2. Reasoning Capability. Do the accuracy improvements persist under matched generation lengths? How can the authors distinguish genuine reasoning capability gains from increased reasoning effort?

  3. Practical Value. Given that the trained Target model is already available, what is the practical advantage of using Base + TIRD? Can AdaTIRD outperform fixed-coefficient suppression and other reasoning-budget baselines without predefined difficulty labels?

  4. Generalization and Robustness. The paper only evaluates cross-task transfer from mathematics to coding, which share similar reasoning patterns. Could the authors investigate generalization to more diverse tasks, such as general QA, planning, and tool use? Additionally, how sensitive is TIRD to the extraction data distribution and sample size? Ablations using different extraction datasets and varying numbers of samples (e.g., 16, 32, 64, 128, 256, and 512) would help evaluate the stability, sample efficiency, and cross-task generalizability of the extracted directions.

[1] Narrow Finetuning Leaves Clearly Readable Traces in Activation Differences, ICLR 2026.

[2] Steering Llama 2 via Contrastive Activation Addition, ACL 2024.

5 comments

We sincerely thank the reviewer for the careful review and valuable feedback. We appreciate the opportunity to further clarify the novelty, methodological design, practical value, and generalizability of our work. To address the questions raised, we have conducted extensive additional experiments and provide our detailed responses below.

1. Novelty and Comparison with Existing Extraction Methods

We thank the reviewer for highlighting these relevant studies and providing an opportunity to further clarify the novelty and contributions of our work.

Please allow us to clarify the novelty of our work. Compared with previous studies that extract and leverage activation differences, our research focuses specifically on activation differences between models before and after training with on-policy sampling-based methods, a setting that has not been explored in previous studies of activation differences. As a continuation of interpretability research on on-policy learning, our work helps researchers better understand how the activation space changes before and after model training.

Furthermore, previous studies have primarily focused on how activation directions change the model's output tendencies. These studies can explain that the model's behavior has changed, but do not specifically explain how these changes arise. Our work takes the analysis further to the token level, examining how activation differences alter generation trajectories through changes in token-level sampling during reasoning.

Regarding the extraction method, our approach differs from previous methods such as ADL [1], CAA [2], and RepE [3] by removing the influence of residual connections when computing layer-wise activation differences. This allows our method to better capture how changes in the parameters of individual Transformer blocks after training affect the activation space.

To further demonstrate the advantages of our extraction strategy, we compared TIRD with these existing approaches on Qwen3-4B. The following table reports accuracy and average generation length on AIME24 and AIME25.

MethodAlphaAIME24AIME25
BaseNone19.17% / 2,93120.00% / 3,113
TargetNone62.50% / 11,83056.67% / 12,357
ADL-Layer10.00% / 40,9600.00% / 40,960
ADL-Layer0.135.00% / 6,49525.00% / 5,797
ADL-LayerTIRD23.33% / 3,30122.50% / 3,123
ADL-All10.00% / 40,9600.00% / 40,960
ADL-All0.0345.83% / 24,45938.33% / 26,155
ADL-AllTIRD21.67% / 3,32620.83% / 2,860
CAA-Layer124.17% / 2,97320.00% / 3,203
CAA-LayerTIRD20.83% / 2,87020.00% / 3,025
CAA-All10.00% / 40,6000.00% / 40,592
CAA-All0.0325.83% / 2,94024.17% / 2,987
CAA-AllTIRD0.00% / 40,9600.00% / 40,960
RepE-Layer121.67% / 3,32620.83% / 2,860
RepE-LayerTIRD20.33% / 3,04218.33% / 2,853
RepE-All10.00% / 40,9600.00% / 40,960
RepE-All0.0119.17% / 2,85520.00% / 2,520
RepE-AllTIRD0.00% / 22,0110.00% / 33,469
TIRD167.50% / 12,42358.33% / 13,993

We injected the directions extracted by each method into Qwen3-4B during generation. Two injection settings were considered: applying the directions to selected layers, following the settings used in the original methods (-Layer), and applying them across all layers, as in our approach (-All).

To account for differences in vector norms, we introduced an intervention coefficient Alpha to control direction magnitude. For example, Alpha = 0.1 scales the direction by 0.1, while Alpha = TIRD adjusts its norm to match that of TIRD. We performed a grid search to identify the optimal layer and Alpha settings.

The results show that, apart from ADL-All and ADL-Layer, most alternative direction injections either cause the model's generation to collapse or have little effect on its behavior. We believe this is because ADL, like our method, focuses on changes before and after model training, whereas directions constructed from positive and negative samples cannot directly capture these training-induced changes.

In terms of direction quality, ADL still requires careful magnitude adjustment to effectively influence the model. Otherwise, the intervention may have little effect or cause generation to collapse. Moreover, even after norm adjustment, ADL-All cannot activate the model's reasoning capability as effectively as TIRD, because its extracted directions contain redundant information accumulated in the residual states.

In contrast, TIRD can be directly applied immediately after extraction, providing a plug-and-play capability that previous direction extraction methods cannot achieve. These results demonstrate the advantages of our extraction strategy and further distinguish TIRD from existing activation-direction methods.

2. Reasoning Capability Beyond Increased Generation Length

We appreciate this insightful question regarding the relationship between reasoning capability and generation length.

To investigate whether simply changing the model's output length can improve its accuracy, we used RepE to construct three steering directions corresponding to increasing response length, suppressing EOS, and encouraging reasoning behavior. We then applied these directions to the model and obtained the following results.

MethodAIME24AIME25
Base19.17% / 2,93120.00% / 3,113
Increasing response length36.67% / 10,98230.83% / 10,138
Suppressing EOS35.00% / 10,63331.67% / 9,535
Encouraging reasoning behavior25.00% / 6,51419.17% / 5,969
TIRD67.50% / 12,42358.33% / 13,993

The intervention results show that simply increasing reasoning length can improve accuracy to some extent, but the improvement is far smaller than that achieved by TIRD. This demonstrates that TIRD enhances reasoning capability, rather than merely increasing reasoning effort.

As analyzed in Section 3 of our paper, what changes during model reasoning is the way the model thinks. This behavioral change is similar to observations made during reinforcement learning, such as those reported for DeepSeek-R1-Zero [4]. As training progresses, model outputs become longer, and the model becomes more inclined to carefully check its reasoning rather than directly produce an answer.

Reasoning capability and reasoning effort are inherently related. Additional reasoning effort can indeed increase the probability that a model arrives at the correct answer, but genuine improvement in reasoning capability requires effective reasoning rather than merely generating more tokens.

This is precisely what TIRD enables. TIRD shifts the model's internal activations toward representations that favor reflective reasoning, thereby activating genuinely effective emergent reasoning capabilities, rather than simply using more tokens to guess the answer.

3. Practical Value of TIRD and AdaTIRD

Thank you for raising this important question. We would like to further clarify the motivation and practical applications of our method.

We apologize if our original presentation led to a misunderstanding of our approach. Our purpose in studying Base+TIRD is to investigate whether the reasoning capability of the Base model can be activated through a feature direction and how TIRD changes the model's reasoning behavior. Based on these investigations, we further discovered that the TIRD direction remains important in the Target model, and that subtracting TIRD from Target can suppress its reasoning capability.

Once a model has completed training, its reasoning capability becomes internalized in the model. However, this often leads to overthinking. Specifically, when facing simple problems, the model may already be able to obtain the correct answer with a short reasoning process, but its internalized and relatively fixed reasoning capability can cause unnecessary hesitation and repeated checking.

AdaTIRD is designed precisely to address this issue. Its practical application is not Base+TIRD, but Target − αTIRD. Our design aims to allocate reasoning capability according to problem difficulty, matching the level of reasoning capability to the demands of each problem. This prevents the model from repeatedly performing unnecessary checks on simpler problems and thereby reduces inference costs.

To evaluate the effectiveness of our approach when problem difficulty is unavailable, we introduced two additional experimental settings using pretrained difficulty classifiers: Jev API and the open-source JevK5. We compared these settings against fixed-coefficient suppression baselines.

MethodAccuracyMean tokensToken savings
Target95.50%3,382.180.00%
Fixed α = −0.294.20%2,960.0518.39%
Fixed α = −0.493.20%2,498.7730.85%
Fixed α = −0.691.25%2,001.3140.83%
Fixed α = −0.888.95%1,756.0448.08%
Fixed α = −183.65%1,522.7054.98%
Temperature95.75%3,416.46−1.01%
Length94.50%3,264.393.48%
Temperature + Length94.10%3,311.762.08%
AdaTIRD Standard95.30%2,921.3713.62%
AdaTIRD + Jev API94.15%2,529.0025.23%
AdaTIRD + JevK593.40%2,243.7333.66%

The results show that even without access to ground-truth difficulty labels, and relying only on existing difficulty classifiers without additional training, our method still achieves a clear advantage in the trade-off between accuracy and reasoning length. These findings demonstrate the practical applicability of our method.

4. Generalization and Robustness of TIRD

We appreciate the reviewer's valuable suggestions regarding broader task generalization and extraction robustness. Following these suggestions, we conducted additional experiments across diverse tasks, extraction datasets, and sample sizes.

Generalization to Diverse Tasks

To further demonstrate the generalization ability of TIRD, we evaluated it on CommonsenseQA [5] for general question answering, PlanBench [6] for planning, and BFCL [7] for tool use.

SettingCommonsenseQAPlanBenchBFCL
Base77.66% / 1322.15% / 57381.15% / 29
Target78.13% / 1919.73% / 2,90182.03% / 35
Base + Math TIRD78.93% / 21328.07% / 5,25481.25% / 31
Target − Math TIRD75.80% / 1671.33% / 77579.49% / 35
Base + Task-specific TIRD77.60% / 16911.27% / 3,06080.66% / 30
Target − Task-specific TIRD77.42% / 1502.56% / 92681.05% / 33

As shown in the results, TIRD exhibits effective capability modulation on planning tasks that require reasoning.

On PlanBench, injecting TIRD extracted from the same task into Base improves its accuracy to a level comparable to that of Target. More importantly, TIRD extracted from mathematics further improves performance, achieving 28.07% accuracy.

This demonstrates that the TIRD extracted by our method captures genuinely generalizable feature information, allowing improved reasoning information acquired from mathematical tasks to transfer to other tasks.

On CommonsenseQA and BFCL, the Target model directly answers the questions, indicating that the model considers these problems solvable without extensive reasoning. TIRD exhibits the same behavior. This demonstrates that TIRD does not simply increase reasoning expenditure and does not waste unnecessary computation on tasks that do not require substantial reasoning.

Robustness Across Extraction Datasets

To demonstrate the robustness of TIRD, we first performed ablation experiments on the extraction datasets. We sampled 256 examples from each of DeepMath, MATH500, and DAPO-17K and repeated the procedure five times.

Extraction Data & InterventionAIME24AIME25
Base19.17% / 2,93120.00% / 3,113
Target62.50% / 11,83056.67% / 12,357
DeepMath Base + TIRD(65.50 ± 2.79)% / 12,489 ± 454(57.50 ± 2.12)% / 13,825 ± 157
DeepMath Target − TIRD(20.50 ± 1.26)% / 4,053 ± 224(20.50 ± 2.47)% / 4,172 ± 221
MATH500 Base + TIRD(66.17 ± 2.17)% / 13,902 ± 347(60.67 ± 1.24)% / 15,522 ± 620
MATH500 Target − TIRD(12.67 ± 2.31)% / 3,376 ± 529(13.00 ± 3.26)% / 3,283 ± 158
DAPO Base + TIRD(64.17 ± 3.97)% / 12,301 ± 563(55.83 ± 4.46)% / 13,779 ± 674
DAPO Target − TIRD(17.17 ± 2.09)% / 3,723 ± 189(17.17 ± 2.33)% / 3,915 ± 199

As shown in the table, TIRDs extracted from different datasets produce similar effects. This demonstrates that the effectiveness of TIRD does not depend on a particular extraction dataset.

Robustness Across Extraction Sample Sizes

We further examined the effect of extraction sample size by sampling different numbers of examples from DeepMath. Each setting was repeated five times.

Extraction Data & InterventionAIME24AIME25
Base19.17% / 2,93120.00% / 3,113
Target62.50% / 11,83056.67% / 12,357
DeepMath 16 · Base + TIRD(65.83 ± 2.28)% / 13,134 ± 397(59.83 ± 3.46)% / 14,466 ± 424
DeepMath 16 · Target − TIRD(18.17 ± 1.24)% / 3,642 ± 202(17.17 ± 2.09)% / 3,834 ± 273
DeepMath 32 · Base + TIRD(65.17 ± 3.56)% / 11,671 ± 549(56.67 ± 2.28)% / 13,151 ± 392
DeepMath 32 · Target − TIRD(18.67 ± 1.26)% / 4,173 ± 364(21.00 ± 2.16)% / 4,049 ± 64
DeepMath 64 · Base + TIRD(63.33 ± 1.95)% / 12,296 ± 421(56.83 ± 1.09)% / 13,187 ± 310
DeepMath 64 · Target − TIRD(20.17 ± 2.85)% / 4,143 ± 167(20.50 ± 1.62)% / 3,973 ± 143
DeepMath 128 · Base + TIRD(62.67 ± 3.08)% / 13,128 ± 121(57.17 ± 2.47)% / 13,857 ± 255
DeepMath 128 · Target − TIRD(20.00 ± 2.57)% / 3,984 ± 165(19.67 ± 1.62)% / 4,014 ± 152
DeepMath 256 · Base + TIRD(65.50 ± 2.79)% / 12,489 ± 454(57.50 ± 2.12)% / 13,825 ± 157
DeepMath 256 · Target − TIRD(20.50 ± 1.26)% / 4,053 ± 224(20.50 ± 2.47)% / 4,172 ± 221
DeepMath 512 · Base + TIRD(60.33 ± 3.85)% / 12,013 ± 262(57.67 ± 3.51)% / 13,099 ± 306
DeepMath 512 · Target − TIRD(21.33 ± 2.25)% / 4,028 ± 196(21.33 ± 3.75)% / 4,127 ± 177

Remarkably, an effective TIRD can be extracted using as few as 16 examples. Across different extraction sample sizes, TIRD consistently provides stable control over the model's reasoning capability. This demonstrates that our extraction method does not depend on a particular number of extraction samples and provides further evidence of its generalizability and robustness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.