acceptodds
Under review as a conference paper at ICLR 2027

Understanding and Exploiting Anisotropy in Post-Training

Abstract

LLM post-training combines supervised fine-tuning (SFT), a mode-covering forward-KL objective, with reinforcement learning (RL), a mode-seeking reverse-KL objective. Frequency-weighted likelihood training leaves a well-known signature: anisotropy, in which a few residual channels carry disproportionately large activations. Anisotropy is widely documented and usually treated as a defect, yet its function and its interaction with post-training remain unclear. We first analyze it. A label-free outlier rule isolates about 5% of residual channels that are essential for language modeling: removing them raises perplexity from 10 to over , versus 35 for count-matched random channels. Yet they barely distinguish correct from incorrect reasoning. SFT reshapes them, whereas RL leaves them largely intact and adapts the complementary channels. These channels therefore form the model's coherence substrate, and reasoning adaptation happens elsewhere. We then exploit this. SphereGate learns one bounded gain per residual channel on a frozen backbone. Its activation-weighted gradients provably limit movement of high-energy coherence channels and leave the remaining channels free. With 0.1M trainable parameters, SphereGate outperforms parameter-efficient baselines by 2.0–7.3 points on MATH-500 across Qwen2.5 (0.5B–7B) and Llama-3-8B, is comparable or exceeds full-model GRPO. Anisotropy is not a defect but a division of labor that post-training can exploit.

Primary area:
reinforcement learning
Keywords:
Reinforcement Learning, Mechanistic Interpretability, Representational Anisotropy

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_YvHfOpenReview ↗
Rating
2
Confidence
1(−1)
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates anisotropic residual channels during LLM post-training and proposes SphereGate, a parameter-efficient method that learns channel-wise gains on a frozen backbone. The authors argue that highly anisotropic channels primarily support coherence, while the remaining channels are responsible for reasoning adaptation. Experiments across Qwen and Llama models report improvements on mathematical reasoning benchmarks.

Critical strengths
  1. I think the paper explores an interesting direction: using insights from model interpretability to guide post-training optimization.

  2. SphereGate is simple and parameter-efficient, and the experiments cover multiple model sizes and architectures.

Critical weaknesses
  1. I am not fully convinced by the proposed separation between coherence and reasoning channels. The subspace-overlap analysis does not establish that the selected channels contain little correctness-related information, since different representations can occupy the same subspace while remaining distinguishable. Similarly, the perplexity ablations show that these channels are important for language modeling, but do not necessarily establish their specific role in coherence. I find the current evidence insufficient to support the paper's central mechanistic interpretation.

  2. I have substantial concerns about the reproducibility and reliability of the reported results. The submission does not provide executable code or complete environment details for independent reproduction. More importantly, the numerical results in Table 1 appear inconsistent with the stated evaluation protocol. For example, the reported 70.0 ± 0.54 on MATH-500 cannot be obtained from three ordinary 500-question accuracy scores using standard mean and SD calculations, even accounting for rounding. Similar inconsistencies occur in other entries. These issues make it difficult to assess the reliability of the claimed improvements.

Questions for authors
  1. What additional evidence supports the proposed functional separation between coherence and reasoning channels? Can the authors provide stronger controls beyond subspace overlap and perplexity ablations?

  2. Can the authors provide the per-seed accuracy scores, evaluation denominators, and aggregation procedure used to produce Table 1, particularly the reported standard deviations? If it was me who misunderstood, I would be more than willing to increase my score.

Reviewer_x8gAOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies residual-channel anisotropy during LLM post-training and proposes SphereGate, a parameter-efficient method for RL adaptation. The authors select channels using an outlier threshold on absolute mean activations. They study subspace overlap between correct and incorrect rollouts and channel changes during SFT and GRPO. SphereGate freezes the backbone and learns positive, bounded, identity-initialized gains on the full residual state after each Transformer block. The paper provides a conditional energy-weighted stability bound and evaluates the method on mathematical reasoning tasks using Qwen2.5 and Llama models. SphereGate outperforms the evaluated PEFT baselines. Restricted-gain training and activation-removal experiments further examine the roles of the channel groups in reasoning adaptation and general-text prediction.

Critical strengths
  1. The method is simple and shows promising empirical results. SphereGate trains only channel-wise gains and introduces no learned channel mixing. It achieves the highest MATH-500 score on all four backbones in Table 1. Results on other math benchmarks also show gains over the evaluated PEFT methods. These findings support channel-wise residual scaling as an effective approach to reasoning adaptation.

  2. The ablations distinguish leaving channels unchanged from removing them. Table 4 separately examines restricted-gain training and inference-time activation removal. Complement-only training scores 69.4, close to 70.0 for all-channel training. This shows that most of the gain can be retained without directly updating the selected outlier channels. The count-matched removal controls and residual-norm restoration also strengthen the observation that the selected channels are important for general-text prediction.

Critical weaknesses

1. The analyses do not yet establish the claimed functional separation between coherence and reasoning. The large perplexity increase after selected-channel removal shows that these channels matter for text prediction. It does not directly identify a coherence-specific function. Count matching and norm restoration also do not match the removed activation energy or the intervention direction. Section 5.5 acknowledges these limits, but the abstract and conclusion still make a strong claim about a division of labor.

The subspace analysis also needs more direct calibration. Equation (46) shows that expected squared overlap depends on the subspace ranks and ambient channel dimension. The empirical ranks, spectral truncation rule, and centering procedure are not sufficiently specified. It is therefore difficult to assess how much of the high overlap follows from a high rank-to-dimension ratio. Smaller relative activation drift also need not imply smaller absolute drift. Proposition 1 provides a conditional stability bound. However, the paper does not sufficiently show whether actual training satisfies its conditions or whether default all-channel training produces the claimed selective adaptation. These gaps limit the support for the paper’s central mechanistic interpretation.

2. The advantage over simple scaling alternatives, and its dependence on the training budget, need more direct validation. The paper includes IA³ and other baselines, but it lacks a comparison that trains only existing normalization gains. Such gains act at different locations from SphereGate and are not an equivalent method. However, this comparison would help show whether full-residual gating offers benefits beyond general channel-wise scaling. In Table 4, matched complement-only training scores 67.5, while random-channel training scores 66.9. These results provide clues about channel selection, but they do not by themselves show that anisotropy explains the default method’s gains.

The main performance experiments also use 90 training steps. This budget alone does not show that full-model GRPO is undertrained. However, the current results do not establish whether the relative advantage persists with longer training. Comparable learning-rate search budgets and learning curves under longer, matched rollout budgets would make the comparison more convincing.

Questions for authors
  1. Mechanism evidence: Please report the empirical ranks, spectral truncation rule, and centering procedure used in the subspace analysis, with calibration based on Equation (46). During default SphereGate training, how do the selected and complementary groups compare in actual log-gain changes, absolute and relative activation drift, and energy-weighted displacement? Do these measurements support the selective-adaptation conditions described by Proposition 1, or should the proposition currently be read only as a conditional characterization?

  2. Baselines and training budget: With comparable learning-rate search budgets, how does SphereGate compare with training only normalization gains, IA³, and full-model GRPO? Do the main performance conclusions hold under longer, matched rollout budgets? Please report the corresponding learning curves and final scores.

Reviewer_AXFAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies anisotropic channels during SFT and RL, and proposes that they support coherence while the remaining channels support reasoning adaptation. Based on this, SphereGate learns channel-wise scaling factors on a frozen model. It achieves good results on several math benchmarks.

Critical strengths
  1. The method is simple and works well across several model sizes and families.

  2. The paper includes useful channel ablations and reports results across multiple training seeds.

Critical weaknesses
  1. I am not convinced by the separation between “coherence” and “reasoning” channels. Removing large-activation channels naturally causes more damage, and matching channel count or restoring the norm does not fully control this. The overlap comparison also needs controls for subspace dimension and rank. These results show that the selected channels are important, but do not clearly establish their specific function.

  2. It is unclear whether the anisotropy analysis explains why the method works. SphereGate trains all channels and does not use the proposed partition. The method is also close to IA³, but the learning rates differ by 100× without a reported tuning study. A better-tuned comparison would help clarify the contribution.

Questions for authors

None

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We thank Reviewers AXFA, x8gA, and YvHf for their constructive feedback. The reviews raised three main themes: (1) the functional separation between coherence channels and the complement, (2) comparison with well-tuned scaling baselines, and (3) robustness to contamination. We added experiments for each, as summarized below. Full results are in the individual responses and will be included in the revision. We thank Reviewer YvHf for raising their score after the clarifications.

1. Coherence vs. reasoning channels (AXFA, x8gA, YvHf).

  • Under energy-matched removal, removing the selected channels is still far more damaging, so the damage depends on which channels are removed rather than on how much energy is removed.
  • Against dimension-matched random groups with null calibration, the selected channels still show higher correct/incorrect overlap, so the effect is not an artifact of the rank-to-dimension ratio.
  • In prompt-grouped linear probes at equal dimension, the selected channels carry clearly less correctness signal than the complement.

2. Link between the analysis and the method (AXFA, x8gA).

  • In the default all-channel run, the complement's energy-weighted displacement dominates throughout training, and the conditions of Proposition 1 hold.
  • The selected channels drift less in both absolute and relative terms, and the trained gate leaves general perplexity essentially unchanged.
  • Complement-only training recovers nearly all of SphereGate's gain. Anisotropic-only training recovers little.

3. Scaling baselines (AXFA, x8gA).

  • SphereGate outperforms both IA³ at its best-tuned learning rate and an RMSNorm-gain-only baseline on every backbone. This shows that placing the gain on the post-block residual matters.
  • Longer full-GRPO runs will be added in the camera-ready version.

4. Contamination (YvHf). Contamination affects every compared method equally, and our claims are comparative. Even so, we show five things:

  • SphereGate's gains hold on Llama-3-8B.
  • It does not increase verbatim memorization.
  • Its gains persist on perturbed GSM-Symbolic problems.
  • After decontaminating the training data, it improves on post-release AIME 2025.
  • Spurious rewards do not reproduce its gains.

Revisions. We will add all new analyses to the appendix and revise the abstract and conclusion so that the mechanistic claim rests explicitly on the controlled evidence above.

We would be grateful if the reviewers could look over the responses and let us know of any remaining concerns before the discussion period ends.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.