- Rating
- 2
- Confidence
- 1(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates anisotropic residual channels during LLM post-training and proposes SphereGate, a parameter-efficient method that learns channel-wise gains on a frozen backbone. The authors argue that highly anisotropic channels primarily support coherence, while the remaining channels are responsible for reasoning adaptation. Experiments across Qwen and Llama models report improvements on mathematical reasoning benchmarks.
Critical strengths
-
I think the paper explores an interesting direction: using insights from model interpretability to guide post-training optimization.
-
SphereGate is simple and parameter-efficient, and the experiments cover multiple model sizes and architectures.
Critical weaknesses
-
I am not fully convinced by the proposed separation between coherence and reasoning channels. The subspace-overlap analysis does not establish that the selected channels contain little correctness-related information, since different representations can occupy the same subspace while remaining distinguishable. Similarly, the perplexity ablations show that these channels are important for language modeling, but do not necessarily establish their specific role in coherence. I find the current evidence insufficient to support the paper's central mechanistic interpretation.
-
I have substantial concerns about the reproducibility and reliability of the reported results. The submission does not provide executable code or complete environment details for independent reproduction. More importantly, the numerical results in Table 1 appear inconsistent with the stated evaluation protocol. For example, the reported 70.0 ± 0.54 on MATH-500 cannot be obtained from three ordinary 500-question accuracy scores using standard mean and SD calculations, even accounting for rounding. Similar inconsistencies occur in other entries. These issues make it difficult to assess the reliability of the claimed improvements.
Questions for authors
-
What additional evidence supports the proposed functional separation between coherence and reasoning channels? Can the authors provide stronger controls beyond subspace overlap and perplexity ablations?
-
Can the authors provide the per-seed accuracy scores, evaluation denominators, and aggregation procedure used to produce Table 1, particularly the reported standard deviations? If it was me who misunderstood, I would be more than willing to increase my score.
General Response to All Reviewers
We thank Reviewers AXFA, x8gA, and YvHf for their constructive feedback. The reviews raised three main themes: (1) the functional separation between coherence channels and the complement, (2) comparison with well-tuned scaling baselines, and (3) robustness to contamination. We added experiments for each, as summarized below. Full results are in the individual responses and will be included in the revision. We thank Reviewer YvHf for raising their score after the clarifications.
1. Coherence vs. reasoning channels (AXFA, x8gA, YvHf).
2. Link between the analysis and the method (AXFA, x8gA).
3. Scaling baselines (AXFA, x8gA).
4. Contamination (YvHf). Contamination affects every compared method equally, and our claims are comparative. Even so, we show five things:
Revisions. We will add all new analyses to the appendix and revise the abstract and conclusion so that the mechanistic claim rests explicitly on the controlled evidence above.
We would be grateful if the reviewers could look over the responses and let us know of any remaining concerns before the discussion period ends.