acceptodds
Under review as a conference paper at ICLR 2027

HIDDEN-STATE GEOMETRY FOR REASONING SHARPENING VIA LAYER-WISE TUNING

Abstract

A wrong answer can be informative about what a reasoning model still needs to learn. This is especially true when the model assigns almost the same score to a correct answer and an incorrect competitor, yet prefers the latter. We study these small-gap errors as a problem of learning a specific distinction. Candidate diagnostics show close competitions in the inspected pools, with incorrect predictions nearer the support boundary than correct predictions. This suggests using the competing answer to guide correction, rather than supplying the correct response alone. We introduce Layer-Selected Geometry Repair: select a hidden layer on held-out candidate data, build a fixed correct-minus-wrong direction from training pairs, and train a pairwise hidden margin alongside an anchored answer-learning objective. The comparison is internal, but its purpose is practical: improve the returned answer without adding a hidden-space selector at inference. Mathematical and medical evaluations support the procedure, while separate diagnostics examine adaptation, layer choice, and direction. A close wrong answer is not only a failure to discard; it can identify the distinction that training should address.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, reasoning, model repair, hidden representations, layer selection, small-gap errors

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_z3GtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This work introduces LTRS, an auxiliary training loss designed for reasoning models. The design is motivated by so-called small-gap errors: cases where the correct candidate only attains a slightly lower score than the top incorrect option. The authors pick one hidden layer using a held-out linear readout; for Qwen2.5-7B this is layer 18 out of 28. They define a fixed direction, computed as the normalized mean of pooled answer-token states from the frozen model, taken from gold minus wrong examples. No extra modules are introduced during inference. Results are reported on five math benchmarks and three medical multiple-choice sets evaluated via content scoring. The paper also includes layer and direction control experiments, plus a sweep over hidden-weight hyperparameters.

Critical strengths
  1. The core objective is simple and carries low overhead. It adds nothing to the inference pipeline.
  2. The authors try to separate fitting data, layer-selection data and audit data, and they discuss known probe validity concerns.
  3. The appendices are quite frank about limitations and negative outcomes.
Critical weaknesses
  1. The headline comparison on math tasks is not valid, and the fair matched comparison shows much weaker gains than what Table 1 implies. All eight non-LTRS math entries come from Zhu et al. (2026): those results use 16 samples per query at T=1.0 with a maximum length of 4096, and the Math500 set may not align exactly with MATH-OAI. Meanwhile, LTRS is scored using a different single-run evaluator. This evaluator discrepancy is visible within the manuscript itself. That is, the imported ASFT entry reports 59.60 on MATH, while the archived matched reference in Table B1 gives 65.80.
  2. Checkpoint selection appears to rely on test benchmark performance. Table D1 lists checkpoints at 200 steps, yet the 30K math result reported uses 600 steps. The medical numbers shown in Table 1 are the maximum value across seven seed-step records in Table C2: 32.65 at 10K and 34.03 at 30K. If averaged over all listed runs, the medical gain over ASFT sits around +1.4 pp for both budgets, not the +4.6 to +5.0 pp suggested by the peak values. The paper does not describe any independent validation split used for checkpoint picking.
  3. Control experiments suggest most observed gains do not stem from the proposed geometric objective. In Table 2(b), a covariance-matched random direction recovers 3.15 out of the total 5.68 point gain over ASFT, and layer 10 alone recovers 3.13 points. In Table 2(a), a random-direction baseline hits 33.35, compared with LTRS at 34.47. One critical ablation is missing: the λ_h = 0 case. That is, continuing DFT + KL fine-tuning starting from the same ASFT checkpoint, trained on the same 3000 gold answers with identical compute budget.
  4. The layer-selection and diagnostic datasets seem to be sampled from the evaluation test set. Appendix G.1 notes that 828 out of the 1517 evaluation problems do not yield valid training pairs. The total 1517 matches the sum 30 + 40 + 500 + 272 + 675, which is exactly the full test suite. The remaining 689 eligible problems form the pool for probe-train (249), layer-selection (132) and audit (158) splits. If true, the so-called held-out layer selection is using test-set questions. The source of the 3000 training pairs is also never clarified.
  5. The small-gap motivation is disconnected from the actual training setup. Section 3.3 and Appendix A.4 state that training pairs were not mined using the content gap defined in Equation (3). Instead, pairs were ranked by a composite support-plus-hidden score, and no threshold was applied to filter for small-gap examples.
Questions for authors
  1. With a fixed evaluator and fixed checkpoint, please report LTRS versus ASFT with per-benchmark counts, micro accuracy and bootstrap confidence intervals. 2.How were checkpoints and seeds chosen for Table 1? Was an independent validation split used? What is the mean and standard deviation across seeds at a fixed training step?
  2. What results do we get for DFT + KL fine-tuning starting from the same checkpoint, on identical pairs and with the same budget? What about a DPO baseline trained on these same pairs? 4.Are the 689-problem diagnostic pool and its probe-train / selection / audit subsets sampled from the 1517 test problems? Where do the 3000 training pairs originate?
1 comment

The authors have not responded to my comments. Therefore, I keep my original rating.

Reviewer_oVbiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Layer-wise Tuning for Reasoning Sharpening. A held-out diagnostic selects a supervision layer, and training pairs define a fixed mean correct-minus-wrong hidden-state direction. Adaptation then encourages a positive pairwise margin along this direction while retaining a token-learning objective and a KL anchor. LoRA updates span the backbone; the selected layer determines the auxiliary supervision location rather than restricting trainable parameters. No hidden-space selector is added at inference. Mathematical and medical evaluations report improvements, but the evidence combines imported baselines, archived evaluations, updated aggregate controls, and selected checkpoints with different levels of experimental traceability.

Critical strengths

The method turns a correct–wrong comparison into an explicit auxiliary objective and does not require an additional verifier or hidden-state ranking module at deployment. Its mathematical definition makes the proposed intervention understandable and potentially straightforward to test independently. The archived mathematical comparison reports macro accuracy increasing from 29.97% to 35.65% in the 10K setting, and the medical content-scoring comparison reports 28.02% to 32.65%. These observations justify further investigation. The manuscript also distinguishes fitting, layer selection, and audit roles instead of treating training-set geometric separation as sufficient evidence of generalization.

Critical weaknesses

The formal motivation concerns small gaps in answer-content likelihood, whereas the archived diagnostics use support-count statistics and the historical pair construction is not verified to follow the stated content-score mining rule. Thus the experiments do not establish that the method preferentially repairs the formally defined small-gap errors.The controls also leave open whether a correctness-derived direction is necessary. In the medical 30K experiment, a matched-budget random-direction run reaches 36.20%, above LTRS’s 34.03%; the paper correctly notes that matching budget alone does not establish complete experimental equivalence. Separately, the listwise diagnostic shows that projection ordering can improve strongly without a corresponding improvement in task accuracy.

Questions for authors

What exact negative-mining procedure produced the training pairs? On the same held-out examples, does improvement depend on the pre-repair content-likelihood gap, and does LTRS outperform ordinary continued training on those gap-defined subsets?

Reviewer_T61EOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper finds and studies small-gap errors in LLM reasoning, where a correct answer is available but loses to a competing incorrect answer. Based on this observation, this paper proposes LTRS, which selects a supervision layer using validation diagnostics and combines a pairwise hidden margin along a fixed representation difference direction with DFT and KL regularization. The experiments show improvements over archived ASFT references on mathematical and medical benchmarks, with no additional hidden space selector required at inference.

Critical strengths
  • LTRS introduces an auxiliary training objective based on the representational differences between correct answers and competitive incorrect answers. It is simple to integrate with DFT, KL regularization, and LoRA without modifying the original inference pipeline.

  • The paper reports random-direction controls and training variability, while explicitly distinguishing hidden-space ranking from final answer accuracy and clarifying the scope of different experimental results, which helps assess the limitations of its empirical evidence.

Critical weaknesses
  • The use of a single global direction averaged across correct and incorrect answers may be overly simplistic. Different questions can involve distinct reasoning errors. Such averaging may obscure important error-specific distinctions or capture superficial patterns rather than meaningful reasoning differences.

  • The mathematical baselines use different evaluation protocols from LTRS, which affects the fairness of comparison. Moreover, the separation between layer-selection data and final test data is unclear, raising my concerns about the potential risk of test-set leakage.

  • The actual negative selection procedure used in practice (as described in Appendix A.4) did not follow the small-gap error definition, which is not well aligned with the motivation.

  • The results are sensitive to random seeds and checkpoint selection. Table C2 shows considerable variation, but the selection criteria are unclear. It seems like only the best results are reported in the main table.

  • Cross-model generalization ability remains unexplored. Most of the experiments are conducted solely on Qwen2.5. Providing more results using different base models would strengthen this paper.

Questions for authors

Please see the weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.