- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This work introduces LTRS, an auxiliary training loss designed for reasoning models. The design is motivated by so-called small-gap errors: cases where the correct candidate only attains a slightly lower score than the top incorrect option. The authors pick one hidden layer using a held-out linear readout; for Qwen2.5-7B this is layer 18 out of 28. They define a fixed direction, computed as the normalized mean of pooled answer-token states from the frozen model, taken from gold minus wrong examples. No extra modules are introduced during inference. Results are reported on five math benchmarks and three medical multiple-choice sets evaluated via content scoring. The paper also includes layer and direction control experiments, plus a sweep over hidden-weight hyperparameters.
Critical strengths
- The core objective is simple and carries low overhead. It adds nothing to the inference pipeline.
- The authors try to separate fitting data, layer-selection data and audit data, and they discuss known probe validity concerns.
- The appendices are quite frank about limitations and negative outcomes.
Critical weaknesses
- The headline comparison on math tasks is not valid, and the fair matched comparison shows much weaker gains than what Table 1 implies. All eight non-LTRS math entries come from Zhu et al. (2026): those results use 16 samples per query at T=1.0 with a maximum length of 4096, and the Math500 set may not align exactly with MATH-OAI. Meanwhile, LTRS is scored using a different single-run evaluator. This evaluator discrepancy is visible within the manuscript itself. That is, the imported ASFT entry reports 59.60 on MATH, while the archived matched reference in Table B1 gives 65.80.
- Checkpoint selection appears to rely on test benchmark performance. Table D1 lists checkpoints at 200 steps, yet the 30K math result reported uses 600 steps. The medical numbers shown in Table 1 are the maximum value across seven seed-step records in Table C2: 32.65 at 10K and 34.03 at 30K. If averaged over all listed runs, the medical gain over ASFT sits around +1.4 pp for both budgets, not the +4.6 to +5.0 pp suggested by the peak values. The paper does not describe any independent validation split used for checkpoint picking.
- Control experiments suggest most observed gains do not stem from the proposed geometric objective. In Table 2(b), a covariance-matched random direction recovers 3.15 out of the total 5.68 point gain over ASFT, and layer 10 alone recovers 3.13 points. In Table 2(a), a random-direction baseline hits 33.35, compared with LTRS at 34.47. One critical ablation is missing: the λ_h = 0 case. That is, continuing DFT + KL fine-tuning starting from the same ASFT checkpoint, trained on the same 3000 gold answers with identical compute budget.
- The layer-selection and diagnostic datasets seem to be sampled from the evaluation test set. Appendix G.1 notes that 828 out of the 1517 evaluation problems do not yield valid training pairs. The total 1517 matches the sum 30 + 40 + 500 + 272 + 675, which is exactly the full test suite. The remaining 689 eligible problems form the pool for probe-train (249), layer-selection (132) and audit (158) splits. If true, the so-called held-out layer selection is using test-set questions. The source of the 3000 training pairs is also never clarified.
- The small-gap motivation is disconnected from the actual training setup. Section 3.3 and Appendix A.4 state that training pairs were not mined using the content gap defined in Equation (3). Instead, pairs were ranked by a composite support-plus-hidden score, and no threshold was applied to filter for small-gap examples.
Questions for authors
- With a fixed evaluator and fixed checkpoint, please report LTRS versus ASFT with per-benchmark counts, micro accuracy and bootstrap confidence intervals. 2.How were checkpoints and seeds chosen for Table 1? Was an independent validation split used? What is the mean and standard deviation across seeds at a fixed training step?
- What results do we get for DFT + KL fine-tuning starting from the same checkpoint, on identical pairs and with the same budget? What about a DPO baseline trained on these same pairs? 4.Are the 689-problem diagnostic pool and its probe-train / selection / audit subsets sampled from the 1517 test problems? Where do the 3000 training pairs originate?
The authors have not responded to my comments. Therefore, I keep my original rating.