- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Asymmetric Intermediate-Answer Distillation (AID) for transferring long-chain reasoning to small language models under limited inference budgets. AID retains supervision on complete teacher traces and adds an auxiliary copy of the final answer conditioned on a randomly sampled reasoning prefix. An asymmetric attention mask prevents the auxiliary answer from entering the original reasoning stream, while position reassignment preserves the positions of the original sequence. The method requires no additional teacher generations or model parameters and uses standard causal decoding at inference. Experiments on mathematical and coding tasks show substantial gains for small students trained on long traces. Analyses of generated reasoning and controlled teacher prefixes support improved answer recovery when reasoning is interrupted.
Critical strengths
-
A practical training objective with substantial gains in its target setting. AID explicitly trains the student to answer from unfinished reasoning while retaining full-trace supervision. In the 0.6B Long-CoT setting, it improves all five benchmarks in Table 2, including MATH-500 from 22.9% to 37.0% and MBPP+ from 25.6% to 35.6%. These are three-seed averages using the same training data and evaluation budgets as SFT. The approach uses existing teacher traces and leaves the inference architecture unchanged.
-
The analyses provide evidence beyond aggregate accuracy. Figure 3 shows that AID's gains do not come from more frequent reasoning completion. Appendix E further compares the same problem-seed pairs on which both methods exhaust the budget, with accuracy improving from 16.5% to 31.1%. The controlled-prefix evaluation holds the supplied reasoning content fixed and also finds improved answerability, with math RS-AUC increasing from 0.191 to 0.419. These complementary comparisons support the value of explicitly learning to answer after interruption.
Critical weaknesses
-
The contribution of interior-state sampling is not fully separated from simpler answer supervision. In Table 3, endpoint-only sampling already achieves 36.6% final accuracy versus 37.0% for full AID, and RS-AUC of 0.383 versus 0.419. Thus, most of the MATH-500 improvement in this ablation is already present without sampling interior states. Table 10 also shows large differences in whether answers terminate within the 64-token answer budget. This is a useful learned behavior, but the current comparisons do not separate it from the benefit of supervising diverse intermediate reasoning states. Separate zero-prefix and full-prefix controls, or an ordinary causal objective combining full-trace SFT with prefix-to-answer examples at matched loss weights, would clarify what the sampling and masking design contributes. This comparison is also practically relevant given the approximately 2.06 to 2.23 times SFT step time reported for the current Long-CoT SDPA implementation in Appendix B.
-
The practical budget dependence remains underexplored. The main Long-CoT downstream results use a single 16K reasoning budget. RS-AUC varies the supplied teacher prefix, which measures a different quantity from accuracy across budgets on the student's own generated reasoning. The paper establishes a clear benefit at the tested operating point, but the size of that benefit at shorter reasoning budgets or different answer budgets is unknown. Testing the same trained models across these budgets would better establish the range in which AID is useful.
Questions for authors
-
How do zero-prefix-only and full-prefix-only auxiliary supervision compare with the endpoint mixture and full AID? Can a causal full-trace-plus-prefix-answer objective with matched answer-loss weights reproduce the gains? These comparisons would clarify the incremental contribution of interior sampling and the asymmetric construction.
-
For the same Long-CoT-trained models, how does downstream accuracy change across shorter self-generated reasoning budgets? How sensitive is the AID-SFT difference to the 64-token math answer budget?
3 comments
Thank you for the additional experiments and clarification. The prefix-position ablations clarify the role of interior-state supervision, while the shared-problem analysis and evaluations across self-generated reasoning budgets address my concerns about answer quality and practical budget dependence. The matched causal baseline is particularly informative: its comparable performance supports intermediate-answer supervision as the main source of the gains and helps clarify the contribution of the asymmetric implementation. I will maintain my rating of 3 (Weak acceptance), with a stronger positive assessment following the rebuttal. Please include the numerical reasoning-completion comparison cited in your response to Reviewer Ntui, together with comparable wall-clock training measurements for AID and the causal baseline, in the revised manuscript. These results would make the trade-off between accuracy, reasoning completion, memory, and training time clearer.
Response to Reviewer cvzi
Thank you for the careful follow-up and the positive assessment. We will include the reasoning-completion comparison and comparable wall-clock training measurements for SFT, AID, and the causal baseline in the revised manuscript, so that the trade-off between accuracy, reasoning completion, memory, and training time is explicit.
Response to Reviewer cvzi
We thank the reviewer for the careful reading and the precise questions.
W1 / Q1. Prefix positions, matched causal objective, and cost.
(a) Prefix positions. Math Long, Qwen3-0.6B-Base, mean over 3 seeds.
\text{Auxiliary answer conditioned on} & \text{MATH-500} & \text{RS-AUC} \\\\ \hline \text{zero prefix only } (n=0) & .358 & .298 \\\\ \text{full prefix only } (n=L) & .331 & .302 \\\\ \text{cut biased early} & .371 & .421 \\\\ \text{cut biased late} & .376 & .407 \\\\ \text{uniform (AID)} & .369 & .421 \end{array}$$ As the reviewer noted, much of the final accuracy is obtained without interior states: zero-prefix supervision is close to AID on MATH-500 (p = .19). However, it cuts RS-AUC from .421 to .298 (p = .001), so interior states are what teach answering from partial reasoning. Full-prefix-only supervision is worse on both (MATH-500 .331, p = .014). Biasing the cut toward early or late positions changes little, so the uniform default is not fragile. (b) Matched causal objective. Full + Truncated SFT (answer-only) supervises only the answer of the truncated copy, so its supervised tokens and answer-loss weights match AID. At matched FLOPs it reaches MATH-500 .377 and RS-AUC .470, comparable to AID. The gain therefore comes from intermediate-answer supervision itself. As reported to Reviewer Ntui (W1), the causal variants differ from AID in that they lower the rate at which the student finishes its reasoning on its own, while AID stays at the SFT level. (c) Cost. In our implementation AID takes about 2x longer per step than SFT, because the custom mask cannot use fused attention kernels; we will state this as a limitation. Input tokens and peak memory are the same as SFT, while the causal baseline needs two sequences per example and about 1.7x the memory. **W1 (Table 10) / Q2. Answer termination vs. answer quality.** Increasing the answer budget $A$ barely helps SFT (0.6B, MATH-500, 3 seeds): $$\begin{array}{l|ccc} \text{Reasoning budget} & 1\text{K} & 4\text{K} & 16\text{K} \\\\ \hline \text{SFT}, A=64 & .099 & .208 & .247 \\\\ \text{SFT}, A=128 & .129 & .220 & .250 \\\\ \text{SFT}, A=256 & .135 & .225 & .250 \\\\ \hline \text{AID}, A=64 & .337 & .378 & .375 \\\\ \text{AID}, A=128 & .337 & .379 & .375 \\\\ \text{AID}, A=256 & .337 & .379 & .375 \end{array}$$ SFT mostly keeps reasoning after the forced `</think>` rather than running out of room. To separate termination from quality, we compare the two methods on the same problems: those where both produce an answer within the budget (3 seeds, 64-token answer budget). $$\begin{array}{l|ccc} \text{Model, budget} & \text{common problems} & \text{AID} & \text{SFT} \\\\ \hline \text{0.6B, 1K} & 82 & .514 & .343 \\\\ \text{0.6B, 2K} & 138 & .564 & .489 \\\\ \text{0.6B, 16K} & 270 & .455 & .432 \\\\ \text{1.7B, 2K} & 244 & .754 & .472 \\\\ \text{4B, 2K} & 250 & .936 & .865 \end{array}$$ On the same problems AID is more accurate, with the largest gaps under tight budgets and a small gap at 16K for 0.6B. The problems SFT answers are easier on average (MATH-500 level 3.01 at 2K vs. 3.44 overall), which explains why its accuracy over its own answered subset can look higher. The gain is thus not explained by termination alone. **W2 / Q2. Self-generated budgets.** We evaluated the students on their own reasoning truncated at budget $B$ (MATH-500, 3 seeds, $A=256$): $$\begin{array}{l|cccccc} B & 0 & 1\text{K} & 2\text{K} & 4\text{K} & 8\text{K} & 16\text{K} \\\\ \hline \text{0.6B SFT} & .012 & .135 & .176 & .225 & .263 & .250 \\\\ \text{0.6B AID} & .055 & .337 & .374 & .379 & .379 & .375 \\\\ \hline \text{1.7B SFT} & .009 & .191 & .275 & .434 & .536 & .566 \\\\ \text{1.7B AID} & .033 & .549 & .629 & .647 & .659 & .657 \\\\ \hline \text{4B SFT} & .007 & .257 & .489 & .688 & .809 & .857 \\\\ \text{4B AID} & .151 & .650 & .777 & .834 & .865 & .885 \end{array}$$ AID improves accuracy at every budget, and the gain grows as interruptions become more frequent. We now report this alongside RS-AUC, since the two measure different quantities. We will add these results to the final version.