- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies whether asking a model to answer first and justify afterward improves its initial answer. It also introduces fine-tuning on post-answer explanations while masking the answer loss. Experiments cover 13 models and nine benchmarks, with generation optionally stopped before the explanation. The broad evaluation is useful, but the contribution needs clearer positioning against recent work, and several repeated results require clarification.
Critical strengths
-
The study addresses a practical question: improving answer accuracy without generating a reasoning chain before the answer. The coverage across model families and tasks is a strength.
-
The masking ablation and measurements of latency when stopping after the answer provide useful evidence for the proposed training and deployment choices.
Critical weaknesses
-
The novelty is overstated. Think or Not Think [1] already introduces Think-After-Answer, explicitly requesting an answer followed by reasoning, and reports improvements over reasoning-first training in some settings. Aux-Think [2] also studies reasoning before versus after the action and uses reasoning supervision with direct prediction at inference. These are multimodal settings, so they do not establish the same prompting gains reported here. Nevertheless, the central output ordering is already established, and neither paper is discussed.
-
The training comparison does not sufficiently establish the remaining contribution. TwT [3] already transfers reasoning benefits into direct-answer inference through staged distillation. Table 16 usefully tests answer masking, but comparisons with answer-only fine-tuning and staged reasoning distillation on the same data would clarify what this particular objective adds.
-
Some repeated results disagree without a documented change in setting. For Qwen3.5-9B on AMC 8, prompt-only Post-Reasoning scores 21.12 in Table 1 but 44.03 as the baseline in Table 9. For Qwen3.5-27B, the corresponding values are 30.67 and 20.67. Most other repeated entries match exactly. These discrepancies affect the claimed incremental training gains and need resolution.
[1] Li M, Zhong J, Zhao S, et al. Think or not think: A study of explicit thinking in rule-based visual reinforcement fine-tuning[J]. arXiv preprint arXiv:2503.16188, 2025.
[2] Wang S, Wang Y, Li W, et al. Aux-think: Exploring reasoning strategies for data-efficient vision-language navigation[J]. Advances in Neural Information Processing Systems, 2026, 38: 30892-30916.
[3] Xu J, Zhou M, Liu W, et al. TwT: Thinking without Tokens by Habitual Reasoning Distillation with Multi-Teachers' Guidance[J]. arXiv preprint arXiv:2503.24198, 2025.
Questions for authors
-
What distinguishes the proposed approach from Think-After-Answer beyond the model and task coverage?
-
Does explanation-only training outperform answer-only fine-tuning and staged reasoning distillation under a common data and evaluation setup?
-
Which AMC 8 baseline values are correct, and how do the corrections affect the reported training improvements?
2 comments
===== COMMENT 2/2 =====
(Continued from the previous comment.)
Self-distillation gives the highest accuracy on 4 of 4 benchmarks, with +8.77 points on GPQA over teacher distillation. Teacher rationales are the central input of TwT. In our setting, they give lower accuracy than the model's own justifications.
More answer supervision decreases accuracy: The unmasked objective in Table 16 uses the same questions, answers, LoRA configuration and seeds, and adds a direct loss on the answer tokens. This added supervision decreases accuracy in 9 of 9 settings, all significant at p < 0.05. If supervision on the answers explained the gain, we expect the opposite result.
New result: answer-only fine-tuning: As you requested, we trained the three models of Table 16 on the same 3,500 questions with the loss on the answer tokens only. The data, LoRA configuration and seeds are identical. The loss target is the only difference:
| Model | AMC 10 Answer-only | AMC 10 Masked | Δ | GPQA Answer-only | GPQA Masked | Δ | MMLU-Pro Answer-only | MMLU-Pro Masked | Δ |
|---|---|---|---|---|---|---|---|---|---|
| Qwen-3.5 (4B) | 9.25 | 10.86 | +1.61* | 39.84 | 42.87 | +3.03* | 44.50 | 45.70 | +1.20* |
| Ministral-3 (14B) | 13.17 | 16.01 | +2.84* | 36.14 | 39.68 | +3.54* | 51.88 | 54.91 | +3.03* |
| Llama-3.3 (70B) | 29.58 | 33.15 | +3.57* | 47.69 | 49.36 | +1.67* | 52.11 | 53.31 | +1.20* |
Mean accuracy (%) over three seeds. Δ is Masked minus Answer-only. * p < 0.05, one-sided Welch t-test, n = 3 seeds per arm.
Explanation-only training exceeds answer-only fine-tuning in 9 of 9 settings, all significant, by +1.20 to +3.57 points (mean +2.41). Mean accuracy orders as answer-only (36.02) < unmasked (36.53, Table 16) < masked (38.43). Accuracy increases as the loss moves from the answer to the justification. Distilling Step-by-Step [R5] puts a direct loss on the label. Our objective puts no loss on the answer, and these results test exactly this design choice.
The staged pipeline, component by component: The two parts of staged distillation are already tested under a common setup. Its final stage trains on the question and the answer only, with no reasoning. This is the answer-only arm above, which is lower in 9 of 9 settings. Its earlier stages depend on teacher rationales, which are lower on 4 of 4 benchmarks (Table 8). Thus, each part of staged distillation gives lower accuracy than our objective on the same data.
[Clarification:3] AMC 8 values in Table 9
Table 1 gives the correct prompt-based values: 21.12 for Qwen-3.5-9B and 30.67 for Qwen-3.5-27B. Both columns of these two cells in Table 9 have a transcription error. The other 88 repeated entries in Tables 9 and 10 match Tables 1–3 are accurate. The correct AMC 8 values are:
| Model | Prompt-based Post | Post-Reason SFT | Δ |
|---|---|---|---|
| Qwen-3.5-9B | 21.12 | 20.94 | −0.18 |
| Qwen-3.5-27B | 30.67 | 32.35 | +1.68 |
The correction does not change the count. Post-Reason SFT exceeds prompt-based post-reasoning in 66 of 90 settings in Tables 9–10, before and after the correction. Against Direct, Post-Reason SFT improves 82 of 90 settings (91.11%), which is the value that the paper reports. The revision corrects the four cells.
All of the authors sincerely appreciate your questions and we are eager to address any further questions. We look forward to a constructive discussion.
References:
[R1] Li, Ming, et al. "To think or not to think: A study of thinking in rule-based visual reinforcement fine-tuning." Advances in Neural Information Processing Systems 38 (2025): 167356-167407.
[R2] Wang, Shuo, et al. "Aux-Think: Exploring reasoning strategies for data-efficient vision-language navigation." Advances in Neural Information Processing Systems 38 (2025): 30892-30916.
[R3] Wei, Jason, et al. "Chain-of-thought prompting elicits reasoning in large language models." Advances in Neural Information Processing Systems 35 (2022): 24824-24837.
[R4] Xu, Jingxian, et al. "TwT: Thinking without tokens by habitual reasoning distillation with multi-teachers' guidance." Findings of the Association for Computational Linguistics: EMNLP 2025 (2025): 16475-16489.
[R5] Hsieh, Cheng-Yu, et al. "Distilling step-by-step! Outperforming larger language models with less training data and smaller model sizes." Findings of the Association for Computational Linguistics: ACL 2023 (2023): 8003-8017.
===== COMMENT 1/2 =====
We thank you for the close reading, down to the cross-check of repeated table entries, and for recognizing the practical question and the coverage across model families. Your three points need three kinds of evidence: that the finding is new relative to earlier answer-first work, that the training objective adds something specific, and that the reported numbers hold.
[Clarification:1] Distinction from Think-After-Answer and Aux-ThinkWe will cite and discuss both works in Related Work. They differ from our work in task, in method and, most importantly, in the direction of the result:
The last row is the key difference. The authors of [R1] state that Think-After-Answer still underperforms No-Thinking RFT. The authors of [R2] state that Pre-Think and Post-Think both fall short of No-Think. In both works, reasoning after the answer gives lower accuracy than no reasoning. Our paper shows the opposite for instruction-tuned text LLMs: a request for a post-answer justification gives higher accuracy than direct answering, in 99 of 117 settings, with no tokens before the answer. Thus, the shared output order does not make the finding known. The prior results predict the opposite of what we observe.
Two more differences follow from the table:
Earlier text-only evidence is also limited. In the original CoT paper, a "chain of thought after answer" prompt gave about the same accuracy as the standard prompt [R3]. Thus, before our work, the evidence did not show that an answer-first instruction increases the accuracy of the answer.
[Clarification:2] What the training objective adds, compared with answer-only training and staged distillation(Continued in the next comment.)