- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
When2Think is an RL post-training framework that learns whether to use an explicit reasoning block and how much computation to spend within it. Its central mechanism, Instance-level Difficulty-Aware Control (IDAC), uses cached reference-model success rates and response lengths to shape a correctness-gated efficiency bonus. It combines this with batch-wise advantage standardization and importance-sampled exploration of THINK/NOTHINK modes.
The main contribution is difficulty-conditioned reward shaping for computation allocation, supported by analyses separating mode selection from within-THINK length control. Importance sampling and advantage standardization build on existing methods. NOTHINK skips the dedicated reasoning block but can still contain substantial reasoning, and its reward does not directly penalize response length.
Validation covers mathematical reasoning benchmarks at 1.5B and 7B scales, difficulty-stratified evaluation on MATH-500, and cross-domain transfer on MMLU-Pro Stratified. The experiments compare accuracy and token usage, examine THINK/NOTHINK behavior, and ablate the main training components and reward coefficients.
Critical strengths
- IDAC provides a simple, explicit mechanism for conditioning efficiency incentives on reference-estimated difficulty and cost. This addresses a consequential limitation of uniform compression: reducing unnecessary reasoning without equally discouraging useful computation on harder problems. It is the paper’s strongest methodological contribution.
Critical weaknesses
- Limited evidence of improvement over the strongest relevant baseline. The headline gains are measured against R1-Distill, whereas DeepScaleR uses the same backbone and training dataset. On AIME24, DeepScaleR achieves higher Pass@3 with fewer tokens (58.0%, 8,473 versus 56.0%, 10,236); on AIME25, When2Think gains only 0.7 percentage points while using more tokens. These results leave the incremental benefit over conventional RL post-training unclear, weakening the central accuracy–efficiency claim.
- The efficiency objective does not consistently track actual computation. Correct NOTHINK responses receive the full efficiency bonus regardless of length, although they can contain lengthy step-by-step reasoning. Moreover, mode-specific comparisons evaluate different model-selected subsets rather than forcing both modes on the same questions. Consequently, the paper neither guarantees nor cleanly isolates the computational benefit of NOTHINK, limiting support for its joint “whether and how much to reason” formulation.
Questions for authors
Please refer to the Weaknesses.
6 comments
W2: NOTHINK Definition and Adaptive Validity
We agree on both points: (1) NOTHINK does not mean zero reasoning, and (2) its bonus is length-invariant within its cap.
NOTHINK denotes direct answer-channel generation that bypasses the dedicated ⟨BOT⟩...⟨EOT⟩ deliberation block under a bounded budget (4,096 tokens vs. 16,384 for THINK; App. A.1, Table 2). This is substantive, not terminological: NoThinking (Ma et al., arXiv:2504.09858) shows bypassing the explicit thinking block can outperform explicit thinking at matched budgets because concise derivation can still occur in the answer channel.
For NOTHINK (), () is length-independent within the budget. The reward therefore does not algebraically guarantee short NOTHINK responses. Brevity is an empirical property of the learned policy: bypassing the think block eliminates recursive self-reflection loops (e.g., "Wait, let me recheck..."; App. E.1-E.3), while the 4K cap prevents unbounded deliberation from spilling into the answer channel.
| Illustrative NOTHINK Case () | ||
|---|---|---|
| Easy, correct, 500 tokens (within 4K) | ||
| Easy, correct, 3,500 tokens (within 4K) | ||
Easy, exceeds 4K (truncated, no valid \boxed{}) |
All token counts measure total generated tokens, including answer-channel tokens. As shown in App. C.1 (Figure 5, KDE), correct NOTHINK trajectories peak sharply at tokens, while THINK trajectories consume substantially more even when correct and exhibit a heavy tail when incorrect.
| Benchmark (App. C.2, Tables 4-8) | NOTHINK Pass@3 | Mean NOTHINK Tokens | Mean THINK Tokens | NOTHINK Rate |
|---|---|---|---|---|
| GSM-Plus | ||||
| Minerva | ||||
| OlympiadBench | ||||
| AIME24 | Not selected | N/A | ||
| AIME25 | Not selected | N/A |
Selected NOTHINK is to shorter than THINK. We will move the KDE analysis (Figure 5) and this summary into the main text, framing NOTHINK brevity as an empirical outcome of bypassing the think block, not as a per-token length penalty.
Thank you for the clarification. Could you please clarify what empirical advantage IDAC offers over DeepScaleR in terms of accuracy and token usage? I do not see multi-stage training itself as a limitation, so a comparison of the resulting performance would help me better understand IDAC’s contribution. Are there specific metrics or comparisons that demonstrate IDAC’s advantages?
We thank Reviewer PL3r for recognizing IDAC as our strongest methodological contribution and for asking us to distinguish checkpoint-level dominance from mechanism-level contribution.
W1: Baseline Competitiveness
We agree with the empirical comparison. On AIME24, DeepScaleR-Preview achieves higher Pass@3 with fewer tokens (58.0% / 8,473 vs. 56.0% / 10,236), and on AIME25, When2Think gains 0.7 pp (40.0% vs. 39.3%) while using more tokens (9,549 vs. 8,074) (Table 1, p. 7). We will state this trade-off directly in Section 6.1 and do not claim universal Pareto dominance.
Our question is distinct from finding a single dominant RL checkpoint:
Conventional accuracy-oriented RLVR does not specify how computation should scale across instances. As our reference-statistics analysis shows, substantial easy-long and hard-short populations exist, so length alone cannot proxy solvability.
DeepScaleR's own diagnostics show longer generations are not inherently productive: on AIME24, its incorrect responses are >3× longer than its correct responses (20,346 vs. 6,395 tokens; DeepScaleR, Section 3.2, Table 1). When2Think addresses the complementary instance-level problem within a single fixed 16K stage (314 steps; App. A.1, Table 2) by encoding solvability (αi) and expected cost (τi) into the reward.
While hybrid interfaces are now standard — e.g., Qwen3 integrates both modes with a user-specified budget (Qwen3 Tech Report, arXiv:2505.09388) — When2Think differs in that allocation is learned autonomously as an instance-conditioned policy, not manually tuned at inference.
This behavior appears across evaluations:
w/o ISin Table 1, achieves 57.3% (10,046 tokens) on AIME24 and 40.0% (9,846) on AIME25, closely matching DeepScaleR (58.0%, 39.3%) in a single 314-step stage (Table 1; Section 7.2).In the revision we will distinguish: (1) unconditional checkpoint dominance over multi-stage curriculum models, which we do not claim; and (2) mechanism-level contribution, which lies in formulating and isolating instance-adaptive computation control.