- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes CoTDog, a training-free inference-time controller for reasoning LLMs. It monitors the generated chain-of-thought and inserts short control text (stage headings, checkpoints, repetition reminders, and a forced hand-off to the final answer) at token positions
Critical strengths
-
Consistent empirical gains under a shared, carefully accounted protocol. CoTDog has the best score in all nine settings and the fewest tokens in six, against baselines that already control reasoning length. Token accounting is explicit (planner/estimator tokens counted for baselines, control tokens for CoTDog).
-
Practical and cheap. No parameter updates or auxiliary reasoning model, per-token throughput overhead <= 0.53%, and code is released. If the gains hold under less restrictive settings (see weaknesses), the method is directly deployable.
Critical weaknesses
-
The 8,192-token completion limit is small for these models and every unfinished output counts as wrong, so the Vanilla baselines are crippled by truncation rather than reasoning quality: 44.7% of all Vanilla outputs are unfinished (e.g., Qwen3.5-27B Vanilla: 23.5% on LCB with 806/1,055 unfinished; 57.4% on MATH-500 with 213/500 unfinished). Much of "7.9% higher score with 23.7% fewer tokens" is the effect of forcing a hand-off before the limit.
-
The T=0.6 / 32k study covers one model, one dataset and only the two prompt-only baselines; TALE-EP, P&B and Think Token Adjustment are never run with a larger limit or sampling.
-
The trace-based ingredients mostly matter on LCB, where Open-Loop is anomalously bad (50.5/49.7/47.2, below Prompt & Budget) without explanation. Meanwhile the policies are hand-crafted per task and per model (checkpoint fractions 0.12-0.90b, a 3b/6b rule only for DS-Qwen-32B, task-specific wording), there is no development split, and preset budgets are computed from 5% of the evaluation inputs.
4,Without a held-out protocol it is unclear how much of the gain comes from iterating on the benchmarks, which also weakens the generality claim (all three models are Qwen-derived).
Questions for authors
Clarifications that could change my assessment:
- Please report Table 1 for all baselines (incl. TALE-EP, P&B, Think Token Adjustment) at a 16,384-token limit and with the recommended sampling settings (T=0.6), with >=3 seeds. How much of the gap survives?
- Why does Open-Loop Control collapse on LiveCodeBench (below Prompt & Budget for all three models) while being competitive elsewhere? Which fixed-position action harms code generation?
- Is Prompt & Budget equivalent to plain s1-style budget forcing (append the think-end token at a fixed budget, no structured prompt)? If not, how does that minimal baseline perform?
7 comments
Response to Weaknesses (2/4)
W2. Strong baselines with a larger limit and sampling
The T=0.6 / 32k study covers one model, one dataset and only the two prompt-only baselines; TALE-EP, P&B and Think Token Adjustment are never run with a larger limit or sampling.
Thank you for the suggestion. As suggested, we reran all methods in Table 1 with a 16,384-token limit, T=0.6, and three seeds (42/43/44); CoTDog now uses a five-stage policy shared across datasets together with adaptive budget initialization (see W3). The table below shows the results on Qwen3.5-27B (mean ± std; SP: Structured Prompt; TTA: Think Token Adjustment). CoTDog's AvgTok includes the cost of per-problem difficulty estimation and budget output, about 182 tokens per problem on average.
\hline \text{Dataset} & \text{Method} & \text{Score}\\,(\\%) & \text{AvgTok} & E^3 \\\\ \hline \text{GPQA} & \text{Vanilla} & 78.5\pm0.4 & 9034\pm71 & 0.683\pm0.001 \\\\ & \text{SP} & 81.8\pm0.0 & 7414\pm5 & 0.903\pm0.001 \\\\ & \text{P\\&B} & 80.3\pm0.7 & 7476\pm110 & 0.863\pm0.003 \\\\ & \text{TALE-EP} & 82.6\pm1.1 & 4518\pm138 & 1.511\pm0.085 \\\\ & \text{TTA} & 81.3\pm0.7 & 4802\pm82 & 1.377\pm0.001 \\\\ & \mathbf{CoTDog} & \boldsymbol{85.1\pm0.4} & \boldsymbol{4168\pm64} & \boldsymbol{1.738\pm0.012} \\\\ \hline \text{MATH-500} & \text{Vanilla} & 73.6\pm0.3 & 8911\pm138 & 0.608\pm0.014 \\\\ & \text{SP} & 58.8\pm0.3 & 10392\pm95 & 0.333\pm0.000 \\\\ & \text{P\\&B} & 96.1\pm0.7 & 5779\pm88 & 1.598\pm0.048 \\\\ & \text{TALE-EP} & 96.4\pm0.3 & 4333\pm51 & 2.145\pm0.038 \\\\ & \text{TTA} & 92.8\pm2.5 & 2981\pm143 & 2.898\pm0.298 \\\\ & \mathbf{CoTDog} & \boldsymbol{98.2\pm0.0} & \boldsymbol{2377\pm27} & \boldsymbol{4.057\pm0.047} \\\\ \hline \text{LCB-v6} & \text{Vanilla} & 34.6\pm0.2 & 14175\pm53 & 0.085\pm0.001 \\\\ & \text{SP} & 61.6\pm0.3 & 11705\pm21 & 0.324\pm0.004 \\\\ & \text{P\\&B} & \boldsymbol{78.9\pm1.0} & 7775\pm23 & 0.801\pm0.023 \\\\ & \text{TALE-EP} & 66.3\pm0.1 & 9569\pm91 & 0.459\pm0.003 \\\\ & \text{TTA} & 73.0\pm1.1 & 5963\pm409 & 0.896\pm0.088 \\\\ & \mathbf{CoTDog} & 76.6\pm0.7 & \boldsymbol{4651\pm48} & \boldsymbol{1.263\pm0.011} \\\\ \hline \end{array}$$ On Qwen3.5-27B, CoTDog achieves the lowest token usage and the highest E³ on all three datasets; its accuracy is the highest on GPQA and MATH-500 and slightly below P&B on LCB (76.6 vs. 78.9) with 40% fewer tokens. Across three models and nine settings, CoTDog has the highest E³ in 7 of 9 settings, and its average accuracy, token usage, and E³ are all better than those of every baseline; the exact differences are given in Q1. To cover all three models, the table below compares CoTDog with the strongest baseline (by E³) in each of the nine settings (mean over three seeds; bold marks the better value in each pair): $$\begin{array}{lll|c|c|c} \hline \text{Dataset} & \text{Model} & \text{Strongest baseline} & \text{Baseline: Score / AvgTok / }E^3 & \mathbf{CoTDog}\text{: Score / AvgTok / }E^3 & E^3\\;\text{ratio} \\\\ \hline \text{GPQA} & \text{QwQ} & \text{P\\&B} & 60.9\\,/\\,5176\\,/\\,0.716 & \mathbf{65.7}\\,/\\,\mathbf{3563}\\,/\\,\mathbf{1.210} & \mathbf{1.69}\times \\\\ & \text{Qwen3.5} & \text{TALE-EP} & 82.6\\,/\\,4518\\,/\\,1.511 & \mathbf{85.1}\\,/\\,\mathbf{4168}\\,/\\,\mathbf{1.738} & \mathbf{1.15}\times \\\\ & \text{DS-Qwen} & \text{P\\&B} & 60.6\\,/\\,3205\\,/\\,1.151 & \mathbf{63.6}\\,/\\,\mathbf{2969}\\,/\\,\mathbf{1.365} & \mathbf{1.19}\times \\\\ \hline \text{MATH-500} & \text{QwQ} & \text{TALE-EP} & \mathbf{95.0}\\,/\\,2583\\,/\\,3.495 & 94.4\\,/\\,\mathbf{2463}\\,/\\,\mathbf{3.618} & \mathbf{1.04}\times \\\\ & \text{Qwen3.5} & \text{TTA} & 92.8\\,/\\,2981\\,/\\,2.898 & \mathbf{98.2}\\,/\\,\mathbf{2377}\\,/\\,\mathbf{4.057} & \mathbf{1.40}\times \\\\ & \text{DS-Qwen} & \text{TALE-EP} & 91.8\\,/\\,\mathbf{1506}\\,/\\,\mathbf{5.603} & \mathbf{94.0}\\,/\\,1958\\,/\\,4.514 & 0.81\times \\\\ \hline \text{LCB-v6} & \text{QwQ} & \text{P\\&B} & \mathbf{72.1}\\,/\\,8036\\,/\\,0.647 & 71.3\\,/\\,\mathbf{6455}\\,/\\,\mathbf{0.788} & \mathbf{1.22}\times \\\\ & \text{Qwen3.5} & \text{TTA} & 73.0\\,/\\,5963\\,/\\,0.896 & \mathbf{76.6}\\,/\\,\mathbf{4651}\\,/\\,\mathbf{1.263} & \mathbf{1.41}\times \\\\ & \text{DS-Qwen} & \text{TTA} & 51.5\\,/\\,\mathbf{2747}\\,/\\,\mathbf{1.239} & \mathbf{61.8}\\,/\\,4352\\,/\\,0.877 & 0.71\times \\\\ \hline \end{array}$$ CoTDog exceeds the strongest baseline's E³ in 7 of 9 settings, by up to 1.69×. In the two exceptions (MATH/DS-Qwen and LCB/DS-Qwen), the baselines with higher E³ are 2.2 and 10.3 points less accurate than CoTDog. Full results for all three models will be included in the revision.Response to Weaknesses (3/4)
W3. Open-Loop on LCB and hand-crafted policies
The trace-based ingredients mostly matter on LCB, where Open-Loop is anomalously bad (50.5/49.7/47.2, below Prompt & Budget) without explanation. Meanwhile the policies are hand-crafted per task and per model (checkpoint fractions 0.12-0.90b, a 3b/6b rule only for DS-Qwen-32B, task-specific wording) …
Thank you for prompting us to check the implementation. We found that the original Open-Loop lacked answer protection in the answering phase after reasoning ended, so actions from the fixed schedule could interrupt code generation and corrupt the answer. After fixing this (New Open-Loop below), we re-evaluated it in the unified runtime (original configuration, full LCB, Pass@1 %):
\hline \text{Model} & \text{Old Open-Loop} & \text{New Open-Loop} & \mathbf{CoTDog} \\\\ \hline \text{QwQ-32B} & 50.5 & 61.5 & \mathbf{62.7} \\\\ \text{Qwen3.5-27B} & 49.7 & 74.2 & \mathbf{75.5} \\\\ \text{DS-Qwen-32B} & 47.2 & 58.3 & \mathbf{58.7} \\\\ \hline \end{array}$$ After the fix, Open-Loop no longer degrades sharply. New Open-Loop differs from CoTDog in whether the actions during reasoning adapt to runtime events. CoTDog remains ahead of New Open-Loop on all three models (by 1.2, 1.3, and 0.4 points), indicating that adjusting actions to runtime events during reasoning provides further gains on top of the fixed schedule. Regarding hand-crafted policies: in the new 16k main comparison, the per-task and per-model settings noted by the reviewer (checkpoint fractions, the DS-Qwen-specific 3b/6b rule, and task-specific wording) are replaced by a single policy shared by all models and datasets, with the same five-stage decomposition (understand → plan → solve → check → answer) and scheduling logic. The budget is set from the model's own difficulty estimate, and the thinking length of each stage is allocated dynamically according to the remaining token budget, without task-specific budgets or thresholds. Under this unified policy, CoTDog's average accuracy, token usage, and E³ remain better than those of every baseline (see Q1), and the policy transfers directly to a non-Qwen model (see W4).
Response to Weaknesses (1/4)
W1. Truncation under the 8k limit
Thank you for this comment. We address it under both the 16k and the 8k budget.
Incompletion and gains under the 16k budget. The table below reports the incompletion rate (%) of each method at 16k, T=0.6 (SP: Structured Prompt; TTA: Think Token Adjustment). Each model has 1,753 problems × 3 seeds = 5,259 samples, for 15,777 samples in total.
\hline \text{Method} & \text{QwQ-32B} & \text{Qwen3.5-27B} & \text{DS-Qwen-32B} & \text{Overall} \\\\ \hline \text{Vanilla} & 13.41 & 47.01 & 5.48 & 22.02\\;(3474/15777) \\\\ \text{SP} & 13.18 & 33.77 & 4.51 & 17.15\\;(2706/15777) \\\\ \text{P\\&B} & 11.75 & 7.30 & 2.34 & 7.07\\;(1115/15777) \\\\ \text{TALE-EP} & 11.52 & 17.86 & 4.34 & 11.53\\;(1819/15777) \\\\ \text{TTA} & 13.86 & 9.30 & 1.25 & 8.23\\;(1298/15777) \\\\ \hline \mathbf{CoTDog} & \mathbf{4.45} & \mathbf{3.54} & \mathbf{0.29} & \mathbf{2.76}\\;(\mathbf{435}/\mathbf{15777}) \\\\ \hline \end{array}$$ CoTDog has the lowest incompletion rate on all three models. The impact of truncation on Vanilla varies by model, and so does CoTDog's gain: - Qwen3.5 has the longest native reasoning, and 47.0% of its Vanilla samples are incomplete. CoTDog lowers this to 3.5% and improves accuracy by **24.4 points** on average. - On QwQ and DS-Qwen, Vanilla's incompletion rates are lower (13.4% and 5.5%). Across these 6 settings, CoTDog matches Vanilla's average accuracy (+0.1 points) while reducing token usage by **40.1%** on average. The per-model relation between truncation and gains is summarized below (ΔScore and ΔAvgTok are CoTDog relative to Vanilla, averaged over the three datasets): $$\begin{array}{lcccc} \hline \text{Model} & \text{Incomplete: Vanilla} & \text{Incomplete: CoTDog} & \Delta\text{Score}\\,(\text{pp}) & \Delta\text{AvgTok} \\\\ \hline \text{QwQ-32B} & 13.41\\% & \mathbf{4.45\\%} & +1.6 & -38.3\\% \\\\ \text{Qwen3.5-27B} & 47.01\\% & \mathbf{3.54\\%} & +24.4 & -64.8\\% \\\\ \text{DS-Qwen-32B} & 5.48\\% & \mathbf{0.29\\%} & -1.4 & -41.9\\% \\\\ \hline \end{array}$$ This shows that CoTDog's efficiency gains also hold in settings with less truncation, whereas its accuracy gains come mainly from cases where the original reasoning tends to exceed the budget. Measured with the same metric as the abstract (relative to the highest-scoring baseline in each setting), CoTDog's token reduction grows from 23.7% at 8k to 33.7% at 16k, while its accuracy becomes on par with that baseline (see Q1). **Under the 8k budget, the gains do not come from forced termination.** At 8k, an important role of CoTDog is to help the model finish reasoning and answer within the budget: in our rerun with the unified runtime, on Qwen3.5/LCB, Vanilla has an incompletion rate of **75.7%** (Pass@1 **24.2%**), versus **13.6%** (Pass@1 **75.5%**) for CoTDog. To test whether this gain merely reflects forced termination, we added two minimal Budget Forcing baselines (BF plain / BF structured; defined in Q3). In the same setting, BF plain reaches **53.7% / 6,027 tokens** and BF structured **48.5% / 6,506 tokens**, both clearly below CoTDog's **75.5% / 4,754 tokens**. Across all 18 comparisons over the nine settings, CoTDog is both more accurate and cheaper in **14**.