- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes VCS, which steers reasoning models using their predicted risk of exhausting a token budget. Intervention strength depends on how much that risk has increased from its initial value, together with deadline pressure. A fixed completion direction is learned from finished and capped trajectories. Experiments show improved accuracy and lower token use across several models and reasoning tasks.
Critical strengths
I find the central idea sensible: a difficult question should not automatically trigger stronger intervention, and completion prospects provide a useful signal without requiring correctness labels. The improvements across models and the transfer from mathematics to code give this idea practical weight. The paper also provides substantial analysis and is reasonably candid about what its diagnostics do and do not establish.
Critical weaknesses
1. Predicting capping does not establish the “hard or stuck” distinction
The probe predicts whether a trajectory will finish, but this does not tell us why it may fail to finish. Risk can rise because useful reasoning needs more time, not necessarily because the model has gone off track. Figure 4 selects trajectories by their eventual outcomes, and Appendix A.4 finds that risk increments and current risk give similar within-question rankings. I therefore find the evidence for predicting budget exhaustion stronger than the evidence for separating difficulty from deterioration. The paper should keep that distinction clear in its central argument.
2. The methodological advance needs sharper positioning
Activation steering and adaptive control from latent verifiers are already explored by SEAL and ATLAS. What seems distinctive here is the use of completion risk relative to an initial reference. That is a potentially useful contribution, but it is narrower than a new adaptive reasoning framework. I would like the paper to explain more directly what this reference enables that existing feedback signals do not. The claim that correctness labels are unnecessary is supported; the case that this is a better basis for control is less settled.
3. The derivation assumes the key design choice
Section 4 chooses the intervention cost so that the initial viability is exactly the point of indifference. This recovers the proposed rule, but does not explain why the initial prediction is the right reference for deciding when intervention is worthwhile. A trajectory that starts with poor prospects and stays there receives no risk-driven correction, leaving the deadline term to act. Conversely, the normalization amplifies the same risk increase more when initial risk is high, as Appendix A.3 acknowledges. These are meaningful consequences of the design, and I would like a clearer argument for why they match the stated goal of protecting difficult but productive reasoning.
4. The experiments support VCS more clearly than its distinctive advantage
The gains over uncontrolled decoding are convincing, and the strength sweeps are useful. However, the risk-level comparison changes both the reference and the deadline term, so it does not cleanly establish what the initial reference adds. The main comparisons also omit a close adaptive steering method such as ATLAS. I do not think the paper needs a long list of additional baselines, but a focused comparison that separates the value of this feedback rule from adaptive steering more generally would make the methodological contribution much more convincing.
Questions for authors
-
Clarification: Why should initial viability be the point at which intervention becomes worthwhile? Section 4 builds this choice into the cost model, but I am less clear on why it is appropriate for trajectories that start with poor completion prospects.
-
Evidence: What is the strongest evidence that relative risk identifies deterioration beyond what current risk and deadline pressure already capture? Appendix A.4 finds similar within-question rankings, while the comparison changes both the risk reference and the deadline term.
4 comments
Author Response to Reviewer GfGW (Part 2/2)
W$_2$&W$_4$: Distinctiveness and empirical justification of trajectory-relative feedback
Measuring completion risk against each trajectory's own starting level lets the controller hold back on a question that is merely hard and act on a rollout that has lost ground since it began, which neither SEAL's fixed vector nor ATLAS's step-quality verifier does. Both comparisons the reviewer asks for support this.
1. No existing feedback signal separates a hard question from a rollout that is falling behind. SEAL's vector has no feedback, and ATLAS's verifier scores the current step without seeing the budget. Current risk sees the budget but also carries the question's difficulty: $\rho_0$ tracks the benchmarks' difficulty labels, so a gate on $\rho_t$ pushes hard questions from the first control point. Measured against $\rho_0$, the risk term is zero until risk rises above where the rollout started. We will clarify how VCS differs from SEAL and ATLAS in the Related Work section.
2. Changing only the reference, with the deadline term kept, lowers accuracy. We ran the ablation the reviewer asks for on Qwen3-4B. At the closest token cost, this arm reaches 87.0% on MATH-500 and 54.0% on OlympiadBench, against 89.8% and 59.2% for ours, and uses 360 and 761 more tokens. Please see Figure 6 for more details.
3. ATLAS matches our accuracy on R1-1.5B but leaves far more trajectories capped. At each reasoning step, ATLAS's verifier, trained on step-quality scores, chooses which of three steering vectors to apply. Its code is not released, so we reimplemented it on R1-1.5B, a model from its paper, in its decoding setting. Each cell is accuracy (%) / capped (%) / mean tokens.
| MATH-500 | OlympiadBench | LiveCodeBench | |
|---|---|---|---|
| SEAL (Table 1) | 77.8 / 14.0 / 2763 | 38.0 / 41.8 / 5047 | 22.3 / 63.2 / 6284 |
| ATLAS (new) | 78.4 / 13.6 / 2755 | 38.1 / 40.4 / 5007 | 22.5 / 61.1 / 6206 |
| Ours (Table 1) | 78.4 / 5.8 / 2735 | 39.3 / 9.3 / 4128 | 23.1 / 11.0 / 4530 |
ATLAS tracks SEAL because its verifier, although accurate, picks the execution vector at over 96% of step boundaries, and that vector coincides with SEAL's direction. VCS adapts steering strength to rising cap risk and the remaining budget, matching or exceeding ATLAS’s accuracy while lowering cap rates and mean token counts.
The related work now states how the trajectory-relative reference differs from SEAL and ATLAS, and the accuracy–token figure (Figure 6) includes the current-risk + deadline baseline.
We hope the revisions address the remaining concerns, and that the revised version is now in a position to merit your continued support. If there is any other confusion you would like us to clarify or other improvements you would like us to implement, please let us know – we would be glad to do so.
Thank you for the clear reply. Your reply has addressed my main concerns, so I’m raising my score from 3 to 4.
Under the same deadline schedule, the authors’ additional comparison directly addressed my main concern about the method. VCS improves accuracy while using fewer tokens. The comparison with ATLAS also shows the method’s effectiveness and efficiency. I also read the other reviews and the authors’ responses, and I appreciate the effort the authors put into addressing them. Some raised concerns similar to mine, and the additional evidence makes me more satisfied with the response.
But I am still not fully convinced that a rise in predicted risk can distinguish a model that is stuck from one that needs more time. The paper’s forced-answer experiments only show that some trajectories give wrong answers at a few specified points, which could still mean that the model needs to continue reasoning.
Given that the authors have narrowed the relevant claims, and the effectiveness of the paper, I think that asking for a full proof of this distinction would go beyond the scope of a single paper. This is a remaining thought I have about the paper, it should not prevent me from raising my score.
Thank you for taking the time to revisit the paper and read our responses to the other reviewers. We really appreciate your careful reading, and we’re glad the comparison under the same deadline schedule addressed your main concern about the method.
We agree that it remains an open question whether a rise in predicted cap risk can reliably distinguish a model that is stuck from one that needs more time. As you point out, a model that answers incorrectly when forced to stop may still be making progress toward a correct solution. Your comment helped us clarify the scope of our claim. In future work, we would like to explore how to tell which trajectories would benefit from additional reasoning time when predicted cap risk rises. Thanks again!
Author Response to Reviewer GfGW (Part 1/2)
We thank the reviewer for recognizing the practical value of viability-based feedback and for identifying where the methodological claims need clarification. Next, we try to address each of the concerns.
We agree that the derivation does not independently establish initial viability as the optimal intervention threshold. The reference is a design choice: we seek to avoid increasing risk-driven intervention solely because a trajectory begins with poor completion prospects. Section 4 expresses this preference through the intervention-cost model. Using the initial state as the reference makes the risk-driven term zero at initialization and positive only when predicted viability falls below its initial level.
A poor start that stays on forecast does not by itself indicate that reasoning has deteriorated. Such a trajectory receives only the deadline-driven component (about 0.1 early, approaching 1 at the budget). The normalization, (ρt−ρ0)/(1−ρ0)=1−Vt/V0, measures the fractional loss of initial predicted viability. Thus, high initial risk alone does not activate the relative-risk term, although a given absolute risk increase produces a larger response when initial viability is lower.
The controlled ablation supports this choice. On the 181 OlympiadBench questions with high initial risk and no risk increase by 2,048 tokens under CoT, VCS achieves 48.6% accuracy versus 43.1% for the current-risk controller at the same steering strength, with the deadline schedule retained in both.
The strongest evidence that relative risk captures what current risk and deadline pressure do not is a controlled comparison that changes only the risk reference: under the same deadline schedule, a gate driven by the rise is more accurate at matched cost than a gate driven by current risk (Please see Figure 6 and related discussion in Section 5.2).
Current risk cannot tell a hard question from a rollout that has fallen behind. A high $\rho_t$ can mean either, and deadline pressure depends only on position; the rise above $\rho_0$ keeps only what has changed since the rollout began. A current-risk gate therefore pushes hard questions from the first control point. On the 181 OlympiadBench questions that start above the median risk and show no rise by 2,048 tokens, a current-risk gate of the same strength ends them about 2,300 tokens sooner at 43.1% accuracy, against 48.6% for ours, and weaker settings leave 54–66% of them capped, against 16%.
The within-question tie and the outcome-selected groups do not change this. Within a question $\rho_0$ is nearly constant, so the rise and current risk differ only by a shift and must rank alike. Those groups are selected by outcome, so we add a grouping that does not: among Qwen3-4B continuations in the top third by $\rho_0$ still running at 4096 tokens, 33% of those whose risk has fallen go on to cap, against 88% of those whose risk has risen.
Our claim is that initial risk provides a reference associated with the question's baseline completion difficulty, while subsequent changes capture how the rollout's completion prospects evolve.
We agree that the probe forecasts budget exhaustion and does not diagnose why risk rises, and we do not claim that the rise predicts capping better than $\rho_t$. The introduction now defines "hard" as the risk a question starts with and "stuck" as risk that has risen above it. The diagnosis section now explains the within-question tie and points to a new appendix (Appendix A.4, Figure 9) grouping that does not select on outcome.
(Please see part 2 of our response.)