- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes GuardStr, an inference-time activation-steering method for chain-of-thought (CoT) reasoning. CoT generation is cast as a finite-horizon stochastic reachability problem over latent states (last-token hidden representations after each "Step t:" segment), where the target set contains representations of correctly terminated prefixes. Adapting a reachability result for stochastic barrier certificates (Zhi et al., 2024), the authors define a reasoning barrier function whose boundary and expected-increase conditions imply a lower bound on the probability of reaching a correct answer within the step budget. A neural barrier function is trained per model and task from outcome-labeled, unsteered trajectories with hinge penalties on these conditions, and is screened on validation trajectories with two empirical criteria. At inference, the softmax-normalized shortfall of each step's barrier increment sets the steering strength, and the normalized barrier gradient with respect to the hidden state sets the steering direction. Experiments with four 7–9B open-weight models on five BBH tasks and two MMLU subjects report higher mean accuracy than unsteered decoding, Static CAA, and K-CAST on all 28 model–task pairs (+5.91 points macro-average over unsteered decoding), complemented by an adaptive-strength ablation and strength/layer sweeps.
Critical strengths
S1. Consistent improvements from a lightweight, outcome-supervised controller. GuardStr improves mean final-answer accuracy over unsteered decoding on all 28 model–task pairs (Table 1; minimum +3.6, macro-average +5.91 points), and its adaptive strength outperforms the fixed-mean variant on all 28 pairs (Table 2), while requiring no fine-tuning of the base model, no step-level annotations, and no ground-truth answers at inference. If the attribution concerns below are resolved, this would be a practical and broadly applicable recipe for improving reasoning in open-weight models at inference time.
S2. A potentially principled source of step-level steering feedback. Casting CoT steering as finite-horizon reachability and deriving per-step credit from barrier-condition residuals offers a conceptual link between local interventions and terminal correctness that is distinct from contrastive or teacher-scored feedback. The experimental protocol is also documented with care (grouped Date splits, official MMLU splits, per-item seed schedule; App. B.1–B.2), which makes the framing worth developing further.
Critical weaknesses
W1. The probabilistic guarantee (Definition 1, Proposition 1, Appendix A) is vacuous or unattainable as stated. Since , Eq. (1) gives , so the bound in Eq. (4) is non-positive when . When , Eq. (3) fails at incorrectly terminated states, which lie in but are frozen after termination (p. 4); the conditions therefore cannot hold for any model that errs with positive probability. The proof also relies on Eq. (16), which fails once the stopped process enters , and on an implicit Markov assumption on . Moreover, the deployed task-level barrier is validated only on unsteered trajectories. Because this guarantee is the first stated contribution and underpins the "guardrail" framing, it should either be corrected, with explicit assumptions and reported bound values, or be withdrawn in favor of presenting the method as a heuristic.
W2. The gains in Tables 1–2 are not yet attributable to the barrier signal. Four issues prevent attribution. (i) No controls isolate the steering direction (e.g., a random equal-norm direction) or the strength schedule, which by Eqs. (9)–(10) reduces to whenever no shortfall occurs. (ii) Base runs use vLLM, whereas steered runs use Transformers (App. B.1, B.5). (iii) Each BBH test split has only 50 items, and no significance tests are reported. (iv) The defaults and appear to have been selected from test-set sweeps (Figs. 3–4). Same-backend reruns, these controls, paired significance tests, and validation-only tuning would settle the attribution.
Questions for authors
-
Satisfiability of Definition 1. Given that incorrect terminal states are frozen, can the authors exhibit a setting where Definition 1 holds with ? Please also clarify how Proposition 1 extends to the task-level target set and to the history-dependent controller.
-
Source of the gains. Under the same strength schedule, how does GuardStr compare with (a) a random equal-norm direction, (b) the schedule , (c) a barrier trained without , and (d) NBF-based best-of-N reranking at matched compute? Please report paired significance tests for each comparison.
-
Baselines and scope. Comparing with closer step-level methods (e.g., GeoSteer, ATLAS) or with self-consistency, and evaluating with native chat templates on larger models and longer-horizon tasks, would strengthen the generality claims.
13 comments
Response to Reviewer pMrv (Part 2)
Proof. All probabilities and expectations are conditioned on the fixed initial state . Let and define the first hitting time of the target set as
with if the target is never reached. Define .
Since consists of correct terminal states and the process remains fixed after termination, the target set is absorbing. Consequently,
and is nonincreasing in .
By the transition condition in Definition 1, the conditional expected barrier increment is at least whenever . Once the target set is reached, the state remains fixed and the barrier increment is zero. Therefore, for every ,
&\mathbb{E}_{s_0}[B_\phi(s_{t+1})] -\mathbb{E}_{s_0}[B_\phi(s_t)]\\\\ &\quad= \mathbb{E}_{s_0}\left[ \mathbb{E}[B_\phi(s_{t+1})\mid s_t] -B_\phi(s_t) \right]\\\\ &\quad\geq \epsilon\Pr_{s_0}(s_t\notin\mathcal{G}_x)\\\\ &\quad=\epsilon q_t. \end{array}$$ Summing over $t=0,\ldots,H-1$ yields $$\begin{array}{rl} \mathbb{E}_{s_0}[B_\phi(s_H)] &\geq B_\phi(s_0) +\epsilon\sum_{t=0}^{H-1}q_t\\\\ &\geq B_\phi(s_0)+\epsilon Hq_H, \end{array}$$ where the last inequality follows from $q_t\geq q_H$ and $\epsilon\geq 0$. On the other hand, the boundary conditions in Definition 1 imply $$\begin{array}{rl} \mathbb{E}_{s_0}[B_\phi(s_H)] &\leq\beta q_H+(1+\beta)(1-q_H)\\\\ &=1+\beta-q_H. \end{array}$$ Combining the preceding inequalities gives $$(1+\epsilon H)q_H \leq1+\beta-B_\phi(s_0).$$ To obtain the linear lower bound, let $$A=1+\beta-B_\phi(s_0), \qquad u=\epsilon H.$$ Since $s_0\notin\mathcal{G}_x$, the non-target boundary condition gives $B_\phi(s_0)\leq\beta$ and hence $A\geq 1$. Moreover, the additional condition $\epsilon T_{\mathrm{Max}}\leq 1$ ensures $0\leq u\leq 1$. It follows that $$\begin{array}{rl} \left(A-\frac{u}{2}\right)-\frac{A}{1+u} &=\frac{u(2A-1-u)}{2(1+u)}\\\\ &\geq\frac{u(1-u)}{2(1+u)}\\\\ &\geq 0. \end{array}$$ Therefore, $$q_H\leq\frac{A}{1+u}\leq A-\frac{u}{2}.$$ By the correspondence between target membership and successful termination, and since the process remains fixed after termination, $$p_{\mathrm{succ}} =\Pr_{s_0}(\kappa\leq H) =1-q_H.$$ Consequently, $$\begin{array}{rl} p_{\mathrm{succ}} &\geq 1-A+\frac{u}{2}\\\\ &=B_\phi(s_0)+\frac{\epsilon T_{\mathrm{Max}}}{2}-\beta. \end{array}$$ Combining this inequality with $p_{\mathrm{succ}}\geq 0$ completes the proof.Response to Reviewer pMrv (Part 3)
W2. The gains in Tables 1–2 are not yet attributable to the barrier signal.
Answer: Barrier-Signal Attribution, Backend Alignment, Statistical Variability, and Hyperparameter Selection
We thank the reviewer for highlighting these important concerns regarding the attribution of GuardSTR's performance improvements. We address the four issues individually below, including additional controlled experiments to distinguish the contributions of barrier-guided steering from alternative explanations.
(a) Steering direction and strength adaptation.
We agree that the steering rule is not a strict violation-triggered gate. The softmax normalization in Eq. (9) assigns relative steering credit based on the history of barrier shortfalls, rather than making a binary intervention decision. In particular, when all observed shortfalls are zero, , and Eq. (10) gives . This means that early reasoning steps may receive nonzero steering even without a barrier violation, while the steering strength progressively decays when no violations occur.
The purpose of this normalization is to allocate bounded, history-dependent steering strength according to the relative severity of barrier shortfalls. As reasoning progresses, accumulated history provides a more informative reference: larger shortfalls receive greater relative weight, whereas smaller or zero shortfalls receive reduced credit. We acknowledge that early normalization can introduce unnecessary interventions, but their magnitudes are controlled by . The effect of steering strength is further examined in Fig. 3 (p. 18, lines 918-930).
To directly assess whether the improvement comes from the barrier-derived direction rather than arbitrary activation perturbations, we conducted an additional random-direction control with matched update norm on Qwen3.5-9B across Date Understanding, Dyck Languages, and High School Mathematics. Both conditions use identical model weights, inference backend, prompts, and decoding settings, while maintaining their respective controller histories.
Table T01. Qwen3.5-9B random-direction control (success rate, %).
| Condition | Date | Dyck | Math. |
|---|---|---|---|
| GuardSTR | 94.0 | 38.0 | 91.5 |
| Random direction | 78.8 | 5.6 | 68.2 |
As shown in Table T01, GuardSTR outperforms random-direction steering by 15.2, 32.4, and 23.3 percentage points on the three tasks, respectively. These substantial differences provide direct evidence that the barrier-derived steering direction contributes beyond arbitrary activation perturbations of the same magnitude.
While Fig. 3 examines sensitivity to steering strength, it does not constitute a direct comparison against a standalone decaying-strength schedule. We therefore distinguish the demonstrated contribution of the barrier-derived direction from the separate question of whether alternative scheduling rules could yield similar benefits.
Response to Reviewer pMrv (Part 1)
We sincerely thank the reviewer for the careful evaluation and constructive feedback. We have carefully considered the comments and provide our detailed responses below.
W1. The probabilistic guarantee (Definition 1, Proposition 1, Appendix A) is vacuous or unattainable as stated.
Answer: Validity, Non-Vacuity, and Scope of the Corrected Probabilistic Guarantee.
We thank the reviewer for carefully examining our theoretical guarantee. The identified issues can be addressed through limited corrections to Proposition 1 and its proof, without modifying GuardSTR or its experimental results. We clarify the validity, non-vacuity, and applicability of the corrected guarantee below.
(1) Non-vacuity and satisfiability. Although the lower bound is trivial when ϵ=0, Definition 1 admits feasible barrier functions with ϵ>0 that yield nontrivial guarantees. For example, consider a deterministic transition s0→g, where g is an absorbing correct terminal state. With β=0.2, α=0.8, ϵ=0.2, TMax=4, Bϕ(s0)=0.2, and Bϕ(g)=1, all conditions in Definition 1 are satisfied, and the corrected proposition yields psucc≥0.4. This demonstrates that the theoretical conditions are satisfiable and the guarantee is not inherently vacuous. A frozen incorrect terminal state that violates positive drift before the horizon simply falls outside the proposition's sufficient conditions.
(2) Straightforward theoretical corrections. We have corrected the proof in Appendix A to accumulate positive drift only on trajectories that have not yet reached the target set, properly accounting for early successful termination. We also identified the previously omitted condition ϵTMax≤1, which is required to derive the stated linear lower bound. This is a simple, directly checkable finite-horizon condition, and we have verified that it already holds in all reported experimental configurations. Therefore, the original lower-bound form is preserved, and no changes to GuardSTR, its hyperparameters, or experimental results are required. Moreover, the revised proof uses the tower property of conditional expectation and does not require a Markov assumption.
(3) Theoretical guarantee and empirical applicability. Proposition 1 establishes a conditional mathematical guarantee when Definition 1 holds for the process being analyzed, including any steering interventions. As explained in Section 5.2, exhaustive verification of these conditions for deployed LLM reasoning is impractical. We therefore leverage the mathematical principles of neural barrier functions to develop an empirically learned, adaptive steering mechanism, rather than claiming a formally certified bound for the deployed controller. Its practical effectiveness is supported by consistent improvements across all 28 model–task pairs, with a macro-average gain of 5.91 percentage points. We will clarify this distinction in the revised manuscript.
In summary, the theoretical issues are resolved through localized, explicit corrections, while the original guarantee, proposed method, and experimental results remain unchanged in form and content. The revised proposition is mathematically valid under its sufficient conditions, and the additional numerical condition is already satisfied in our experiments.
We provide the complete revised Proposition 1 and its proof below.
Proposition 1 (Finite-Horizon Correct-Answer Reachability).
Fix TMax∈N>0 satisfying ϵTMax≤1. Suppose Bϕ satisfies Definition 1, and let s0=hℓ(x)∈/Gx. Under the correspondence between target membership and successful termination specified above, the success probability from s0 satisfies
p_{\mathrm{succ}} &=\Pr_{s_0}\left( \exists\\,t\in\\{1,\ldots,T_{\mathrm{Max}}\\}: s_t\in\mathcal{G}_x \right)\\\\ &\geq\max\\{ 0,\\, B_\phi(s_0)+\frac{\epsilon T_{\mathrm{Max}}}{2}-\beta \\}. \end{array}$$