acceptodds
Under review as a conference paper at ICLR 2027

Neural Barrier Functions as Guardrails for Adaptive Chain-of-Thought Steering

Abstract

Neural barrier functions (NBFs) provide a principled way to certify probabilistic safety and reachability in stochastic systems. Leveraging the inherent stochasticity of Large Language Model (LLM) generation, we introduce GuardSTR, a framework that formulates Chain-of-Thought (CoT) reasoning as a finite-horizon stochastic reachability problem, where successful reasoning corresponds to reaching latent states of correct final answers. Based on this formulation, we learn a reasoning barrier function as a guardrail to steer CoT reasoning. Theoretically, it provides probabilistic guarantees under a set of principled conditions for the desired terminal outcomes. Practically, it assigns a quantitative feedback credit for adaptive activation steering at every reasoning step. GuardSTR requires neither parameter fine-tuning nor ground-truth answers during inference, ensuring compatibility with different open-source LLMs. Across four LLMs and seven tasks from BBH and MMLU, GuardSTR improves final-answer accuracy over unsteered baselines on all 28 model–task pairs, yielding a macro-average gain of 5.91 percentage points. It achieves maximum gains of 12.8 and 11.6 percentage points over two state-of-the-art methods, Static CAA and K-CAST, respectively.

Primary area:
interpretability and explainable AI
Keywords:
Chain-of-Thought, Large Reasoning Models, Activation Steering

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_pMrvOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes GuardStr, an inference-time activation-steering method for chain-of-thought (CoT) reasoning. CoT generation is cast as a finite-horizon stochastic reachability problem over latent states (last-token hidden representations after each "Step t:" segment), where the target set contains representations of correctly terminated prefixes. Adapting a reachability result for stochastic barrier certificates (Zhi et al., 2024), the authors define a reasoning barrier function whose boundary and expected-increase conditions imply a lower bound on the probability of reaching a correct answer within the step budget. A neural barrier function is trained per model and task from outcome-labeled, unsteered trajectories with hinge penalties on these conditions, and is screened on validation trajectories with two empirical criteria. At inference, the softmax-normalized shortfall of each step's barrier increment sets the steering strength, and the normalized barrier gradient with respect to the hidden state sets the steering direction. Experiments with four 7–9B open-weight models on five BBH tasks and two MMLU subjects report higher mean accuracy than unsteered decoding, Static CAA, and K-CAST on all 28 model–task pairs (+5.91 points macro-average over unsteered decoding), complemented by an adaptive-strength ablation and strength/layer sweeps.

Critical strengths

S1. Consistent improvements from a lightweight, outcome-supervised controller. GuardStr improves mean final-answer accuracy over unsteered decoding on all 28 model–task pairs (Table 1; minimum +3.6, macro-average +5.91 points), and its adaptive strength outperforms the fixed-mean variant on all 28 pairs (Table 2), while requiring no fine-tuning of the base model, no step-level annotations, and no ground-truth answers at inference. If the attribution concerns below are resolved, this would be a practical and broadly applicable recipe for improving reasoning in open-weight models at inference time.

S2. A potentially principled source of step-level steering feedback. Casting CoT steering as finite-horizon reachability and deriving per-step credit from barrier-condition residuals offers a conceptual link between local interventions and terminal correctness that is distinct from contrastive or teacher-scored feedback. The experimental protocol is also documented with care (grouped Date splits, official MMLU splits, per-item seed schedule; App. B.1–B.2), which makes the framing worth developing further.

Critical weaknesses

W1. The probabilistic guarantee (Definition 1, Proposition 1, Appendix A) is vacuous or unattainable as stated. Since s0∉Gxs_0 \notin G_x, Eq. (1) gives Bϕ(s0)≤βB_\phi(s_0) \le \beta, so the bound in Eq. (4) is non-positive when ϵ=0\epsilon = 0. When ϵ>0\epsilon > 0, Eq. (3) fails at incorrectly terminated states, which lie in S∖GxS \setminus G_x but are frozen after termination (p. 4); the conditions therefore cannot hold for any model that errs with positive probability. The proof also relies on Eq. (16), which fails once the stopped process enters GxG_x, and on an implicit Markov assumption on sts_t. Moreover, the deployed task-level barrier is validated only on unsteered trajectories. Because this guarantee is the first stated contribution and underpins the "guardrail" framing, it should either be corrected, with explicit assumptions and reported bound values, or be withdrawn in favor of presenting the method as a heuristic.

W2. The gains in Tables 1–2 are not yet attributable to the barrier signal. Four issues prevent attribution. (i) No controls isolate the steering direction (e.g., a random equal-norm direction) or the strength schedule, which by Eqs. (9)–(10) reduces to cα/tc_\alpha / t whenever no shortfall occurs. (ii) Base runs use vLLM, whereas steered runs use Transformers (App. B.1, B.5). (iii) Each BBH test split has only 50 items, and no significance tests are reported. (iv) The defaults ℓ=1\ell = 1 and cα=1.0c_\alpha = 1.0 appear to have been selected from test-set sweeps (Figs. 3–4). Same-backend reruns, these controls, paired significance tests, and validation-only tuning would settle the attribution.

Questions for authors
  1. Satisfiability of Definition 1. Given that incorrect terminal states are frozen, can the authors exhibit a setting where Definition 1 holds with ϵ>0\epsilon > 0? Please also clarify how Proposition 1 extends to the task-level target set and to the history-dependent controller.

  2. Source of the gains. Under the same strength schedule, how does GuardStr compare with (a) a random equal-norm direction, (b) the schedule αt=cα/t\alpha_t = c_\alpha / t, (c) a barrier trained without L3\mathcal{L}_3, and (d) NBF-based best-of-N reranking at matched compute? Please report paired significance tests for each comparison.

  3. Baselines and scope. Comparing with closer step-level methods (e.g., GeoSteer, ATLAS) or with self-consistency, and evaluating with native chat templates on larger models and longer-horizon tasks, would strengthen the generality claims.

13 comments
AuthorsOpenReview ↗

Response to Reviewer pMrv (Part 1)

We sincerely thank the reviewer for the careful evaluation and constructive feedback. We have carefully considered the comments and provide our detailed responses below.

W1. The probabilistic guarantee (Definition 1, Proposition 1, Appendix A) is vacuous or unattainable as stated.

Answer: Validity, Non-Vacuity, and Scope of the Corrected Probabilistic Guarantee.

We thank the reviewer for carefully examining our theoretical guarantee. The identified issues can be addressed through limited corrections to Proposition 1 and its proof, without modifying GuardSTR or its experimental results. We clarify the validity, non-vacuity, and applicability of the corrected guarantee below.

(1) Non-vacuity and satisfiability. Although the lower bound is trivial when ϵ=0\epsilon=0, Definition 1 admits feasible barrier functions with ϵ>0\epsilon>0 that yield nontrivial guarantees. For example, consider a deterministic transition s0→gs_0\rightarrow g, where gg is an absorbing correct terminal state. With β=0.2\beta=0.2, α=0.8\alpha=0.8, ϵ=0.2\epsilon=0.2, TMax=4T_{\mathrm{Max}}=4, Bϕ(s0)=0.2B_\phi(s_0)=0.2, and Bϕ(g)=1B_\phi(g)=1, all conditions in Definition 1 are satisfied, and the corrected proposition yields psucc≥0.4p_{\mathrm{succ}}\geq0.4. This demonstrates that the theoretical conditions are satisfiable and the guarantee is not inherently vacuous. A frozen incorrect terminal state that violates positive drift before the horizon simply falls outside the proposition's sufficient conditions.

(2) Straightforward theoretical corrections. We have corrected the proof in Appendix A to accumulate positive drift only on trajectories that have not yet reached the target set, properly accounting for early successful termination. We also identified the previously omitted condition ϵTMax≤1\epsilon T_{\mathrm{Max}}\leq1, which is required to derive the stated linear lower bound. This is a simple, directly checkable finite-horizon condition, and we have verified that it already holds in all reported experimental configurations. Therefore, the original lower-bound form is preserved, and no changes to GuardSTR, its hyperparameters, or experimental results are required. Moreover, the revised proof uses the tower property of conditional expectation and does not require a Markov assumption.

(3) Theoretical guarantee and empirical applicability. Proposition 1 establishes a conditional mathematical guarantee when Definition 1 holds for the process being analyzed, including any steering interventions. As explained in Section 5.2, exhaustive verification of these conditions for deployed LLM reasoning is impractical. We therefore leverage the mathematical principles of neural barrier functions to develop an empirically learned, adaptive steering mechanism, rather than claiming a formally certified bound for the deployed controller. Its practical effectiveness is supported by consistent improvements across all 28 model–task pairs, with a macro-average gain of 5.91 percentage points. We will clarify this distinction in the revised manuscript.

In summary, the theoretical issues are resolved through localized, explicit corrections, while the original guarantee, proposed method, and experimental results remain unchanged in form and content. The revised proposition is mathematically valid under its sufficient conditions, and the additional numerical condition is already satisfied in our experiments.

We provide the complete revised Proposition 1 and its proof below.

Proposition 1 (Finite-Horizon Correct-Answer Reachability).

Fix TMax∈N>0T_{\mathrm{Max}}\in\mathbb{N}_{>0} satisfying ϵTMax≤1\epsilon T_{\mathrm{Max}}\leq 1. Suppose BϕB_\phi satisfies Definition 1, and let s0=hℓ(x)∉Gxs_0=h^\ell(x)\notin\mathcal{G}_x. Under the correspondence between target membership and successful termination specified above, the success probability from s0s_0 satisfies

p_{\mathrm{succ}} &=\Pr_{s_0}\left( \exists\\,t\in\\{1,\ldots,T_{\mathrm{Max}}\\}: s_t\in\mathcal{G}_x \right)\\\\ &\geq\max\\{ 0,\\, B_\phi(s_0)+\frac{\epsilon T_{\mathrm{Max}}}{2}-\beta \\}. \end{array}$$
AuthorsOpenReview ↗

Response to Reviewer pMrv (Part 2)

Proof. All probabilities and expectations are conditioned on the fixed initial state s0s_0. Let H=TMaxH=T_{\mathrm{Max}} and define the first hitting time of the target set as

κ=inf⁡t≥0:st∈Gx,\kappa=\inf\\{t\geq 0:s_t\in\mathcal{G}_x\\},

with κ=∞\kappa=\infty if the target is never reached. Define qt=Pr⁡s0(κ>t)q_t=\Pr_{s_0}(\kappa>t).

Since Gx\mathcal{G}_x consists of correct terminal states and the process remains fixed after termination, the target set is absorbing. Consequently,

qt=Pr⁡s0(st∉Gx),q_t=\Pr_{s_0}(s_t\notin\mathcal{G}_x),

and qtq_t is nonincreasing in tt.

By the transition condition in Definition 1, the conditional expected barrier increment is at least ϵ\epsilon whenever st∉Gxs_t\notin\mathcal{G}_x. Once the target set is reached, the state remains fixed and the barrier increment is zero. Therefore, for every 0≤t<H0\leq t < H,

&\mathbb{E}_{s_0}[B_\phi(s_{t+1})] -\mathbb{E}_{s_0}[B_\phi(s_t)]\\\\ &\quad= \mathbb{E}_{s_0}\left[ \mathbb{E}[B_\phi(s_{t+1})\mid s_t] -B_\phi(s_t) \right]\\\\ &\quad\geq \epsilon\Pr_{s_0}(s_t\notin\mathcal{G}_x)\\\\ &\quad=\epsilon q_t. \end{array}$$ Summing over $t=0,\ldots,H-1$ yields $$\begin{array}{rl} \mathbb{E}_{s_0}[B_\phi(s_H)] &\geq B_\phi(s_0) +\epsilon\sum_{t=0}^{H-1}q_t\\\\ &\geq B_\phi(s_0)+\epsilon Hq_H, \end{array}$$ where the last inequality follows from $q_t\geq q_H$ and $\epsilon\geq 0$. On the other hand, the boundary conditions in Definition 1 imply $$\begin{array}{rl} \mathbb{E}_{s_0}[B_\phi(s_H)] &\leq\beta q_H+(1+\beta)(1-q_H)\\\\ &=1+\beta-q_H. \end{array}$$ Combining the preceding inequalities gives $$(1+\epsilon H)q_H \leq1+\beta-B_\phi(s_0).$$ To obtain the linear lower bound, let $$A=1+\beta-B_\phi(s_0), \qquad u=\epsilon H.$$ Since $s_0\notin\mathcal{G}_x$, the non-target boundary condition gives $B_\phi(s_0)\leq\beta$ and hence $A\geq 1$. Moreover, the additional condition $\epsilon T_{\mathrm{Max}}\leq 1$ ensures $0\leq u\leq 1$. It follows that $$\begin{array}{rl} \left(A-\frac{u}{2}\right)-\frac{A}{1+u} &=\frac{u(2A-1-u)}{2(1+u)}\\\\ &\geq\frac{u(1-u)}{2(1+u)}\\\\ &\geq 0. \end{array}$$ Therefore, $$q_H\leq\frac{A}{1+u}\leq A-\frac{u}{2}.$$ By the correspondence between target membership and successful termination, and since the process remains fixed after termination, $$p_{\mathrm{succ}} =\Pr_{s_0}(\kappa\leq H) =1-q_H.$$ Consequently, $$\begin{array}{rl} p_{\mathrm{succ}} &\geq 1-A+\frac{u}{2}\\\\ &=B_\phi(s_0)+\frac{\epsilon T_{\mathrm{Max}}}{2}-\beta. \end{array}$$ Combining this inequality with $p_{\mathrm{succ}}\geq 0$ completes the proof.
AuthorsOpenReview ↗

Response to Reviewer pMrv (Part 3)

W2. The gains in Tables 1–2 are not yet attributable to the barrier signal.

Answer: Barrier-Signal Attribution, Backend Alignment, Statistical Variability, and Hyperparameter Selection

We thank the reviewer for highlighting these important concerns regarding the attribution of GuardSTR's performance improvements. We address the four issues individually below, including additional controlled experiments to distinguish the contributions of barrier-guided steering from alternative explanations.

(a) Steering direction and strength adaptation.

We agree that the steering rule is not a strict violation-triggered gate. The softmax normalization in Eq. (9) assigns relative steering credit based on the history of barrier shortfalls, rather than making a binary intervention decision. In particular, when all observed shortfalls are zero, A^t−=1/t\widehat A_t^-=1/t, and Eq. (10) gives αtℓ=cα/t\alpha_t^\ell=c_\alpha/t. This means that early reasoning steps may receive nonzero steering even without a barrier violation, while the steering strength progressively decays when no violations occur.

The purpose of this normalization is to allocate bounded, history-dependent steering strength according to the relative severity of barrier shortfalls. As reasoning progresses, accumulated history provides a more informative reference: larger shortfalls receive greater relative weight, whereas smaller or zero shortfalls receive reduced credit. We acknowledge that early normalization can introduce unnecessary interventions, but their magnitudes are controlled by cαc_\alpha. The effect of steering strength is further examined in Fig. 3 (p. 18, lines 918-930).

To directly assess whether the improvement comes from the barrier-derived direction rather than arbitrary activation perturbations, we conducted an additional random-direction control with matched update norm on Qwen3.5-9B across Date Understanding, Dyck Languages, and High School Mathematics. Both conditions use identical model weights, inference backend, prompts, and decoding settings, while maintaining their respective controller histories.

Table T01. Qwen3.5-9B random-direction control (success rate, %).

ConditionDateDyckMath.
GuardSTR94.038.091.5
Random direction78.85.668.2

As shown in Table T01, GuardSTR outperforms random-direction steering by 15.2, 32.4, and 23.3 percentage points on the three tasks, respectively. These substantial differences provide direct evidence that the barrier-derived steering direction contributes beyond arbitrary activation perturbations of the same magnitude.

While Fig. 3 examines sensitivity to steering strength, it does not constitute a direct comparison against a standalone decaying-strength schedule. We therefore distinguish the demonstrated contribution of the barrier-derived direction from the separate question of whether alternative scheduling rules could yield similar benefits.

Reviewer_KVVpOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GuardSTR, a framework for adaptive chain-of-thought (CoT) steering based on neural barrier functions (NBFs). The central idea is to formulate LLM reasoning as a finite-horizon stochastic reachability problem in the model's latent representation space, where reaching a target set corresponds to producing a correct final answer. The proposed connection between stochastic reachability certificates and online reasoning steering is interesting.

Critical strengths
  1. Interesting conceptual framework. Modeling CoT reasoning as stochastic reachability offers an alternative to existing activation-steering methods based predominantly on contrastive directions or heuristic feedback signals.

  2. A potentially useful combination of training and control. Learning a scalar function from outcome-labeled trajectories and using its gradient to guide online interventions is a coherent and potentially generalizable approach.

  3. Consistent empirical improvements. The reported improvements across four different model families and seven tasks suggest that the technique may capture a useful signal. The especially large gains on Dyck Languages warrant further investigation.

Critical weaknesses

Proposition 1 appears mathematically incorrect: Section 3.2, Definition 1 and Proposition 1, Eqs. (1)–(4). The paper claims that if the barrier conditions hold, then : p_success >= max(0, B(s0) + epsilon * T_max / 2 - beta). However, the stated assumptions do not appear sufficient to establish a lower bound that grows linearly with the maximum reasoning horizon Tmax⁡T_{\max}. In particular, the process may terminate successfully well before Tmax⁡T_{\max}. The expected-progress condition only constrains transitions before termination, so the number of transitions contributing positive drift need not be proportional to Tmax⁡T_{\max}.

A simple counterexample illustrates the issue is as below:

Consider two states, s0s_0 and gg, where gg is a successful terminal state. The transition is deterministic:

s0→g,g→g.s_0 \rightarrow g, \qquad g \rightarrow g.

Set

B(s0)=0,B(g)=0.5,B(s_0)=0,\quad B(g)=0.5,

\epsilon=0.1,\quad T_{\max}=30.$$ All three barrier conditions hold: 1. Non-target boundary: $B(s_0)=0\leq\beta$. 2. Target boundary: $\alpha\leq B(g)\leq 1+\beta$. 3. Expected progress: $\mathbb{E}[B(s_1)\mid s_0]=0.5\geq B(s_0)+\epsilon$. The trajectory terminates successfully after one transition, so $p_{\mathrm{succ}}=1$. Yet Proposition 1 gives $$p_{\mathrm{succ}} \geq 0+\frac{0.1}{2}(30)-0.1 =1.4.$$ This contradicts the fact that $p_{\mathrm{succ}}=1$. Thus, under the assumptions stated in the main text, Proposition 1 cannot hold generally. An additional assumption involving the stopping time or a different bound appears necessary. This is a fundamental concern because the theoretical certificate is central to the paper's claimed novelty.
Questions for authors

Please respond corresponding to the above comments about theorem.

5 comments
AuthorsOpenReview ↗

Response to Reviewer KVVp (Part 1)

We sincerely thank the reviewer for the careful evaluation and constructive feedback. We have carefully considered the comments and provide our detailed responses below.

Weakness. Proposition 1 appears mathematically incorrect: Section 3.2, Definition 1 and Proposition 1, Eqs. (1)–(4). The paper claims that if the barrier conditions hold, then: psuccess≥max⁡(0,B(s0)+ϵTMax/2−β)p_{\mathrm{success}} \geq \max(0, B(s_0) + \epsilon T_{\mathrm{Max}}/2 - \beta). However, the stated assumptions do not appear sufficient to establish a lower bound that grows linearly with the maximum reasoning horizon TMaxT_{Max}. In particular, the process may terminate successfully well before TMaxT_{Max}. The expected-progress condition only constrains transitions before termination, so the number of transitions contributing positive drift need not be proportional to TMaxT_{Max}.

A simple counterexample illustrates the issue is as below: Consider two states, s_0 and g, where g is a successful terminal state. The transition is deterministic: s_0 → g, g → g. Set B(s_0) = 0, B(g) = 0.5, β = 0.1, α = 0.5, ϵ = 0.1, TMaxT_{Max} = 30.

All three barrier conditions hold: Non-target boundary: B(s_0) = 0 ≤ β. Target boundary: α ≤ B(g) ≤ 1 + β. Expected progress: E[B(s_1) | s_0] = 0.5 ≥ B(s_0) + ϵ. The trajectory terminates successfully after one transition, so p_succ = 1. Yet Proposition 1 gives p_succ ≥ 0 + (0.1/2)(30) − 0.1 = 1.4. This contradicts the fact that p_succ = 1.

Thus, under the assumptions stated in the main text, Proposition 1 cannot hold generally. An additional assumption involving the stopping time or a different bound appears necessary. This is a fundamental concern because the theoretical certificate is central to the paper's claimed novelty.

Answer: Corrected Proposition 1 and Response to the Counterexample

We thank the reviewer for providing this counterexample, which helped us identify an omitted condition in Proposition 1 and clarify the treatment of early termination in its proof.

Upon revisiting the theoretical analysis, we identified a sufficient condition for deriving the stated linear lower bound: ϵTMax≤1\epsilon T_{\mathrm{Max}}\leq 1, which was inadvertently omitted from Proposition 1. This is a simple and readily checkable finite-horizon condition, involving only the existing drift parameter and maximum reasoning horizon. Importantly, we have verified that this condition already holds in all experimental configurations reported in the paper. Incorporating it therefore requires no changes to Definition 1, the proposed method, or the experimental results.

In the reviewer's counterexample, ϵTMax=0.1×30=3>1\epsilon T_{\mathrm{Max}}=0.1\times30=3>1, which violates this condition and therefore falls outside the scope of the corrected proposition. The impossible lower bound of 1.4 illustrates why the original unrestricted linear extrapolation must be corrected; it does not contradict the newly restricted proposition.

We have also corrected the proof in Appendix A to properly account for early successful termination. Specifically, the revised proof accumulates positive drift only on trajectories that have not yet reached the correct-answer set. Once the target set is reached, the state remains fixed and no further positive drift is assumed. For example, in the reviewer's deterministic one-step process, q0=1q_0=1 and qt=0q_t=0 for all t≥1t\geq1, so the revised proof counts positive drift only at the first step, not across all TMaxT_{\mathrm{Max}} steps.

With these corrections, the original success-probability lower bound remains unchanged. We provide the complete revised proposition and proof below.

Proposition 1 (Finite-Horizon Correct-Answer Reachability).

Fix TMax∈N>0T_{\mathrm{Max}}\in\mathbb{N}_{>0} satisfying ϵTMax≤1\epsilon T_{\mathrm{Max}}\leq 1. Suppose BϕB_\phi satisfies Definition 1, and let s0=hℓ(x)∉Gxs_0=h^\ell(x)\notin\mathcal{G}_x. Under the correspondence between target membership and successful termination specified above, the success probability from s0s_0 satisfies

p_{\mathrm{succ}} &=\Pr_{s_0}\left( \exists\\,t\in\\{1,\ldots,T_{\mathrm{Max}}\\}: s_t\in\mathcal{G}_x \right)\\\\ &\geq\max\\{ 0,\\, B_\phi(s_0)+\frac{\epsilon T_{\mathrm{Max}}}{2}-\beta \\}. \end{array}$$
AuthorsOpenReview ↗

Response to Reviewer KVVp (Part 2)

Proof. All probabilities and expectations are conditioned on the fixed initial state s0s_0. Let H=TMaxH=T_{\mathrm{Max}} and define the first hitting time of the target set as

κ=inf⁡t≥0:st∈Gx,\kappa=\inf\\{t\geq 0:s_t\in\mathcal{G}_x\\},

with κ=∞\kappa=\infty if the target is never reached. Define qt=Pr⁡s0(κ>t)q_t=\Pr_{s_0}(\kappa>t).

Since Gx\mathcal{G}_x consists of correct terminal states and the process remains fixed after termination, the target set is absorbing. Consequently,

qt=Pr⁡s0(st∉Gx),q_t=\Pr_{s_0}(s_t\notin\mathcal{G}_x),

and qtq_t is nonincreasing in tt.

The drift condition here is applied to the post-termination-extended process at each step 0≤t<H0\leq t < H; if a frozen incorrect terminal state violates this condition, the proposition does not apply to that process.

By the transition condition in Definition 1, the conditional expected barrier increment is at least ϵ\epsilon whenever st∉Gxs_t\notin\mathcal{G}_x. Once the target set is reached, the state remains fixed and the barrier increment is zero. Therefore, for every 0≤t<H0\leq t < H,

&\mathbb{E}_{s_0}[B_\phi(s_{t+1})] -\mathbb{E}_{s_0}[B_\phi(s_t)]\\\\ &\quad= \mathbb{E}_{s_0}\left[ \mathbb{E}[B_\phi(s_{t+1})\mid s_t] -B_\phi(s_t) \right]\\\\ &\quad\geq \epsilon\Pr_{s_0}(s_t\notin\mathcal{G}_x)\\\\ &\quad=\epsilon q_t. \end{array}$$ Summing over $t=0,\ldots,H-1$ yields $$\begin{array}{rl} \mathbb{E}_{s_0}[B_\phi(s_H)] &\geq B_\phi(s_0) +\epsilon\sum_{t=0}^{H-1}q_t\\\\ &\geq B_\phi(s_0)+\epsilon Hq_H, \end{array}$$ where the last inequality follows from $q_t\geq q_H$ and $\epsilon\geq 0$. The preceding expectation identity uses the tower property and does not assume that $s_t$ is Markov. On the other hand, the boundary conditions in Definition 1 imply $$\begin{array}{rl} \mathbb{E}_{s_0}[B_\phi(s_H)] &\leq\beta q_H+(1+\beta)(1-q_H)\\\\ &=1+\beta-q_H. \end{array}$$ Combining the preceding inequalities gives $$(1+\epsilon H)q_H \leq 1+\beta-B_\phi(s_0).$$ To obtain the linear lower bound, let $$A=1+\beta-B_\phi(s_0), \qquad u=\epsilon H.$$ Since $s_0\notin\mathcal{G}_x$, the non-target boundary condition gives $B_\phi(s_0)\leq\beta$ and hence $A\geq 1$. Moreover, the additional condition $\epsilon T_{\mathrm{Max}}\leq 1$ ensures $0\leq u\leq 1$. It follows that $$\begin{array}{rl} \left(A-\frac{u}{2}\right)-\frac{A}{1+u} &=\frac{u(2A-1-u)}{2(1+u)}\\\\ &\geq\frac{u(1-u)}{2(1+u)}\\\\ &\geq 0. \end{array}$$ Therefore, $$q_H\leq\frac{A}{1+u}\leq A-\frac{u}{2}.$$ By the correspondence between target membership and successful termination, and since the process remains fixed after termination, $$p_{\mathrm{succ}} =\Pr_{s_0}(\kappa\leq H) =1-q_H.$$ Consequently, $$\begin{array}{rl} p_{\mathrm{succ}} &\geq 1-A+\frac{u}{2}\\\\ &=B_\phi(s_0)+\frac{\epsilon T_{\mathrm{Max}}}{2}-\beta. \end{array}$$ Combining this inequality with $p_{\mathrm{succ}}\geq 0$ completes the proof. **In summary, the revised Proposition 1 addresses the reviewer's counterexample through an explicit, readily verifiable condition, while the corrected proof properly accounts for early successful termination without accumulating drift after reaching the target set.** The correction preserves the original lower-bound form, requires no changes to Definition 1 or GuardSTR, and is consistent with all reported experimental configurations. We appreciate the reviewer for helping us improve the precision and rigor of our theoretical analysis.
AuthorsOpenReview ↗

Follow-up on the Rebuttal

Dear Reviewer KVVp,

Thank you again for your careful review and for providing a concrete counterexample to Proposition 1. We would like to briefly follow up on our rebuttal, as this appears to be the central concern underlying your assessment.

We have carefully revised Proposition 1 and provided a complete corrected proof to address your mathematical concern. Specifically, we added the previously omitted condition ϵTMax≤1\epsilon T_{\mathrm{Max}}\leq 1, which is required to establish the stated linear lower bound, and corrected the proof to properly account for early successful termination without accumulating positive drift after reaching the target set. In your counterexample, ϵTMax=3>1\epsilon T_{\mathrm{Max}}=3>1, so it falls outside the scope of the corrected proposition.

Importantly, this additional condition is simple to check and has already been verified to hold in all our reported experimental configurations. It involves only the existing drift parameter ϵ\epsilon and maximum reasoning horizon TMaxT_{\mathrm{Max}}, without introducing additional assumptions on the model dynamics or requiring changes to the experimental setup. Thus, the revision is a straightforward and localized theoretical correction: it preserves the original lower-bound form and requires no changes to the GuardSTR algorithm, hyperparameters, experimental settings, or reported results. We have provided the complete revised proposition and its rigorous proof in our rebuttal.

We would greatly appreciate knowing whether the revised proposition and proof have addressed your primary concern, or whether any questions remain. We would be happy to provide further clarification if needed. If you find that the revised analysis satisfactorily resolves your concern, we would sincerely appreciate your reconsideration of the overall assessment.

Thank you again for your time and constructive feedback.

Sincerely,

The Authors

Reviewer_rsqrOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GuardSTR, a framework that uses neural barrier functions to guide chain-of-thought reasoning in large language models. The authors formulate reasoning as a finite-horizon stochastic reachability problem and derive conditions intended to bound the probability of reaching a correct final answer. A neural barrier function is learned from outcome-labeled reasoning trajectories and used to evaluate individual reasoning steps. During inference, the method adjusts hidden-state activations based on barrier residuals and gradients without updating the base model. Experiments on four language models and seven BBH/MMLU tasks compare GuardSTR with unsteered inference and two activation-steering baselines.

Critical strengths

(i) The paper studies the important problem of guiding intermediate reasoning steps toward correct final answers, rather than relying only on the final outcome to evaluate a reasoning trajectory.

(ii) The method connects neural barrier functions with adaptive activation steering, and the experiments report accuracy improvements across four language models and seven tasks, with an additional ablation examining adaptive steering strength.

Critical weaknesses

(i) Definition 1 requires a positive expected barrier increment only for states outside the target set. However, Appendix A defines a stopped process whose state remains unchanged after reaching the target set, while Eq. (16) still assumes a positive increment at every time step. Since the barrier value remains constant after stopping, this condition does not hold for ε > 0. The paper does not explain how Eq. (17) follows under these conditions.

(ii) The online steering rule can apply interventions even when the barrier condition is satisfied. In Eq. (9), the softmax-normalized shortfall remains positive when the raw violation is zero. At the first eligible reasoning step, the normalized shortfall is always 1, regardless of whether a violation occurs. Consequently, Eq. (10) assigns a positive steering strength whenever cα > 0, and the activation update can be applied without any constraint violation. This is inconsistent with the proposed violation-driven steering mechanism, since the rule does not suppress interventions when the barrier condition is already satisfied.

Questions for authors

(i) Please explain how the positive expected barrier increment in Eq. (16) of Appendix A can hold after the stopped process reaches the target set, and how Eq. (17) follows under this condition.

(ii) Please clarify why Eq. (9) produces a positive normalized shortfall even when the barrier condition is satisfied, and how GuardSTR avoids unnecessary steering in such cases.

After Rebuttal

5 comments
AuthorsOpenReview ↗

Response to Reviewer rsqr (Part 1)

We sincerely thank the reviewer for the careful evaluation and constructive feedback. We have carefully considered the comments and provide our detailed responses below.

Weakness 1. Definition 1 requires a positive expected barrier increment only for states outside the target set. However, Appendix A defines a stopped process whose state remains unchanged after reaching the target set, while Eq. (16) still assumes a positive increment at every time step. Since the barrier value remains constant after stopping, this condition does not hold for ϵ>0\epsilon>0. The paper does not explain how Eq. (17) follows under these conditions.

Answer: Corrected Proposition 1 and Finite-Horizon Proof

We thank the reviewer for identifying this issue. We agree that the original proof incorrectly accumulated positive drift after the process had reached the target set, where the barrier remains constant. We have corrected this step in Appendix A, ensuring that positive drift is accumulated only on trajectories that have not yet reached the correct-answer set, rather than unconditionally at every step.

Separately, we identified an omitted numerical condition in Proposition 1: the algebraic derivation of the stated linear lower bound from the intermediate probability inequality uses ϵTMax≤1\epsilon T_{\mathrm{Max}}\leq 1, which was inadvertently omitted. This is a simple and readily checkable finite-horizon condition, requiring only that the product of the drift parameter and the maximum reasoning horizon does not exceed the unit scale of the barrier bounds. Importantly, we have verified that this condition already holds in all experimental configurations reported in the paper. The numerical condition is already met without retuning the method or changing the reported experimental results. The guarantee, however, applies only when Definition 1 holds for the process being analyzed.

With these two corrections, the original success-probability lower bound remains unchanged. The revised proof explicitly accounts for stopping and establishes the bound under the clarified condition. We provide the complete revised proposition and proof below.

Proposition 1 (Finite-Horizon Correct-Answer Reachability).

Fix TMax∈N>0T_{\mathrm{Max}}\in\mathbb{N}_{>0} satisfying ϵTMax≤1\epsilon T_{\mathrm{Max}}\leq 1. Suppose BϕB_\phi satisfies Definition 1, and let s0=hℓ(x)∉Gxs_0=h^\ell(x)\notin\mathcal{G}_x. Under the correspondence between target membership and successful termination specified above, the success probability from s0s_0 satisfies

p_{\mathrm{succ}} &=\Pr_{s_0}\left( \exists\\,t\in\\{1,\ldots,T_{\mathrm{Max}}\\}: s_t\in\mathcal{G}_x \right)\\\\ &\geq\max\\{ 0,\\, B_\phi(s_0)+\frac{\epsilon T_{\mathrm{Max}}}{2}-\beta \\}. \end{array}$$
AuthorsOpenReview ↗

Response to Reviewer rsqr (Part 2)

Proof. All probabilities and expectations are conditioned on the fixed initial state s0s_0. Let H=TMaxH=T_{\mathrm{Max}} and define the first hitting time of the target set as

κ=inf⁡t≥0:st∈Gx,\kappa=\inf\\{t\geq 0:s_t\in\mathcal{G}_x\\},

with κ=∞\kappa=\infty if the target is never reached. Define qt=Pr⁡s0(κ>t)q_t=\Pr_{s_0}(\kappa>t).

Since Gx\mathcal{G}_x consists of correct terminal states and the process remains fixed after termination, the target set is absorbing. Consequently,

qt=Pr⁡s0(st∉Gx),q_t=\Pr_{s_0}(s_t\notin\mathcal{G}_x),

and qtq_t is nonincreasing in tt.

The drift condition here is applied to the post-termination-extended process at each step 0≤t<H0\leq t < H; if a frozen incorrect terminal state violates this condition, the proposition does not apply to that process.

By the transition condition in Definition 1, the conditional expected barrier increment is at least ϵ\epsilon whenever st∉Gxs_t\notin\mathcal{G}_x. Once the target set is reached, the state remains fixed and the barrier increment is zero. Therefore, for every 0≤t<H0\leq t < H,

&\mathbb{E}_{s_0}[B_\phi(s_{t+1})] -\mathbb{E}_{s_0}[B_\phi(s_t)]\\\\ &\quad= \mathbb{E}_{s_0}\left[ \mathbb{E}[B_\phi(s_{t+1})\mid s_t] -B_\phi(s_t) \right]\\\\ &\quad\geq \epsilon\Pr_{s_0}(s_t\notin\mathcal{G}_x)\\\\ &\quad=\epsilon q_t. \end{array}$$ Summing over $t=0,\ldots,H-1$ yields $$\begin{array}{rl} \mathbb{E}_{s_0}[B_\phi(s_H)] &\geq B_\phi(s_0) +\epsilon\sum_{t=0}^{H-1}q_t\\\\ &\geq B_\phi(s_0)+\epsilon Hq_H, \end{array}$$ where the last inequality follows from $q_t\geq q_H$ and $\epsilon\geq 0$. The preceding expectation identity uses the tower property and does not assume that $s_t$ is Markov. On the other hand, the boundary conditions in Definition 1 imply $$\begin{array}{rl} \mathbb{E}_{s_0}[B_\phi(s_H)] &\leq\beta q_H+(1+\beta)(1-q_H)\\\\ &=1+\beta-q_H. \end{array}$$ Combining the preceding inequalities gives $$(1+\epsilon H)q_H \leq 1+\beta-B_\phi(s_0).$$ To obtain the linear lower bound, let $$A=1+\beta-B_\phi(s_0), \qquad u=\epsilon H.$$ Since $s_0\notin\mathcal{G}_x$, the non-target boundary condition gives $B_\phi(s_0)\leq\beta$ and hence $A\geq 1$. Moreover, the additional condition $\epsilon T_{\mathrm{Max}}\leq 1$ ensures $0\leq u\leq 1$. It follows that $$\begin{array}{rl} \left(A-\frac{u}{2}\right)-\frac{A}{1+u} &=\frac{u(2A-1-u)}{2(1+u)}\\\\ &\geq\frac{u(1-u)}{2(1+u)}\\\\ &\geq 0. \end{array}$$ Therefore, $$q_H\leq\frac{A}{1+u}\leq A-\frac{u}{2}.$$ By the correspondence between target membership and successful termination, and since the process remains fixed after termination, $$p_{\mathrm{succ}} =\Pr_{s_0}(\kappa\leq H) =1-q_H.$$ Consequently, $$\begin{array}{rl} p_{\mathrm{succ}} &\geq 1-A+\frac{u}{2}\\\\ &=B_\phi(s_0)+\frac{\epsilon T_{\mathrm{Max}}}{2}-\beta. \end{array}$$ Combining this inequality with $p_{\mathrm{succ}}\geq 0$ completes the proof. **In summary, the revision addresses both issues raised by the reviewer:** the drift is now correctly accumulated only before reaching the target set, and the previously omitted condition $\epsilon T_{\mathrm{Max}}\leq 1$ justifies the final linear lower bound. **The correction is limited to the proposition statement and its proof, preserves the original guarantee, and is consistent with all reported experimental configurations.** We appreciate the reviewer for helping us improve the precision and rigor of our theoretical presentation.
AuthorsOpenReview ↗

Response to Reviewer rsqr (Part 3)

Weakness 2. The online steering rule can apply interventions even when the barrier condition is satisfied. In Eq. (9), the softmax-normalized shortfall remains positive when the raw violation is zero. At the first eligible reasoning step, the normalized shortfall is always 1, regardless of whether a violation occurs. Consequently, Eq. (10) assigns a positive steering strength whenever cα>0c_\alpha>0, and the activation update can be applied without any constraint violation. This is inconsistent with the proposed violation-driven steering mechanism, since the rule does not suppress interventions when the barrier condition is already satisfied.

Answer: Clarification of History-Normalized Steering Credit

We thank the reviewer for this observation. We agree that our steering rule is not a strict violation-triggered gate: softmax normalization assigns a positive credit even when the raw barrier shortfall is zero. In particular, at the first eligible reasoning step, the normalized credit is always 1, regardless of whether a violation occurs. We clarify that the purpose of Eq. (9) is to allocate steering strength based on the relative barrier shortfall over the reasoning history, rather than to make a binary intervention decision.

Importantly, a zero-shortfall step has a steering-strength upper bound that decreases with the history length. Since the raw shortfalls are nonnegative, a zero-shortfall step receives a normalized credit of at most 1/t1/t, giving αtℓ≤cα/t\alpha_t^\ell\leq c_\alpha/t. In particular, if all observed shortfalls are zero, we have

\qquad \alpha_t^\ell=\frac{c_\alpha}{t}.$$ Thus, **when no violations occur, the steering strength decays toward zero as reasoning progresses**. Conversely, larger shortfalls receive greater relative weight under softmax normalization, allowing the controller to prioritize steps with more pronounced barrier violations. We acknowledge that the limited history at early reasoning steps can make the normalized credit less informative, potentially introducing unnecessary interventions. However, this behavior is confined in magnitude by the steering coefficient $c_\alpha$, while the accumulation of historical information provides an increasingly informative reference for allocating steering strength. The normalization also yields bounded, history-dependent credits, avoiding the direct use of potentially varying raw residual magnitudes as activation strengths. Therefore, **GuardSTR employs violation-informed, history-normalized steering rather than a strict violation-triggered gate**. Although this design does not completely suppress interventions on satisfied steps or guarantee that early interventions are harmless, the consistent accuracy improvements across **all 28 model–task pairs, with a macro-average gain of 5.91 percentage points**, provide empirical evidence of its effectiveness in the evaluated settings. These results suggest that the potential noise introduced by early normalization does not negate the overall benefits of barrier-guided steering. **Question (i)** Please explain how the positive expected barrier increment in Eq. (16) of Appendix A can hold after the stopped process reaches the target set, and how Eq. (17) follows under this condition. **Answer:** Please refer to our detailed response to **Weakness 1**, where we clarify the treatment of the stopping time, correct the accumulation of positive drift, and provide the complete revised proof showing how the corrected inequality leads to Eq. (17). **Question (ii)** Please clarify why Eq. (9) produces a positive normalized shortfall even when the barrier condition is satisfied, and how GuardSTR avoids unnecessary steering in such cases. **Answer:** Please refer to our detailed response to **Weakness 2**, where we clarify why softmax normalization assigns positive credit even when the barrier condition is satisfied, explain why zero-shortfall steps can still receive nonzero but bounded steering, and clarify why this history-dependent normalization does not amount to a strict intervention gate.
Reviewer_yjSJOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GUARDSTR, which frames chain-of-thought reasoning as a finite-horizon stochastic reachability problem over an LLM's hidden states. It trains a neural barrier function on sampled trajectories and uses it at each reasoning step to set the direction and strength of activation steering. The paper claims a probabilistic lower bound on the chance of reaching a correct answer (Proposition 1). Across four LLMs and seven BBH and MMLU tasks, it reports improvements over unsteered decoding on all 28 model–task pairs, with a macro-average gain of 5.91 points. An ablation shows that adaptive steering strength outperforms a fixed mean strength on all 28 pairs.

Critical strengths
  1. The framing is novel and interesting. Treating reasoning as stochastic reachability gives step-level feedback a clear meaning, and the barrier function is trained using only final-answer labels.
  2. GUARDSTR achieves the highest mean accuracy on all 28 model–task pairs, with macro-average gains of 5.91, 5.80 and 5.61 points over Base, Static CAA and K-CAST.
  3. The method requires no fine-tuning of the LLM and no ground-truth answers at inference. Results are averaged over five decoding runs and reported with standard deviations.
  4. The fixed-mean ablation keeps the same barrier function and data, and shows that adapting the steering strength helps on all 28 pairs, by 3.04 points on average.
Critical weaknesses
  1. Proposition 1 does not hold as stated. Its proof (Eq. 16, p.14) assumes the barrier rises by ε at every step up to TmaxT_\text{max}, including after the process has stopped, where the state, and therefore the barrier, is held fixed. As a counterexample, take a one-step model that is correct 20% of the time, with B = 0 at the question, 0.5 on a wrong answer, 1.5 on a correct answer, β=0.5\beta = 0.5 and Tmax= 3T_\text{max} = 3. All stated conditions hold with ϵ=0.7\epsilon = 0.7, yet the bound gives 55% against a true success rate of 20%. Could the authors correct the theorem or its conditions?
  2. The guarantee is not checked for the process that is actually run. Definition 1 requires the conditions to hold under the steered process, but the barrier is trained only on unsteered trajectories. Validation then accepts it if 75% of held-out states satisfy the conditions and the bound exceeds half the base model's success rate. No certificate therefore applies to the reported results. Could the authors verify the conditions on steered trajectories?
  3. No experiment isolates the effect of the barrier function. The Base results reuse vLLM runs, while all steered conditions use Hugging Face Transformers, and there is no zero-strength run on Transformers. There is also no control with random steering directions or a simple decaying schedule. Could the authors add these controls, so that the gains can be attributed to the barrier function rather than the software backend or the steering itself?
Questions for authors

See weakness

7 comments
AuthorsOpenReview ↗

Response to Reviewer yjSJ (Part 1)

We sincerely thank the reviewer for the careful evaluation and constructive feedback. We have carefully considered the comments and provide our detailed responses below.

Weakness 1. Proposition 1 does not hold as stated. Its proof (Eq. 16, p.14) assumes the barrier rises by ε at every step up to TMaxT_{Max}, including after the process has stopped, where the state, and therefore the barrier, is held fixed. As a counterexample, take a one-step model that is correct 20% of the time, with BB = 0 at the question, 0.5 on a wrong answer, 1.5 on a correct answer, β\beta = 0.5 and TMaxT_{Max} = 3. All stated conditions hold with ϵ\epsilon = 0.7, yet the bound gives 55% against a true success rate of 20%. Could the authors correct the theorem or its conditions?

Answer: Revised Proposition 1 and Proof

We thank the reviewer for this insightful counterexample, which helped us identify an omitted condition in Proposition 1 and an issue in the treatment of the stopping time in its proof. We have carefully revisited both issues and made the corresponding corrections.

(1) Clarification of the omitted condition. We identified that the derivation of the stated linear lower bound uses the sufficient condition ϵTMax≤1\epsilon T_{\mathrm{Max}}\leq 1, which was inadvertently omitted from Proposition 1. Specifically, this condition justifies the algebraic passage from the intermediate survival-probability inequality to the stated linear bound; it does not replace the barrier conditions required of the process being analyzed. It is a simple and readily checkable finite-horizon condition, involving only the existing drift parameter ϵ\epsilon and maximum reasoning horizon TMaxT_{\mathrm{Max}}. In our setting, this condition imposes no additional changes to the model or steering mechanism. Importantly, we have verified that it already holds in all experimental configurations reported in the paper, so no modifications to GuardSTR, its parameter settings, or the experimental results are required.

In the reviewer's counterexample, ϵTMax=0.7×3=2.1>1\epsilon T_{\mathrm{Max}}=0.7\times3=2.1>1, which violates the newly stated condition and therefore falls outside the scope of the corrected proposition. This example also illustrates why the original linear bound should not be extrapolated beyond the range justified by this condition.

(2) Correction of the stopping-time argument. We have also revised the proof in Appendix A to properly account for the first hitting time κ\kappa of the correct-answer set. In particular, positive drift is accumulated only on trajectories that have not yet reached the target set, rather than unconditionally at every step. Once the target set is reached, the state remains fixed and no further positive drift is assumed. The corrected proof therefore accounts for early successful termination and avoids the issue identified by the reviewer.

With these two corrections, the original success-probability lower bound is preserved. The numerical condition is already satisfied by the reported experimental settings, and the mathematical revision concerns the proposition statement and Appendix A rather than the GuardSTR algorithm. The guarantee remains conditional on Definition 1 holding for the analyzed process. We provide the complete revised proposition and proof below.

Proposition 1 (Finite-Horizon Correct-Answer Reachability).

Fix TMax∈N>0T_{\mathrm{Max}}\in\mathbb{N}_{>0} satisfying ϵTMax≤1\epsilon T_{\mathrm{Max}}\leq 1. Suppose BϕB_\phi satisfies Definition 1, and let s0=hℓ(x)∉Gxs_0=h^\ell(x)\notin\mathcal{G}_x. Under the correspondence between target membership and successful termination specified above, the success probability from s0s_0 satisfies

p_{\mathrm{succ}} &=\Pr_{s_0}\left( \exists\\,t\in\\{1,\ldots,T_{\mathrm{Max}}\\}: s_t\in\mathcal{G}_x \right)\\\\ &\geq\max\\{ 0,\\, B_\phi(s_0)+\frac{\epsilon T_{\mathrm{Max}}}{2}-\beta \\}. \end{array}$$
AuthorsOpenReview ↗

Response to Reviewer yjSJ (Part 2)

Proof. All probabilities and expectations are conditioned on the fixed initial state s0s_0. Let H=TMaxH=T_{\mathrm{Max}} and define the first hitting time of the target set as

κ=inf⁡t≥0:st∈Gx,\kappa=\inf\\{t\geq 0:s_t\in\mathcal{G}_x\\},

with κ=∞\kappa=\infty if the target is never reached. Define qt=Pr⁡s0(κ>t)q_t=\Pr_{s_0}(\kappa>t).

Since Gx\mathcal{G}_x consists of correct terminal states and the process remains fixed after termination, the target set is absorbing. Consequently,

qt=Pr⁡s0(st∉Gx),q_t=\Pr_{s_0}(s_t\notin\mathcal{G}_x),

and qtq_t is nonincreasing in tt.

The drift condition here is applied to the post-termination-extended process at each step 0≤t<H0\leq t < H; if a frozen incorrect terminal state violates this condition, the proposition does not apply to that process.

By the transition condition in Definition 1, the conditional expected barrier increment is at least ϵ\epsilon whenever st∉Gxs_t\notin\mathcal{G}_x. Once the target set is reached, the state remains fixed and the barrier increment is zero. Therefore, for every 0≤t<H0\leq t < H,

&\mathbb{E}_{s_0}[B_\phi(s_{t+1})] -\mathbb{E}_{s_0}[B_\phi(s_t)]\\\\ &\quad= \mathbb{E}_{s_0}\left[ \mathbb{E}[B_\phi(s_{t+1})\mid s_t] -B_\phi(s_t) \right]\\\\ &\quad\geq \epsilon\Pr_{s_0}(s_t\notin\mathcal{G}_x)\\\\ &\quad=\epsilon q_t. \end{array}$$ Summing over $t=0,\ldots,H-1$ yields $$\begin{array}{rl} \mathbb{E}_{s_0}[B_\phi(s_H)] &\geq B_\phi(s_0) +\epsilon\sum_{t=0}^{H-1}q_t\\\\ &\geq B_\phi(s_0)+\epsilon Hq_H, \end{array}$$ where the last inequality follows from $q_t\geq q_H$ and $\epsilon\geq 0$. The preceding expectation identity uses the tower property and does not assume that $s_t$ is Markov. On the other hand, the boundary conditions in Definition 1 imply $$\begin{array}{rl} \mathbb{E}_{s_0}[B_\phi(s_H)] &\leq\beta q_H+(1+\beta)(1-q_H)\\\\ &=1+\beta-q_H. \end{array}$$ Combining the preceding inequalities gives $$(1+\epsilon H)q_H \leq 1+\beta-B_\phi(s_0).$$ To obtain the linear lower bound, let $$A=1+\beta-B_\phi(s_0), \qquad u=\epsilon H.$$ Since $s_0\notin\mathcal{G}_x$, the non-target boundary condition gives $B_\phi(s_0)\leq\beta$ and hence $A\geq 1$. Moreover, the additional condition $\epsilon T_{\mathrm{Max}}\leq 1$ ensures $0\leq u\leq 1$. It follows that $$\begin{array}{rl} \left(A-\frac{u}{2}\right)-\frac{A}{1+u} &=\frac{u(2A-1-u)}{2(1+u)}\\\\ &\geq\frac{u(1-u)}{2(1+u)}\\\\ &\geq 0. \end{array}$$ Therefore, $$q_H\leq\frac{A}{1+u}\leq A-\frac{u}{2}.$$ By the correspondence between target membership and successful termination, and since the process remains fixed after termination, $$p_{\mathrm{succ}} =\Pr_{s_0}(\kappa\leq H) =1-q_H.$$ Consequently, $$\begin{array}{rl} p_{\mathrm{succ}} &\geq 1-A+\frac{u}{2}\\\\ &=B_\phi(s_0)+\frac{\epsilon T_{\mathrm{Max}}}{2}-\beta. \end{array}$$ Combining this inequality with $p_{\mathrm{succ}}\geq 0$ completes the proof.
AuthorsOpenReview ↗

Response to Reviewer yjSJ (Part 3)

Weakness 2. The guarantee is not checked for the process that is actually run. Definition 1 requires the conditions to hold under the steered process, but the barrier is trained only on unsteered trajectories. Validation then accepts it if 75% of held-out states satisfy the conditions and the bound exceeds half the base model's success rate. No certificate therefore applies to the reported results. Could the authors verify the conditions on steered trajectories?

Answer: Clarification of Theoretical Guarantees and Empirical Validation

We thank the reviewer for raising this important concern. We agree that learning a neural barrier function (NBF) from unsteered trajectories, including both successful and unsuccessful reasoning attempts, does not by itself establish a formal guarantee for the deployed steered process. We have not exhaustively verified the barrier conditions on the steered trajectories, and therefore do not present the reported results as certified lower bounds.

As clarified in the manuscript (lines 380–383), Proposition 1 provides a conditional guarantee that requires the boundary conditions to hold throughout the specified state space and the state-conditional expected-drift condition to hold for non-target states under the process being analyzed. Exhaustively verifying these conditions for LLM reasoning remains challenging due to the complexity of the underlying state space. The 75% empirical constraint-satisfaction threshold and bound-value screening are checkpoint-selection criteria, not a formal certificate for the steered process. We will make this distinction explicit in the revised manuscript.

Nevertheless, GuardSTR demonstrates consistent empirical effectiveness. Across four LLMs and seven tasks from BBH and MMLU, it improves final-answer accuracy over the unsteered baselines on all 28 model–task pairs, with a macro-average gain of 5.91 percentage points. It also achieves improvements of up to 12.8 and 11.6 percentage points over Static CAA and K-CAST, respectively. These results support the practical effectiveness of NBF-guided reasoning, while the formal guarantee remains conditional on the assumptions of Proposition 1.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.