acceptodds
Under review as a conference paper at ICLR 2027

When More Caution Can Make LLMs Less Safe: Risk Control for Iterative Reasoning

Abstract

Iterative LLM systems emit candidate answers over several rounds and must decide when to act, continue, or escalate; given a budget , we seek a stopping threshold controlling the wrong-action rate , which is what "safer" means here. Greater caution can increase this risk, since a higher threshold may delay action until a later round overturns an already-correct answer, so the pointwise loss monotonicity conformal risk control (CRC) requires can fail; decomposing the risk change between nested rules says exactly when caution helps. More strongly, the full risk and automation curves, the score-process distribution and every round's accuracy do not reveal whether that premise holds: two populations can agree on all of them, yet in one no trajectory violates pointwise monotonicity and in the other all do. They differ only in how correctness at one round is coupled to correctness at another, so no non-trivial bound on the violating fraction follows from any of those curves: a failure of identification, not of estimation, which an oracle would not resolve. Empirically the difficulty grows with reasoning: on MMLU-Pro, using prefixes of one run, accuracy gains saturate while correct answers broken climb from 1.9% to 8.7% of instances and the share of calibration draws with an upward risk step rises from 33% to 90%, while the aggregate curve remains nearly flat. We therefore certify stopping policies directly, keeping calibration labels isolated from score fitting, grid and budget, which gives without monotonicity and without any assumption on the score. Iterative systems need trajectory-aware certification, because aggregate risk cannot reveal whether trajectory-level monotonicity holds.

Primary area:
learning theory
Keywords:
iterative reasoning, risk control, selective autonomy, conformal risk control, non-monotone losses, stopping policies, trajectory-level reliability, multi-agent debate

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_8sfJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the stopping mechanism with risk control in iterative systems, where predictions may be revised over multiple iterations. Authors consider three policies: taking action based on the current prediction, continuing reasoning, or postponing the decision. The goal is to maximize the degree of automation while limiting the risk of incorrect operation (R(λ) = P(execution and error)).

The paper first proves that increasing the stopping threshold does not necessarily reduce this risk, as additional reasoning processes may overturn the initially correct prediction. Theorem 1 decomposes the risk change into transitions from correct to incorrect, from incorrect to correct, and delays in executing the incorrect operation. Subsequently, the author establishes non-identifiability results.

To this end, this paper adopts fixed-sequence LTT, which obtains risk guarantees without monotonicity assumption. Experiments on multi-agent negotiation and single-agent self-correction study the instability of the answer, the non-monotonicity of risk, and the resulting trade-off between automation and risk.

Critical strengths
  1. This paper uses rigorous probabilistic language to describe a common phenomenon: demanding a higher degree of confidence before taking action may lead to subsequent reasoning overturning the previously correct answer and resulting in worse decisions. The decomposition in Theorem 1 clearly depicts this effect. The non-identifiability construction in Theorem 4 further explains why the overall performance usually fails to reflect the transformation of the underlying correctness. It is meaningful to describe this phenomenon using strict probabilistic language

  2. The approach proposed by this paper is reasonable, and the theoretical proof seems to be correct (based on my initial assessment). The article also conducted a series of experiments to prove the idea.

Critical weaknesses
  1. While the studied problem is of interest, much of the formal results follows from relatively elementary probability arguments or already established risk-control framework. The contribution appears primarily interpretive rather than a new statistical framework. Moreover, Theorem 6 is an application of existing fixed-sequence LTT with appropriate sample splitting, as the authors acknowledge. From the perspective of methodology and the depth of theory, the contribution needs to be clarified. More importantly, the paper seems to have failed to adequately discuss the existing literature on early exit under risk control. This largely determines my assignmenton the new contributions made to the article. Specifically, Schuster et al. (Confident adaptive language modeling, NeurIPS 2022) seem have discussed non-monotonic performance as exit thresholds vary and use LTT for PAC calibration. Jazbec et al. (Fast yet Safe: Early-Exiting with Risk Control, NeurIPS 2024) seem have distinguished pointwise loss monotonicity from marginal risk monotonicity and study both expected-risk and high-probability guarantees for sequential exit decisions. Although these papers considered different settings, their concepts and methods seem to have significant overlap with existing work. Therefore, a more detailed comparison is necessary.

  2. Although the paper shows that stopping risk can be non-monotone, its practical impact seems to be limited. Figure 3 shows that harmful threshold pairs account for at most 1.6% of evaluated pairs, and Appendix K.1 reports a maximum automation loss of only 0.73% from monotone search. Figure 4 primarily demonstrates the known distinction between expected-risk and high-probability guarantees, rather than the benefit of addressing non-monotonicity. Stronger empirical evidence is needed to demonstrate the practical significance of this work.

Questions for authors
  1. Could the authors clarify the precise theoretical and methodological differences from Schuster et al. (NeurIPS 2022) and Jazbec et al. (NeurIPS 2024)? Can the approach described in the article for handling non-monotonic risks be transferred to some of the existing works, such as Yu et al. (Anytime Safe PAC Efficient Reasoning, ICML 2026)?

  2. How does the proposed method compare with other high-probability risk-control procedures under the same risk definition? The comparisons in Appendix I are very useful, but they would be even more helpful when compared with more appropriate baselines.

  3. I read the article very carefully, but to understand it, I had to make guesses based on my domain knowledge. The main thread of the article is mixed with too much supplementary clarifications (maybe you can clarify these issues in the remark or other places), which makes the reading less smooth. There are also some confusing notations in the article, such as the subscript of the probability in equation (6). Making detailed revisions to these issues will significantly enhance the clarity of the article.

I don't mind improving my score if the author can provide reasonable responses to the aforementioned potential weaknesses and questions.

6 comments

We thank Reviewer 8sfJ for the close reading, and for finding the decomposition clear and the non-identifiability construction meaningful. The revision addresses the novelty, significance and baseline questions directly. Changes are in red, and the old Theorem 6 (validity) is now Theorem 7.

W1. THEORETICAL NOVELTY AND RELATION TO EXISTING FRAMEWORKS

"Elementary arguments or an existing framework; Theorem 6 applies existing LTT."

The identification problem behind risk-controlled stopping. Each ingredient is familiar: answers flip, CRC needs monotone losses, and CRC can fail when losses are not monotone. The contribution is the distinction between them that is easy to miss, namely that a monotone population risk curve, the quantity one can estimate and check, does not certify the instance-level monotonicity the guarantee requires. That the distinction is subtle is shown by the literature itself: one step of the marginal-risk argument of Jazbec et al. relies on it (W1 cont. below).

The central result is identification. Theorem 4 constructs one set of aggregate summaries (the full risk and automation curves, score-process distribution and all round-wise accuracies) compatible with every pointwise violation rate between 0 and 1. Even exact knowledge of these quantities therefore cannot settle the premise. This is stronger than observing that average monotonicity does not imply individual monotonicity.

Its consequence for CRC (new Proposition 6). In a separate construction, two populations with identical aggregate summaries of the same kind make the same raw CRC selector meet its expected-risk target on one and violate it on the other. With w=⌊ε(n+1)⌋−1w=\lfloor\varepsilon(n+1)\rfloor-1 and π=Pr⁡(Bin(n,p)≤w)>0\pi=\Pr(\mathrm{Bin}(n,p)\le w) > 0, the two expected risks are

pπ≤ε,p1−(1−π)T,p\pi\le\varepsilon,\qquad p\\{1-(1-\pi)^T\\},

the second exceeding ε\varepsilon exactly when

T>log⁡(1−ε/p)log⁡(1−π).T > \frac{\log(1-\varepsilon/p)}{\log(1-\pi)}.

SettingTarget ε\varepsilonP0P_0 (valid)P1P_1Source
p=0.3p=0.3, n=100n=100, T=12T=1228%8.891%29.558% (5.6% over)Table 28 (closed form and simulation)
p=0.015p=0.015, n=500n=500, T=20T=201%0.195%1.408% (40.8% over)Appendix P.4

These are theoretical constructions, not violations observed in LLM data.

Attribution. The independent-loss population P1P_1 is the counterexample in the proof of CRC's Proposition 1 (Angelopoulos et al., 2024), also analysed by Aldirawi et al. (2026); the revision credits Angelopoulos et al. in §3 and both in Appendix A.

Our additions are a valid population P0P_0 with the same aggregate summaries, and a curve-dependent upper bound on the excess for every coupling:

E[R(λ^CRC)]≤ε+Δn(R),\mathbb{E}[R(\hat{\lambda}_{\mathrm{CRC}})]\le\varepsilon+\Delta_n(R),

\Delta_n(R)=\int_\varepsilon^1\min\left\\{1,\sum_{j:\\,R(\lambda_j) > r}F_{R(\lambda_j)}\right\\}\\,dr,

with

Fr=Pr⁡(Bin(n,r)≤⌊ε(n+1)⌋−1).F_r=\Pr(\mathrm{Bin}(n,r)\le\lfloor\varepsilon(n+1)\rfloor-1).

Part (ii) concerns CRC on the raw loss; the loss-monotonised variant of Angelopoulos et al. is valid by construction, at a cost in automation (Appendix A).

From selection bias to non-identifiability. In P0P_0 each instance is right at every round or wrong at every round, so the calibration error counts at the acting thresholds move together. In P1P_1 correctness is redrawn each round, so the counts vary independently, and CRC, which selects the least cautious passing threshold, can select an over-budget threshold that passes by chance.

The mechanism is a selection effect; what is new is that the dependence driving it is not identified by the summaries above, so they cannot, in general, determine whether CRC meets its target. Proposition 6(i) bounds the possible cost, with equality approached by this family as the horizon grows; we do not claim finite-horizon sharpness.

The results form one chain:

  • Identification (Theorem 4, Corollary 5): Aggregate summaries do not identify trajectory structure.
  • Statistical consequence (Proposition 6(ii)): The hidden dependence can change CRC's expected-risk validity.
  • Quantitative bound (Proposition 6(i)): The curve still bounds the excess.

The probabilistic arguments are relatively simple; their contribution is the identification result they establish. We claim no novelty for fixed-sequence LTT: Theorem 7 states the independence conditions under which it certifies our fitted stopping policies.

1/3

W1 (cont.). RELATION TO EARLY EXIT WITH RISK CONTROL

"Insufficient discussion of early exit with risk control (Schuster et al.; Jazbec et al.)."

Related Work (§6) now discusses both papers and Yu et al. (2026), and Appendix B adds the point on Jazbec et al. below. Their relation to ours:

Prior workTargetMonotonicity assumedRelation to this paper
CALM (2022)Consistency with full model, high-prob.None (fixed-sequence LTT)Shared calibration machinery; no identification result
Jazbec et al. (2024)Performance gap; CRC and UCBMarginally monotone network, hence monotone risk (their Lemma 1)Prop. 6: marginal risk monotonicity alone does not validate raw CRC for wrong-action risk
Yu et al. (2026)Performance loss, anytime-validNone needed (anytime-valid sequential testing)Different feedback model (online, partial)

The marginal-risk argument of Jazbec et al. The step from their Eq. (14) to the final loss comparison (Appendix A.1) requires additional justification: the augmented-sample threshold depends on the test instance, so its expected population risk and its expected loss on that instance need not coincide.

Pointwise loss monotonicity would establish the required inequality directly, whereas marginal risk monotonicity alone does not; the counterexample in CRC's own Proposition 1 already has a non-increasing risk curve. Proposition 6 makes this distinction precise for wrong-action risk.

Their UCB guarantee is a fixed-sequence test and is unaffected, and we do not claim a failure of their performance-gap guarantees (Appendix B).

W2. PRACTICAL SIGNIFICANCE AND EMPIRICAL EVIDENCE

"Limited practical impact: harmful pairs ≤1.6%, a 0.73 pp search cost; Figure 4 shows a known distinction."

As the reviewer notes, harmful pairs and the measured automation penalty are small. Neither quantity directly establishes whether the pointwise monotonicity assumption required by standard CRC holds. The revision distinguishes them from direct measurements of premise violations and from the statistical consequences of those violations:

QuantityValueWhat it measures
Harmful threshold pairs≤1.6% (Fig. 3)Net risk increases; depends on the grid and on the balance of breakage, repair and deferral
Premise violations (own loss non-monotone)0.54–3.48% of instances; 1.1% → 4.2% with the horizon (§5.1, Table 25)The assumption standard CRC needs
Violations under couplings preserving the evaluated summariesMMLU-Pro 1.15% → 12.9% (Table 29)What aggregate summaries cannot detect
CRC's expected risk under those couplingsBelow β\beta in every simulated coupling (Table 29)Observed size of the effect
Prop. 6(i) upper bound on the excess≤0.9 pp (10-threshold grid), ≤4.9 pp (paper's grid), n=100n=100Worst case over compatible couplings, not an observed violation
Monotone-search cost0.73 pp (App. K.1)Automation lost on an exactly known curve, not a validity violation
Draw-wise exceedancePlug-in ≤66%, CRC ≤52%, LTT ≤8.5% of draws (Fig. 4a)Expected-risk versus draw-wise control

A nearly monotone risk curve is therefore insufficient to settle the premise, and our case rests on the identification and validity results rather than on a large measured automation penalty.

Figure 4 does address a separate question, expected-risk versus draw-wise control, and does not contradict CRC's own expected-risk target; Figure 3 studies structural non-monotonicity, and the revision makes the division explicit.

2/3

Q1. RELATION TO PRIOR WORK AND TRANSFER TO B-PAC

"Differences from Schuster and Jazbec? Transfer to Yu et al.?"

The comparison above identifies the shared tools and the distinct results. B-PAC's validity rests on anytime-valid sequential testing under online partial feedback, not on monotonicity, so Proposition 6 does not show a failure of its guarantee.

Our approach transfers directly to an offline wrong-action evaluation when complete labelled trajectories are available: fix the candidate policies independently of an i.i.d. certification set and apply Corollary 12 (validity for any pre-specified family), with no monotonicity requirement.

With online partial feedback, preserving B-PAC's anytime guarantee would instead require valid policy-specific feedback estimators and sequential tests, which the offline binomial test does not provide. Theorem 1 applies whenever the candidate routing or stopping rules are nested stopping times.

Q2. COMPARISON WITH OTHER HIGH-PROBABILITY PROCEDURES

"Comparison with other high-probability procedures under the same risk?"

The original Appendix I includes Bonferroni LTT, plus singleton-set and stability policies certified by the same fixed-sequence wrapper (Tables 4 and 10); fixed-sequence versus Bonferroni automation is 82.3% versus 77.1% on MMLU-Pro and 82.7% versus 60.0% on BBH.

The revision adds a matched comparison with the same risk, score, grid, budget, calibration draws and untouched deployment set for every selector (Tables 26–27; m=150m=150, 400 draws, γ=0.1\gamma=0.1).

Cells show deployment automation / % of draws exceeding β\beta:

SelectorMMLU-ProLogiQAARCBBHMuSR
Fixed-sequence LTT, exact binomial70.1 / 8.556.5 / 1.221.4 / 4.054.1 / 7.021.9 / 0.0
UCB inversion, Clopper–Pearson70.1 / 8.556.5 / 1.221.4 / 4.054.1 / 7.021.9 / 0.0
Fixed-sequence LTT, Hoeffding–Bentkus66.4 / 4.850.2 / 0.221.4 / 4.022.5 / 2.519.4 / 0.0
Holm, exact binomial44.3 / 0.236.9 / 0.00.0 / 0.00.0 / 0.015.0 / 0.0
Bonferroni, exact binomial44.3 / 0.236.9 / 0.00.0 / 0.00.0 / 0.015.0 / 0.0

UCB inversion that scans thresholds cautious-first and stops at the first uncertified one is itself a fixed-sequence test; with an exact Clopper–Pearson bound it coincides with our selector on every draw.

All five procedures are valid under their stated assumptions, and exact fixed-sequence testing ties for the highest automation among those evaluated. This is an empirical comparison, not a universal dominance claim: a fixed sequence can miss safe candidates beyond its first non-rejection.

The gain here comes from an exact test for the binary loss and a pre-specified order with no grid-wide multiplicity penalty.

Q3. CLARITY AND NOTATION

"Clarity; the subscript in Eq. (6)."

Eq. (6) now defines both subscripts as randomness over the certification sample, with score, grid and budget held fixed. We shortened §2 and §5.3 and made the thread explicit:

Mechanism (Thm 1) → Identification (Thm 4) → Consequence for CRC (Prop 6) → Certification (Thm 7)

Remark 1 states exactly where revision enters: without it, the individual loss is monotone automatically.


CONCLUDING REMARKS

Reviewer 8sfJ's questions helped us sharpen both the attribution and the theoretical contribution.

In light of Proposition 6, the matched comparisons and the additional evidence on practical significance, we would appreciate a reassessment of the novelty and significance concerns, and we welcome further discussion of any step.

3/3

Reviewer_a1pCOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

(I don't quite understand this paper so my review and summarization could include inaccuracies)

The paper suggests that the "stop when confident" principle in reasoning model early stopping can be problematic, because the answer could flip, and an aggregated curve could hide this behavior. The authors then proposed a new type of risk for reducing such behavior during the calibration process. The empirical results show that the proposed method works better than LTT and CRC, while reducing the number of flipping answers significantly.

Critical strengths
  • The problem studied is important: Answers do get flipped after the model reaches a confident solution and could hurt the performance of early exiting algorithms

  • The decomposition of risk looks interesting and sensible.

  • I didn't go through the theorme carefully but the proposed approach seems to be backed by statistical gurantees.

Critical weaknesses
  • The paper is motivated by reasoning LLM, but it looks mostly like a pure conformal prediction paper, the entire Sec 3 and 4 are decoupled from anything related to reasoning models.

  • The experiments focus on short simple tasks with short reasoning horizon.

Questions for authors

The paper mentions multi-agent debate a couple of times and I don't understand how is it reelvant, can the authors elaborate more?

3 comments

We thank Reviewer a1pC for recognising the importance of the problem and the value of the risk decomposition. The revision makes the connection between iterative reasoning and the theory explicit. Changes are in red, and the old Theorem 6 (validity) is now Theorem 7.

ON THE SUMMARY: WHAT THE PAPER CLAIMS

We study the standard wrong-action rate, R(λ)=Pr⁡(act∧wrong)R(\lambda)=\Pr(\text{act}\wedge\text{wrong}); it is not a new risk. The paper does not introduce a method that beats LTT or suppresses answer flipping: it uses existing Learn-then-Test as the tool for certifying stopping policies when answers may change.

The contribution is to explain how revision changes risk and what aggregate reliability statistics cannot determine.

ONE TRAJECTORY

On MMLU-Pro question 2104 (Figure 1, panel 1), the deliberating agents are right at round 1, switch to a wrong answer at round 2, and become more confident throughout: 0.22→0.62→0.77→0.910.22\to0.62\to0.77\to0.91.

OperatorThresholdActs atOutcome
Lenientλ=0.20\lambda=0.20Round 1Right
More cautiousλ′=0.60\lambda'=0.60Round 2Wrong

More caution made this instance less safe; 229 instances across the six benchmarks share this pattern. Answer flips themselves are familiar. What the example shows is subtler: the more cautious operator does worse, while an aggregate curve over all instances need not show any increase in risk (Figure 1, panel 2, from a single-agent self-revision run).

The example motivates the three central results:

1. MECHANISM (Theorem 1). Raising the threshold changes risk by exactly B−F−DB-F-D: answers broken by waiting, minus answers repaired by waiting, minus wrong actions deferred away.

Caution is safe exactly when B≤F+DB\le F+D.

2. IDENTIFICATION (Theorem 4) AND ITS CONSEQUENCE (New Proposition 6). CRC's standard guarantee needs every instance's loss to be monotone in λ\lambda. Even the full risk and automation curves, score-process law and every round's accuracy cannot determine whether this holds.

Proposition 6, building on the counterexample in CRC's own Proposition 1 (Angelopoulos et al., 2024), shows the consequence: two systems with identical aggregate reliability statistics give the same CRC procedure expected risks of 0.195% and 1.408% against a target of 1%.

An exactly known monotone risk curve therefore cannot, in general, establish the validity of standard CRC.

3. CERTIFICATION (Theorem 7). With the score, grid and budget fixed independently of the i.i.d. certification sample, fixed-sequence testing gives

Pr⁡(R(λ^)≤β)≥1−γ\Pr(R(\hat\lambda)\le\beta)\ge1-\gamma

without monotonicity or a calibrated confidence score.


The paper therefore certifies the wrong-action risk of acting on revisable answers; it does not require those answers to stop flipping.

1/2

W1. CONNECTION BETWEEN THE THEORY AND REASONING MODELS

"The paper is motivated by reasoning LLMs, but Sections 3 and 4 are decoupled from anything related to reasoning models."

New Remark 1 makes the connection exact. If an instance's answer never changes, Y^t=Y^1\hat Y_t=\hat Y_1 for all tt, then its loss is

L(λ)=E1⋅1[max⁡tSt≥λ],L(\lambda)=E_1\cdot\mathbf 1\big[\max_t S_t\ge\lambda\big],

a fixed error times an indicator that is non-increasing in λ\lambda, so pointwise loss monotonicity holds automatically.

This is also why Section 3 needs Theorem 1. In single-shot prediction, raising the threshold can only make the system defer an instance it used to act on. In an iterative system it can also make the system act on a later, different answer; the BB and FF terms of Theorem 1 measure exactly this effect, which exists only because answers are revised.

The theory does not require a particular LLM architecture because its subject is answer revision, a central feature of many iterative reasoning, debate and self-correction systems. None of the 21,116 never-revised instances violates the premise (Table 25).

The expected-risk versus draw-wise distinction in Section 4 is a separate issue, which can arise with or without revision. Remark 1 sits in Section 3, where CRC's premise is introduced.

W2. TASK COMPLEXITY AND REASONING HORIZON

"The experiments focus on short, simple tasks with a short reasoning horizon."

AspectEvidence
TasksGPQA (graduate-level science) and MuSR (multi-step narrative reasoning) among six benchmarks; each round requests an explicit reasoning response, not only a final answer
Horizon (original study)Eight-round self-revision (§5.1, App. L): breakage 1.9% → 8.7% while the accuracy gain levels off (+0.5 points at T=2T=2, +2.2 at T=8T=8)
Horizon (added)Per-trajectory premise violations 1.1% → 4.2% (§5.1, Table 25); bootstrap interval for the increase in breakage, 6.8 points [5.3, 8.4]
Regenerations (added)Two fresh eight-round runs on the same 499 questions break 6.6% and 6.2% (8.0% originally); a conservative instruction does not remove the effect (Table 34)

These results establish the mechanism within the tested horizons; they do not replace experiments on much longer agentic workflows such as multi-step tool use.

The theory has no small-horizon restriction: Theorem 4 needs only T≥3T\ge3, and the gap between the two populations in Proposition 6(ii) widens as TT grows.

Q. RELEVANCE OF MULTI-AGENT DEBATE

"The paper mentions multi-agent debate a couple of times; how is it relevant?"

Debate is the main test bed because revision is explicit there: each round, every agent reads the previous-round answers and may change its own.

Peer interaction is not an assumption of the theory, which needs only a predictor that emits an answer and a score each round. The same mechanism appears in single-agent self-revision and in code self-revision on 591 HumanEval/MBPP problems, where correctness is decided by tests the model never sees.

In the matched 707-question study (Appendix N), single-agent self-revision breaks 3.5% of answers, our three-agent protocol 1.8%, and a disjoint three-agent protocol 5.2%, so peer conformity is not necessary for the effect.


CONCLUDING REMARKS

Reviewer a1pC's question on reasoning led directly to Remark 1. We would appreciate a reassessment in light of these clarifications, and we are glad to clarify any part of the argument further.

2/2

Dear Reviewer a1pC,

Thank you again for your valuable feedback. We have clarified the connection between iterative reasoning and our theoretical results and added further empirical evidence.

We would greatly appreciate your thoughts on whether these clarifications address your concerns. Please let us know if there are any remaining questions that we could help clarify.

Thank you for your time and consideration.

Reviewer_WVvVOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies iterative (could be a multi agent debate, code self-revision etc.) LLM systems, and when such a system should commit on a particular answer, keep deliberating or escalate to a human, under some budget constraints. The paper works on a fundamental observation that unlike direct prediction, being more cautious can lead to mispredictions caused by later turns overruling an already correct answer during revision or debate. Authors claim the standard monotonicity claim in crc can fail and propose formalization of stopping threshold directly with existing hypothesis testing procedure (LTT). This according to authors makes sure deployed threshold meets budget with high probability and has no requirement of monotonicity and makes no assumptions on the confidence scores.

Critical strengths
  1. Authors provide a fairly rounded methodology to counter against existing pointwise loss monotonicity requirements of crc treatment of how iterative llms can fail.
  2. Nice comprehensive set of experimental designs.
Critical weaknesses
  1. Their claimed observation of the phenomenon that as T increases so does B is actually fairly rare in practice, and it goes along with standard practice in test time scaling, where overthinking / answer-flipping is a well-known effect people routinely cap or verify around.. In their own experiments they have seen B is 6.6 to 15x smaller than D etc. The claim that accuracy saturates while the correct answer broken climbs from 1.9-8.7% is slightly misleading since its no net loss, if the accuracy is indeed saturating that means similar and if not more cases are wrong to right climbs a the same time. So the motivating effect is limited in practice (I am not saying that means we should not formalize it, but I am rather saying its not as bad as the current work is potraying it to be).

  2. The experimental section is very very weak and non-transparent. It is not very easy to follow nor is clear how the authors are are running the pipelines and several key design choices that seems to be driving the results are hidden from readers.

  3. a. The confidence signal seems to be verbalized prob distribution across answer labels. Moreover the paper is uniformly averaging them across haiku, r1 and qwen3. It is well established in practice that verbalized confidence signals from an llm is erroneous/hallucination prone, poorly calibrated, and averaging it across 3 very different models without any per-model calibration is hard to justify. It does not break validity of theorem 6 perhaps but definitely makes the automation numbers hard to interpret.

2.b. The central experimental object, how the per option scores are elicited, the exact prompts, how the authors are reweighing the iterative debate/revision etc are not shared. The main text never concretely defines these but handwaives. More indepth experimental design is necessary to explain how the iterative pipelines are run. More specifically, we dont know if the right-wrong flipping is an artifact of how these models are doing iterative reasoning or an artifact of conformity inducing prompts. Needs more ablations.

2.c. The experiments are just single generations, debates are run once for exampel and then cached. The redraws are some repartitioning over fixed set of generations only. As such its difficult to interpret any of these results (break rate, B, F, D etc. ) very strongly as they are just noisy estimates.

  1. At a high level the paper is very difficult to follow, not because of the material but because its presented in a dense controved fashion. For a contribution whose value is in usable recipe for the practitioners this type of exposition I feel needs a bit more work to make it more lucid for readers.

3.a. More on practical realism, my larger concern is whether and how this is usable. The method requires (algo 1) before a threshold is set, a declared budget, a confidence score fit into data splits, a candidate grid on a 3rd data split, a 4th labeled calibration set drawn iid from deployment distribution (may not happen in reality), then budget estimated. In practice many of these wont work, for example the score grid budget etc all are frozen, any change in model or prompt or shift in traffic or harness modification invalidates them and need fresh recollection/recalibration. We may need potentially hundreds or thousands of labeled examples to even certify anything. How is the budget even set in practice? Some insights here would be useful.

Questions for authors

see weaknesses

4 comments

We thank Reviewer WVvV for the detailed review and for concrete experimental suggestions, several of which became new analyses in the revision. Changes are in red, and the old Theorem 6 (validity) is now Theorem 7.

SUMMARY OF THE REVISION

ConcernWhat the revision addsWhere
W1: Rare; no net lossPer-instance premise violations separated from breakage and harmful pairs; why rarity does not settle the guaranteeTable 25, Prop. 6, Table 29
W2: Prompts and design hiddenEvery prompt verbatim; protocol overview; split constructionApp. P.1, Table 24, App. I.6
W2.a: Averaged confidencesFour stopping scores under the same certification procedureTable 31
W2.b: Conformity promptsNo-peer runs and a conservative-instruction runTable 34
W2.c: Single generationsTwo fresh regenerations; instance bootstrapTables 30, 34
W3: Four splits, labels, budgetOne-labelled-set recipe; label floors; budget semantics§5.3, Table 32

W1. PRACTICAL SIGNIFICANCE AND NET LOSS

"Rare in practice (B is 6.6–15× smaller than D), overthinking is known, and 1.9% → 8.7% broken is misleading since there is no net loss."

The point is not that answers flip. That more reasoning can overturn correct answers is known, and the paper does not claim it as new. Its point is a subtler one inside that fact: answers can flip while accuracy improves and the population risk curve looks nearly monotone, so the aggregate statistics an operator typically checks can conceal the failure of CRC's instance-level monotonicity assumption. The responses below separate these quantities.

Breakage is not net loss (§5.1). Repairs also increase, reaching 10.8% by T=8T=8 (Appendix L, Table 17). Our claim is that improved average accuracy does not establish pointwise loss monotonicity, not that additional reasoning lowers accuracy. The 1.9–8.7% statistic counts instances with any correct-to-wrong revision; BB instead counts cases where, for a given nested pair of stopping rules, the earlier policy acts correctly and the later one wrongly. New Table 25 separates these from the quantity CRC actually needs, whether an instance's own wrong-action loss is monotone in the threshold:

BenchmarkBroken (r→w)Own loss non-monotone (premise fails)Harmful pairs
MMLU-Pro2.4%1.80% (150 / 8,312)0.12%
LogiQA3.5%2.65%0.25%
ARC0.6%0.54%1.60%
BBH1.2%0.92%1.05%
MuSR3.7%3.04%0.89%
GPQA5.1%3.48%0.62%
Self-revision, T=8T=88.7%4.20% (1.1% at T=2T=2, §5.1)not computed

The first two numerical columns are shares of instances. The last column is a share of threshold pairs (λ,λ′)(\lambda,\lambda') whose risk rises, taken from the Figure 3 scan (local k-NN score, 10 splits, six grid spacings), so it is not directly comparable with the other two.

The pointwise premise fails on all six benchmarks, more often at longer horizons, and never among the 21,116 instances whose answer does not change (new Remark 1). The small ratio of BB to DD explains why caution usually reduces aggregate risk, as Theorem 1 predicts; it does not establish the premise required by standard CRC.

Beyond answer flipping. Overthinking is known (Kaya et al., 2019; Chen et al., 2024), and we now cite both. Theorem 1 gives the exact condition under which caution is safe, B≤F+DB\le F+D, and Theorem 4 shows that even exact aggregate reliability statistics cannot determine whether the pointwise premise holds.

New Proposition 6 makes the consequence concrete. Building on the CRC counterexample (Angelopoulos et al., 2024, Proposition 1), it constructs two populations with identical risk and automation curves, score-process laws and round-wise accuracies, on which CRC meets a 1% expected-risk budget in one case and exceeds it in the other: 0.195% versus 1.408%. This is a theoretical construction, not an observed LLM violation.

Practical size (Table 29). On real trajectories, changing only the coupling while preserving the evaluated aggregate summaries moves premise violations by an order of magnitude (MMLU-Pro: 1.15% → 12.9%), while CRC's expected risk stays below β\beta under every simulated coupling. The result therefore shows that aggregate diagnostics cannot settle the premise; it does not claim a large routine penalty on these benchmarks.

Practical significance in risk control is assessed relative to the permitted budget: for a system operating under a 0.1% wrong-action budget, an increase in risk from 0.1% to 0.2% would put the actual wrong-action rate at twice that budget.

The rarity of a structural violation is therefore not, by itself, a reason to disregard it, and Proposition 6 shows why the choice of certification procedure can matter even when the population risk curve is monotone.

1/3

W2. EXPERIMENTAL ROBUSTNESS AND TRANSPARENCY

"Weak, non-transparent experiments; design choices hidden."

The original Appendix G defines elicitation and aggregation, and the supplement provides the exact prompts (prompts/system_prompt.txt, prompts/user_prompt_builder.py) with the model manifest and runner, although these should have been easier to find. At every round each agent reports a probability distribution over the options; we uniformly average the normalized distributions and take the argmax as the answer and the maximum as the raw stopping score. Later-round prompts contain all agents' previous-round responses; there is no learned reweighting across rounds.

The revision prints every prompt verbatim in Appendix P.1 (deliberation, self-revision, vote elicitation, code self-revision and the conservative variant), adds a side-by-side protocol overview (Table 24), and gives the redraw study's split construction (Appendix I.6).

W2.a. AVERAGING THREE MODELS' VERBALIZED CONFIDENCES

We tested alternative stopping scores under the same certification procedure (Table 31):

Automation (%), ensemble answer fixedMMLU-ProLogiQAARCBBHMuSRGPQA
Uniform average (paper)82.367.536.082.743.262.1
Per-model isotonic, averaged81.768.826.484.133.158.0
Learned per-model weights82.269.442.084.837.557.5
Haiku 4.5 confidence alone79.660.934.982.137.953.3

Every row keeps the social argmax as the prediction and changes only the stopping score, so the Haiku row is a single-model score, not a single-model answering pipeline.

Better calibration does not consistently improve automation: on MuSR, per-model calibration lowers ECE from 0.149 to 0.049 while automation falls from 43.2% to 33.1%. Theorem 7 certifies each independently fixed score without requiring calibrated probabilities, and all variants also have pooled WA/β<0.91\beta<0.91.

W2.b. CONFORMITY-INDUCING PROMPTS

The main debate instruction does say "Be open to changing your opinion…", and §5 now quotes it. The original eight-round study (§5.1, Appendix L) uses single-agent self-revision with no peers and a neutral instruction ("You may keep or change your distribution"), and Appendix N adds single-agent runs alongside three-agent protocols.

We also regenerated the self-revision protocol with an instruction to change the answer only on finding a concrete error (Table 34):

Same 499 questions, T=8T=8BrokenChanges / roundAccuracy t=1→8t=1\to8Draws with upward step
Original generation8.0%4.7%78.0 → 79.484%
Regeneration 16.6%4.4%76.4 → 80.468%
Regeneration 26.2%4.9%78.0 → 81.886%
Conservative instruction15.2%8.4%78.4 → 77.697%

Breakage persists without peers and under the conservative instruction. Because that instruction raised rather than lowered the revision rate, this single run does not isolate the effect of the debate wording, and it does not show that such prompting generally increases breakage; it shows that the instruction did not remove the phenomenon, and that prompting alone cannot replace certification.

W2.c. SINGLE GENERATIONS

The 10 main splits reuse one cached generation; the revision identifies them as re-splits, not independent generation replicates (Appendix P).

The two fresh self-revision runs above reproduce the effect, and a question broken in one run is broken in another with probability 0.34, against a base rate of 0.069.

An instance bootstrap puts the 95% interval of BB (summed over adjacent pairs of the 100-quantile grid) away from zero on every benchmark, and the increase in broken answers from T=2T=2 to T=8T=8 is 6.8 points [5.3, 8.4] (Table 30).

The regenerations cover the self-revision setting; they do not establish generation stability for every three-agent result.

2/3

W3. PRESENTATION, CALIBRATION REQUIREMENTS, AND DEPLOYMENT

"Dense presentation; four splits, frozen components, many labels, budget setting."

Presentation. We shortened §2 and §5.3; the thread is:

Mechanism (Thm 1) → Identification (Thm 4) → Consequence for CRC (Prop 6) → Certification (Thm 7)

and Table 24 states which data each step uses.

Four splits are not required. Theorem 7 needs the score, grid and budget fixed independently of the i.i.d. certification sample. The extra splits support fitted scores and data-based grid and budget choices. With a raw score, a pre-specified grid and an operator-set β\beta, one independent labelled certification set suffices (§5.3, Table 32).

How many labels. Even with zero observed wrong actions, certifying a nontrivial acting policy requires

m≥⌈log⁡γlog⁡(1−β)⌉:m\ge\left\lceil\frac{\log\gamma}{\log(1-\beta)}\right\rceil:

22, 45, 114 and 230 labels for β=10,5,2,1%\beta=10,5,2,1\% at γ=0.1\gamma=0.1. These are feasibility floors, not guarantees of useful automation.

In a simulation that treats each benchmark as the population, m=200m=200 gives 82% automation on MMLU-Pro at β=10%\beta=10\% (oracle 89%), with exceedance at or below 9.7% across benchmarks. Table 32 reports 5% and 10% budgets; tighter budgets may need more labels for useful automation.

Setting the budget and updating the system. The operator specifies the tolerated rate of wrong autonomous actions among all incoming instances: β=1%\beta=1\% means population wrong-action risk at most 1%, certified with probability 1−γ1-\gamma over the calibration draw. Our relative choice β=κe^T\beta=\kappa\hat e_T only makes the budgets bind in the experiments.

A changed model, prompt or traffic mix requires renewed certification, as for any held-out estimate. An existing labelled set may be reused only if it remains representative of deployment and the revised pipeline was chosen independently of that set and of its previous certification outcomes; otherwise a fresh independent set is required.


CONCLUDING REMARKS

The new runs, the protocol overview and the one-labelled-set experiment were shaped by Reviewer WVvV's questions. We would appreciate a reassessment in light of these results, and we welcome any further questions on the experiments.

3/3

Dear Reviewer WVvV,

Thank you again for your detailed and constructive feedback. We have added several experiments and analyses in direct response to your concerns.

We would greatly appreciate your thoughts on whether the revision addresses your main concerns. If any questions or reservations remain, we would be happy to discuss them further.

Thank you for your time and consideration.

Comments on the paper

2 comments

We thank the reviewers for their careful assessment. Every change is marked in red in the revised PDF, and the main text still fits within 10 pages.

1. STRENGTHS RECOGNISED IN THE REVIEWS

The reviewers identified complementary strengths of the original submission:

StrengthReviewerIn the reviewer's words
An important, practically relevant problema1pC"The problem studied is important"
8sfJ"meaningful to describe this phenomenon using strict probabilistic language"
Meaningful theoretical contributions8sfJ"The decomposition in Theorem 1 clearly depicts this effect"; Theorem 4 "further explains why the overall performance usually fails to reflect the transformation of the underlying correctness"
a1pCthe decomposition of risk is "interesting and sensible"
Methodology and experimental breadthWVvV"a fairly rounded methodology"; "Nice comprehensive set of experimental designs"
8sfJ"The approach proposed by this paper is reasonable"

These assessments recognise the main elements of the original contribution: the importance of the problem, the theoretical insight and the breadth of the empirical study. The revision builds on them while addressing the concerns about novelty, practical significance, experimental robustness and deployment.

2. WHAT IS KNOWN, AND WHAT THE PAPER ADDS

While several ingredients are known, the paper establishes previously uncharacterised relationships between answer revision, aggregate reliability and the validity of risk control:

What is knownWhat the paper shows
More reasoning can overturn correct answers (overthinking)It does so while the population risk curve stays nearly monotone, so the curve an operator checks hides it (Figure 1; Tables 17 and 25)
Standard CRC needs monotone lossesEven an exactly known monotone risk curve cannot, in general, certify that premise (Theorem 4)
CRC can fail under non-monotone losses (CRC's Proposition 1)Identical aggregate summaries can accompany both valid and invalid CRC, so they cannot tell which case holds (Proposition 6)
Fixed-sequence LTT needs no monotonicityThe independence conditions under which it certifies fitted stopping policies, and its equivalence with exact UCB inversion (Theorem 7; Appendix P.3, Tables 26–27)

The distinction between population-level and instance-level monotonicity is easy to miss: one step of the published marginal-risk argument for early-exit CRC relies on it (Appendix B).

3. CENTRAL CONTRIBUTION

The original submission establishes sharp partial-identification bounds for the break mass (Proposition 3) and complete non-identifiability of trajectory-level monotonicity (Theorem 4): even exact knowledge of the risk and automation curves, score-process distribution and round-wise accuracies cannot, in general, determine whether individual losses are monotone.

The revision adds a concrete statistical consequence: two systems with identical aggregate summaries can differ in whether the same CRC procedure meets its expected-risk target (Proposition 6). Existing fixed-sequence Learn-then-Test then certifies wrong-action risk directly, without relying on monotonicity (Theorem 7).

The contribution is this identification analysis, not a new calibration framework.

4. THE MAIN CONCERNS, AND WHAT THE REVISION ADDS

ConcernWhat the revision adds
Novelty relative to CRC and early-exit risk control (8sfJ)Proposition 6, with attribution to the CRC counterexample it builds on; a direct comparison with CALM, Jazbec et al. and B-PAC (§6, App. B)
Practical significance; the effect is rare (WVvV, 8sfJ)Per-instance premise violations (Table 25); a coupling study holding observed aggregate summaries fixed (Table 29); bootstrap intervals (Table 30)
Experimental robustness and transparency (WVvV)Per-model stopping-score calibration (Table 31); two independent regenerations and a conservative-prompt run (Table 34); all prompts verbatim (Appendix P.1); protocol overview (Table 24)
Link to reasoning, horizon (a1pC)Remark 1: without answer revision the loss is monotone automatically; eight-round violation rates and regenerations
Usability and baselines (WVvV, 8sfJ)Like-for-like high-probability selectors (Tables 26–27); a one-labelled-set recipe (Table 32)

All new numbers are produced by scripts, and the supplement's verifier regenerates every table. The old Theorem 6 (validity) is now Theorem 7.

1/2

5. NOVELTY AND ATTRIBUTION

Proposition 6 establishes a mathematical non-identifiability result through an explicit probabilistic construction: it proves that two populations with identical aggregate reliability statistics can differ in whether the same CRC procedure meets its expected-risk target, and part (i) adds a general finite-sample upper bound. These results are established analytically; Table 28 illustrates the closed-form expressions.

The independent-loss construction in Proposition 6(ii) comes from the proof of CRC's Proposition 1 (Angelopoulos et al., 2024), also analysed by Aldirawi et al. (2026). Our addition is a second population on which CRC is valid, with the entire risk and automation curves, score-process law and round-wise accuracies unchanged.

At a 1% budget (p=0.015p=0.015, n=500n=500, T=20T=20), CRC's expected risk is 0.195% in one population and 1.408% in the other (Appendix P.4). Proposition 6(i) bounds the excess for every compatible coupling using the risk curve alone, and the construction attains this bound asymptotically as the horizon grows.

These are theoretical statements about raw CRC, not failures observed in LLM data; the loss-monotonised variant is valid by construction.

6. WHAT THE EXPERIMENTS ESTABLISH, AND WHAT THEY DO NOT

QuestionEvidenceScope
Does the premise fail in practice?Violations on all six benchmarks, 0.54–3.48% of instances; 1.1% → 4.2% from two to eight rounds (§5.1, Table 25)Measured directly
Can aggregate summaries detect it?MMLU-Pro violations move 1.15% → 12.9% with every evaluated risk and automation value fixed (Table 29)Identification, not an observed CRC failure; CRC stays within budget
Is it one generation or one prompt?Fresh regenerations break 6.6% and 6.2% (8.0% originally); a conservative instruction does not remove it (Table 34)Self-revision setting
Can it be deployed?One labelled set suffices (Table 32); exact UCB inversion equals exact-binomial LTT on every draw, while Holm, Bonferroni and Hoeffding–Bentkus automate less or the same (Tables 26–27)Benchmark-as-population simulation (Table 32); held-out deployment evaluation (Tables 26–27)

Figure 4(a) addresses the separate distinction between expected-risk and draw-wise control.

7. CONCLUDING REMARKS

The revised thread is:

Mechanism (Theorem 1) → Identification (Theorem 4) → Consequence for CRC (Proposition 6) → Certification (Theorem 7)

with the empirical scope of each claim stated explicitly.

The revision preserves and strengthens the contributions already recognised in the reviews: it extends the original identification results to a concrete consequence for CRC, adds evidence across reasoning protocols, and addresses the experimental and deployment concerns.

Together with the strengths of the original submission, these results provide a clearer basis for reassessing the paper's contribution and significance. We thank the reviewers and the AC again and are happy to answer further questions.

2/2

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.