acceptodds
Under review as a conference paper at ICLR 2027

Self-Calibrating Interruption of Agent Doom Loops via Risk-Controlled E-processes

Abstract

Language-model agents run in harness loops that call an API-served policy and execute tools, so every iteration is billed. Much of that cost buys nothing: agents fall into *doom loops*, spending long after success has become improbable. Turn caps, token budgets and repetition heuristics kill salvageable episodes, per-step judges add a model call per step, and activation probes need internals that an API does not expose. The target is undefined: the conditional success probability is a zero-drift martingale, so no rule can wait for it to decline. Because a controller inspects every step, fixed-sample calibration is either invalid, realizing a false-kill rate at a nominal , or crippled by a horizon union bound that cuts cost by only . Least recognized, an interruption policy censors its own labels: the episodes it kills are exactly those whose uninterrupted outcomes identify its risk. We present Circe, which defines a doom loop as a *value-stall* (low salvage probability with vanishing information yield per dollar) and certifies it with a test supermartingale over a log-derived stall indicator, bounding by the probability of ever halting a salvageable trajectory under adaptive inspection, with no horizon penalty and asymptotically optimal delay. Its central mechanism is *randomized reprieve*: a firing certificate halts with probability and otherwise releases the episode to natural termination with known propensity , restoring identification and yielding a doubly robust salvage-loss estimator with a time-uniform confidence sequence and self-calibration regret over deployment rounds. On software-engineering, terminal and web-navigation suites and a near-saturated tool-calling control with `glm-5.2`, Circe cuts cost at a pp pass@1 difference and false kills, using log-derived features and no model internals, extra calls, or GPUs. Under a mid-stream model and difficulty shift every tested baseline breaches its risk constraint by up to unnoticed, while scheduled reprieve caps violation at and flags it within two rounds at exploration spend.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agents, Agent Harnesses, Anytime-Valid Testing, Distribution-Free Risk Control, Cost-Aware Interruption, Off-Policy Evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_DxnBOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The authors propose CIRCLE, which addresses all three while staying black-box (no internals, logits or gradients), free at the margin (every feature is a by-product of harness logs) and self-calibrating (valid after it starts intervening). Randomizing the irreversible action is not a regularizer; it is what keeps the risk estimate a function of observable data once the controller is live.

Critical strengths

1.The authors express the experimental setup in detail. Experiments are pretty comprehensive and multi-facet.

  1. Strong mathematical analysis that makes the method loves more reliable.
Critical weaknesses

To be honest, I scanned all assigned papers before writing reviews, this paper has the best framing and figures, which is the only paper that leaves a positive impression on me. However, it was disappointing to see that the abstract was purely generated by AI, detected by GPTZero, which is just like ALL other assigned papers. I will never give positive assessments on papers like this: Abstract being the very core part of a paper, is purely generated by AI, I can image how much effort is put into the paper. Also, the authors say that “Large language models were used only to polish the language of the manuscript.” The code can not be totally hand-crafted.

Questions for authors
  1. The author said the problem arises from "agents fall into doom loops (re-running the same failing test, re-reading the same file, re-proposing a patch already rejected)", can this be solved or mitigated by Recursive Self-Improvement (RSI)?
Reviewer_e6owOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes CIRCE, a controller that uses agent logs to detect stalled execution, gates interruption with an e-process, and randomly reprieves some flagged runs to retain outcome labels for online calibration. Experiments across four task suites report 36.35% lower macro cost with a -0.38 percentage-point pass@1 difference and 0.52% pooled false kills.

Critical strengths
  1. Randomized reprieve addresses a consequential problem: interruption removes the outcome labels needed to audit the controller. Making this feedback problem explicit is a useful contribution.

  2. The evaluation includes cache-aware cost accounting, a negative-control suite, and component ablations, giving a meaningful view of the cost-quality trade-off.

Critical weaknesses
  1. The deployed calibration does not establish the main certificate assumption. Theorem 3 requires a history-conditional stall-rate bound on salvageable trajectories, whereas Section 3.4 estimates a pooled bound from successful calibration runs. A population-mean bound does not imply this conditional bound, and the delayed-breakthrough successes explicitly fall outside it. Moreover, Table 2 compares population FK with alpha, rather than reporting the conditional error among uninterrupted-success trajectories required to validate the stated guarantee.

  2. Reprieve does not clearly identify the counterfactual in Eq. (1). Algorithm 1 can compact, rewind, or escalate before a certificate fires; continuing from that modified state does not produce the untouched harness outcome. Eq. (4) also omits outcome changes on runs with reversible actions but no certificate. Thus the claimed unbiased estimator of full-controller salvage loss, and the subsequent calibration guarantees, need a different identification argument or a narrower estimand.

Questions for authors
  1. What justifies using the pooled calibration bound as a bound on every relevant conditional stall probability? Please report certificate crossings and false kills within the uninterrupted-success subset.

  2. Is a reprieved outcome generated from the original untouched checkpoint or from the already modified controller state? How does Eq. (4) account for reversible-only runs and support risk estimation for alternative configurations?

  3. Please identify the exact suite versions, task splits, and B7 implementation. After a reversible intervention, does branching replay execute fresh model/tool calls? These are clarification requests about the reported experiments.

  4. The writing and figures can be largely improved.

Reviewer_HMgLOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The authors observe that LLM agents have a failure mode where they can appear to be doing work, incurring large API cost, but aren't actually making progress towards task success. The authors propose a method to save cost by detecting such stalled progress using heuristic features extracted from the agent’s log, and early-exit when enough evidence about stalling is collected. They motivate their method by formally defining an “information budget” that is spent over the course of a trajectory, and also provide theorems about the type 1 and type 2 error rates of their method. The authors evaluate their method with GLM-5.2, and on software engineering, terminal, web navigation, and tool calling, and show overall good cost savings with little performance degradation against baselines.

Critical strengths
  • Addressing the problem of stalling progress in LLM agents is an important and timely problem.
  • For online detection of stalling, the approach of applying e-values based on feature frequencies is a reasonable framework.
  • The results seem promising on the tested domains, and there are four domains tested.
Critical weaknesses
  • The paper is very difficult to read. (This is part of why I marked my confidence as low.)
  • Theory in §3.2 proposes a definition capturing the amount of information that would be gained about the agent’s success, were it allowed to continue. It is not intuitively clear why information gain is relevant to task success.
  • Theory in §3.2 and §3.4 makes strong assumptions whose empirical validity remains unclear: assumption that there’s an upper bound on information yield for all time steps in §3.2, and assumption that there’s an upper bound for all time steps on the probability of stall in successful trajectories in §3.4.
  • There doesn’t seem to be a strong connection between the motivating definition of information yield and information budget in §3.2 and the actual algorithm implemented in §A.3 which is based on stalling features.
  • It is not clear that the features would generalize to other language models.

Note: I have not attempted to check the proofs.

Questions for authors

I'd like to better understand the need for a specialized method. Can you comment on why RL training didn't already learn to avoid this failure mode? Is there any way to fix RL training to avoid this failure mode (and perhaps others as a side effect)?

(It seems that the RL policy should condition on the same log used by this method, and perhaps also should be given the same heuristic features of the log, and that RL training should receive negative reward for each additional token, which should incentivize it to avoid useless work. In this setup, would a specialized method still be needed?)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.