acceptodds
Under review as a conference paper at ICLR 2027

ObliGate: Obligation-Gated Runtime Repair for LLM Workflow Optimization

Abstract

LLM workflow optimization has largely focused on selecting or constructing workflows before execution, while failures after partial execution are commonly handled by retrying or regenerating the failed output. We study how to transition an already-executed workflow from a failed state to a verified state while preserving valid computation. We formulate post-failure repair as a transactional state transition over a versioned artifact DAG: runtime evidence opens a typed obligation that restricts legal interventions; cloned root probes test candidate repairs; dependency-closed replay restores consistency; and verifier improvement determines promotion. On strict MBPP+ (378 tasks, three seeds), ObliGate reaches 80.95±0.70% raw pass@1 at 325 task-action tokens, reducing tokens by 50.5% relative to fixed execution. In controlled latent-root failures, it recovers 76.9% of actionable cases versus 28.1% for draft-only repair. Across 228 naturally occurring failures, ObliGate reaches 33.3% accepted recovery, close to full-artifact replay with root probing (33.8%) and above direct draft repair (31.6%) and full rerun (20.6%); dependency-closed replay lowers normalized unnecessary recomputation by 0.083 in paired contrasts. The same evidence-to-action interface is instantiated for code, mathematics, and QA under adapter-registered, single-writer workflows.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
LLM Workflow Optimization, Runtime Repair, Agentic Workflows, Verifier-Guided Repair, Dependency-Aware Execution

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_8MdvOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies repairing an LLM workflow after part of it has already executed. ObliGate records artifacts and their dependencies, uses runtime evidence to restrict possible interventions, probes candidate repair roots on cloned states, and replays the affected downstream computation. A repaired state is adopted if its score under the fixed verifier improves. The evaluation includes strict MBPP+, injected failures, and runtime-flagged checkpoints, as well as code, mathematics, and QA workflow comparisons. On strict MBPP+, the full method reports 80.95% raw pass@1 at 325 task-action tokens under the 1,800-token ceiling, with lower token use than fixed deep execution.

Critical strengths
  1. Avoiding a full restart after a localized failure is a useful objective. The dependency-closed replay mechanism gives a concrete way to reuse valid work while keeping the affected artifacts consistent. This is a more careful treatment of repair execution than simply retrying the final answer.

  2. The paper includes feedback-based repair controls and distinguishes controlled faults from naturally flagged cases. These comparisons allow readers to separate the benefit of ordinary draft correction from that of the broader repair mechanism.

Critical weaknesses
  1. The full method's added value is unclear in the natural-failure study. Full ObliGate and closed-draft replay both improve 76 of 228 cases, while their reported end-to-end token costs are approximately 19,549 and 7,935, respectively. The full method has a lower false-repair rate, but probing adds no aggregate recoveries. All improvements are draft-local. None of the 37 upstream-involving cases improves. These results support local draft replay more directly than they support the broader cross-artifact repair proposal.

  2. The other results offer limited evidence for the full method's advantage. On strict MBPP+ at the 1,800-token ceiling, Trace-Repair reaches 79.98% at 290 task-action tokens, close to the full method's 80.95% at 325. The cross-domain comparisons use different internal policies, including trace-only on MBPP, static-minimal on MATH, and Fixed-Deep on QA. Their scores do not establish the complete repair method's benefit across domains. With so many variants and tables, it takes unnecessary effort to tell which gains belong to the full method.

The writing in this paper omits context needed for human readers, while creating unnecessarily long, repetitive basic claims and presenting long lists of experimental numbers with little interpretation.

Questions for authors
  1. How should a user choose between the full method and closed-draft replay, given the equal recovery count and the full method's lower false-repair rate but higher cost? Which naturally occurring failures actually require probing?

  2. What explains the gap between the controlled-fault results and the lack of improvement on natural upstream failures? Are the failure types different, or do the adapters lack the necessary interventions?

  3. How does the complete repair method compare with Trace-Repair on mathematics and QA, using the same starting workflow and matched resource budgets? Table 7 uses different internal policies across datasets.

  4. For a new workflow, which interventions and replay plans must be specified by hand? How much of the existing adapters can be reused?

Reviewer_2QYiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ObliGate formulates post-failure LLM workflow repair as a transaction over a versioned artifact DAG, combining evidence-based action authorization, isolated candidate testing, dependency-closed replay, and verifier-gated promotion. The paper provides sufficient conditions for dependency consistency and evaluates the framework through three-seed MBPP+ experiments, controlled faults, and 228 natural failures, reporting modest accuracy gains over feedback-only repair and computation savings relative to fixed execution.

Critical strengths
  1. The framework provides a coherent execution contract connecting admissibility, artifact versions, isolated repair, and promotion, while explicitly distinguishing dependency consistency from semantic correctness. This is a useful systems formulation for making repairs inspectable and preserving unaffected computation.

  2. The paper reports unfavorable natural-failure outcomes and distinguishes correction-stage savings from end-to-end savings, which strengthens the credibility of its reporting.

Critical weaknesses

The complete policy’s practical advantage remains unclear on natural failures. ObliGate and closed-draft replay both improve 76/228 cases, but ObliGate uses 2.46× more tokens, and improves none of the 37 upstream-involving cases (Table 4). Its improvement over direct draft repair is only 1.75 percentage points, leaving the central benefit of broader intervention selection supported primarily by controlled faults.

Questions for authors

N/A

Reviewer_TjN7OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This article addresses a practical problem: restoring failed LLM workflows while preserving valid intermediate results. The scope of experimental evaluation is extensive, and the report is fully transparent about negative findings and limitations.

However, the expression of the article is difficult to understand because too many coined terms obscure the underlying mechanisms. This paper does not explicitly establish theoretical or methodological contributions beyond common engineering practices, and although it increases complexity and cost, the experiments provide insufficient evidence to demonstrate gains compared to simpler baselines

Critical strengths
  1. The problem is practically relevant. The paper studies how to recover from failures in LLM workflows without discarding valid intermediate computation. This is a useful direction, with clear potential for improving efficiency and reliability.

2.Comprehensive empirical evaluation. The paper includes controlled fault injection, naturally occurring failures, ablation studies, and detailed resource accounting. The transparent reporting of negative findings is appreciated, although the results do not convincingly establish the incremental value of the proposed framework

Critical weaknesses
  1. The idea is a recombination of known workflow-engineering practice, and the paper never pins down what is actually new.

Dependency tracking, replaying only downstream artifacts on a cloned checkpoint, gating promotion on a verifier, reserving budget before an action—none of this is new, and the paper says as much by citing Sagas and the existing agent-governance systems. The question is what the combination adds. The paper's answer is essentially “a contract,” but it is hard to point to a design decision here that a competent engineer, given the same requirements, would not have reached. The terminology (typed obligations, admissible action sets, dependency-consistent replay) makes the system read as more novel than it is, and it also obscures which component is responsible for the reported gains. I would have been more sympathetic to a paper that called this what it is—a well-specified execution contract with an audit trail—but the submission frames it as a new optimization method, and that framing is not supported by the content.

  1. The experiments do not isolate the contribution of the central mechanism from cheaper baselines.

On the controlled fault-injection matrix, ObliGate recovers 76.9% of actionable cases; Any-failure Replay recovers exactly the same. On the natural-failure factorial, ObliGate and Closed Draft Replay both improve 76 of 228 cases, and ObliGate gets there with about 2.5× the tokens (19,549 vs. 7,935). The authors' own summary is that probing “does not add aggregate recoveries in this corpus.” These are the two evaluations closest to the claimed use case, and in both the obligation-gated machinery is indistinguishable from a much simpler alternative. The smaller gains that do appear—+0.97 points over Trace-Repair at budget 1800, for instance—are of the same size as the 0.79-point spread across four reasonable selectors in Table 6, so I cannot tell whether they come from the gate, the ranker, or noise. The paper needs an ablation that holds the ranker fixed and turns the registry on and off. That is the comparison that would justify the mechanism, and it is missing.

Questions for authors
  1. What design decision here is not already implied by standard transactional execution, dependency tracking, incremental recomputation, and verification-gated updates? The paper cites Sagas and the existing agent-governance systems, so I can see the ingredients. What I cannot see is which specific choice—typed obligations, the registry’s hard legality filter, clone–probe–promotion, or the replay plan—would be different if one implemented the same requirements with those established practices, and why that difference matters for the reported results. A concrete example of a design decision that a competent engineer would not have reached from the standard practices would be more useful than a general novelty claim.

  2. Which comparison isolates the obligation gate from replay and repair? In the controlled fault matrix (Section 5.3 / Appendix C), ObliGate and Any-failure Replay reach the same 76.9% recovery. In Table 4, ObliGate and Closed Draft Replay improve the same 76/228 natural-failure cases, while ObliGate uses roughly 2.5× the tokens (19,549 vs. 7,935). Could the authors point to a comparison that holds the replay scope and repair policy fixed and varies only the gate—or explain which result is intended to show a benefit attributable specifically to obligation-gated selection rather than to replay or repair in general?

Reviewer_ayb5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a partially executed workflow as a graph of versioned artifacts. When a verifier detects a failure, the available evidence may limit which edits are allowed. The executor tests an edit on a separate copy, then recomputes the affected downstream steps, and only keeps the result if the verifier score improves. I think it is a good design to separate permission to test an edit from the evidence needed to finalize it.

Critical strengths

The strongest experiment runs competing repair policies starting from the same 228 natural failure checkpoints. With root probing enabled, replaying only affected downstream steps saved 5k tokens per case compared to full replay. This is a meaningful result.

The authors also note where probing did not help. They state that natural failures do not establish a sufficient gain from root probing, and typed triggers did not give a stable gain. This frankness is quite valuable.

Critical weaknesses

Because the variation is larger than the gain, Table 1 does not establish that ObliGate beats Trace-Repair. While there are large gains over Fixed-Deep and Value-Routed, they spent tokens unconditionally. Trace-Repair already has conditional invocation, which I would not credit to ObliGate.

On strict MBPP+, ObliGate averaged only 1.11–1.21 calls per task. Its token use of 282–337 was far below the limits of 1200 - 2400, so the budget wasn't nearly used. The natural-failure corpus was 74% draft-local. I think the authors should run at least one long-horizon workflow with many interdependent artifacts and naturally occurring upstream faults.

The paper was written very densely. The terms such as "reserved-correction contract,” “calibration identities," and "correction-exempt” make reading and understanding quite challenging.

Coming back to the overall paper, I found the execution layer of ObliGate leading to cost reduction, but the paper’s main claimed contribution, the obligation-gated repair policy, isn't strong enough as the experiments have indicated.

Questions for authors

Please address the weakness points above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.