acceptodds
Withdrawn from ICLR 2027

StableEdge: Certifying Early Workflow Routes from Structured Agent Streams

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Agent workflows increasingly branch on structured model output while bytes are still arriving. At a partial JSON prefix, a route chosen by an ordinary predicate can change under a valid continuation, producing wrong or duplicate workflow dispatch. We formalize safe early admission as completion-invariant routing: a prefix may be acted upon only if every schema-valid completion selects the same ordered route. StableEdge decides this condition by computing a conservative route image over a bounded policy profile with required route discriminators, certifying singleton images, and binding each certificate to a grammar-attested stream contract, a digest-bound commit envelope, a durable ledger, and an idempotent receiver. On 1,616 matched final-valid traces from BFCL v4 and AgentDojo, StableEdge makes exactly one correct admission per trace with zero observed wrong or duplicate dispatches, reaching admission a median 1,340 ms of recorded generation time before the final message and certifying 19.9 percentage points more traces early than a safe closed-field policy. Byte-prefix law audits, trust negative controls, crash and concurrency conditions, a structural grid, and three native runtimes locate when this gain is available and when it collapses.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
LLM agents, streaming structured generation, safe tool execution, workflow routing

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_wRFVOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes StableEdge, a system that can determine the workflow route before the model finishes generating a structured response. It computes a conservative route image and issues a route certificate when only one candidate remains. A durable ledger and an idempotent receiver then prevent duplicate admissions and side effects during retries and recovery. The evaluation compares StableEdge with full buffering, closed-field routing, and eager prefix routing. It measures wrong and duplicate dispatches, early coverage rate, and recorded generation-time lead.

Critical strengths
  1. The paper addresses a practical problem: safely determining workflow routes before generation finishes.
  2. The system design is comprehensive. It covers early route certification, durable admission, retries, and crash recovery, with an idempotent receiver to prevent duplicate admissions and external effects.
Critical weaknesses
  1. The paper is difficult to follow and poorly organized. Many technical terms, including route image, route certification, and commit envelope, are introduced before they are clearly explained through concrete examples. The examples in the paper show when a route is selected but do not provide a complete walkthrough of how the route image is computed. A step-by-step example and pseudocode would improve clarity.
  2. The method has limited novelty, as it combines early routing from partial outputs with standard mechanisms for preventing duplicate dispatches. The paper does not establish a clear technical advance over existing mechanisms for early outcomes from partial outputs. For example, LMQL's final semantics in "Prompting Is Programming: A Query Language for Large Language Models" (Beurer-Kellner et al., 2023) identifies expression values that cannot change under continuation, and jump-forward decoding in "XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models" (Dong et al., 2025) uses grammar constraints to directly fill in text that is already determined by the generated prefix. The contribution of applying these ideas to workflow routing also needs stronger evidence of practical value.
  3. The paper lacks sufficient implementation details. While it describes enum-prefix filtering and possible predicate truth values, the complete implementation procedure remains unclear. It should explain how the parser state and schema constraints produce feasible candidates, how these candidates pass through the ordered predicates, and how the required-path profile is checked. These details are essential to the claimed safety guarantee.
  4. The evaluation lacks comparisons with schema-aware prefix routing and a setup using jump-forward decoding. In particular, when an enum prefix uniquely determines the remaining characters of a string field, jump-forward decoding can emit the remaining characters and closing delimiter without additional model forward passes, which would allow simple closed-field routing to trigger at the same decoding step as StableEdge. Without these comparisons, it is unclear what advantage StableEdge offers over existing approaches.
  5. The paper is missing an end-to-end execution evaluation. Earlier routing does not establish earlier tool execution when arguments are unfinished. The reported time leads exclude downstream execution, and routing gains and ledger recovery are mainly evaluated through separate metrics. Furthermore, the efficiency benchmark in Section 5.2.5 reports router-only latency (26 μs) while excluding the I/O overhead of durable write-ahead logging (WAL). The paper should compare end-to-end task completion latency under identical ledger and receiver settings and report the full admission overhead. This is especially important given the paired median gain of only about 55 ms over closed-field routing.
  6. Early certification relies on restrictive schema and field-ordering assumptions. It requires routing discriminators to follow the required-path profile and appear early in the serialized JSON stream. In many structured agent workflows, placing reasoning or chain-of-thought fields before the route or tool discriminator is necessary to preserve model accuracy, which collapses the early-routing window toward full buffering.
Questions for authors
  1. Could the authors provide formal pseudocode and a step-by-step walkthrough showing how the parser state, schema constraints, and required-path checks interact to construct and prune the route image across multi-field or nested predicates?
  2. If the constrained decoding engine uses jump-forward decoding (such as in XGrammar) to immediately emit deterministic enum suffixes and closing quotes without extra LLM forward passes, how much of StableEdge's 55 ms lead and 19.9 percentage point early-certification advantage over closed-field routing remains?
  3. Since downstream tools generally cannot execute until their arguments finish generating, what concrete preparation tasks can be completed within the roughly 55 ms median window gained over closed-field routing, and what is the resulting wall-clock reduction in end-to-end task completion time?
  4. Section 5.2.5 reports a p50 router update cost of 26 μs for router-only operations. What is the full per-admission latency when including digest computation, durable WAL persistence, and receiver coordination?
  5. How does the conservative route-image abstraction differ formally and algorithmically from abstract interpretation under LMQL's final semantics over constrained decoding masks?
Reviewer_LB22OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes StableEdge for safe early routing in agent workflows. It uses partial JSON output and schema constraints to compute a conservative set of possible routes. Early dispatch is allowed when only one route remains. The system also uses a durable ledger and an idempotent receiver to prevent duplicate execution during retries and crash recovery.

Critical strengths
  1. The paper gives a precise and intuitive safety criterion for early routing. Defining admission in terms of a singleton set of routes over all schema-valid completions is substantially clearer than relying on prefix stability, confidence thresholds, or fixed chunk debouncing.
  2. The paper recognizes that route correctness is not sufficient for reliable external effects. The digest-bound envelope, durable ledger and crash/concurrency audits provide thoughtful system treatment of retries and races.
Critical weaknesses
  1. The central latency result lacks detailed explanation and discussion. (1) It is not an end-to-end tool-execution result. The reported 1.34s median gain is the time from route certification to the final generated byte, and the paper explicitly excludes downstream tool execution. (2) The evaluation does not report when all required arguments become complete, when an effect becomes executable, or how much user-visible task latency is actually reduced. In realistic calls, the route may be fixed while a decisive argument remains unknown until near the end of generation; in that case, early route certification may provide little or no opportunity to execute the tool. The paper does not report the fraction of traces in this situation, nor the distribution of the intervals from route certification to argument-ready or effect-ready states. (3) The evaluation tests broad BFCL and AgentDojo categories but does not provide a complete per-tool inventory, tool invocation counts or argument-completion timing.
  2. The evaluation does not discuss how StableEdge relates to the existing early routing systems. The baselines are limited to naive approaches, such as full buffering, closed-field waiting and eager routing. They do not include complex approaches such as Conveyor’s partial tool execution, AsyncLM’s asynchronous function calling, speculative tool-calling systems, or argument-aware systems such as PASTE[1-4]. These works already demonstrate early or overlapping tool execution, including cases where arguments are only partially available. Without comparison to such baselines, it is difficult to determine whether StableEdge provides a practically important advantage, or whether its contribution is mainly a narrower safety certificate layered on top of an already established execution paradigm. The novelty claim therefore appears overstated unless it is restricted to the specific combination of completion-invariant route certification and durable admission.

[1] Xu, Yechen, et al. "Conveyor: Efficient tool-aware LLM serving with tool partial execution." arXiv preprint arXiv:2406.00059 (2024).

[2 ]Gim, In, Seung-seob Lee, and Lin Zhong. "Asynchronous llm function calling." arXiv preprint arXiv:2412.07017 (2024).

[3] Hooper, Coleman, et al. "Speculative Interaction Agents: Building Real-Time Agents with Asynchronous I/O and Speculative Tool Calling." arXiv preprint arXiv:2605.13360 (2026).

[4] Sui, Yifan, et al. "Parallelizing Tool Execution and LLM Generation for Low-Latency Agent Serving." arXiv preprint arXiv:2603.18897 (2026).

Questions for authors

The main evaluation uses a required finite-enum route field. Do any early routing decisions require analysis beyond filtering compatible enum values and mapping them to routes? The broader predicate tests mainly report correctness and early-certification coverage. Do they also establish the latency benefit for more complex routing predicates? How StableEdge relates to existing early routing systems such as Conveyor, AsyncLM, and PASTE?

Reviewer_RNUWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

StableEdge formalizes safe early routing as agreement among all schema-valid completions of a streamed JSON prefix, implements a conservative route-image evaluator, and connects singleton certificates to durable admission and an idempotent receiver. On 1,616 matched BFCL/AgentDojo trace instances, the paper reports zero wrong or duplicate dispatches, a median 1.34-second decision lead over full buffering, and a 19.9% early-coverage improvement over its closed-field baseline

Critical strengths
  1. The paper states a precise safety condition and its dependencies. Separating agreement over valid completions from decoder trust, durable admission, and receiver idempotency gives a useful execution contract and prevents the route certificate from being mistaken for a guarantee about unfinished arguments or arbitrary external effects.

  2. The evaluation examines several consequential failure boundaries. Matched stream replay, route-position/chunk sensitivity, misattestation controls, and crash/recovery experiments help explain both when early certification is useful and which system assumptions it requires.

Critical weaknesses

The evaluation does not establish an end-to-end workflow speedup. The reported 1.34-second improvement measures how much earlier a route is known, while downstream execution is excluded (Section 5.1). Because arguments may remain unfinished, earlier routing may offer limited usable work to overlap with generation. Furthermore, the paired median advantage over the safe closed-field baseline is only 55.4 ms (Section 5.2.1). An integrated workflow experiment measuring completion latency, including durable admission and downstream preparation, is needed to establish practical benefit.

Questions for authors

N/A

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.