acceptodds
Under review as a conference paper at ICLR 2027

SAGE: Lookahead Action Gating for Safe, Budget-Aware Edge–Cloud AI Agents

Abstract

Autonomous AI agents execute requests through multi-step trajectories in which each action can be generated by a fast local edge-based small language model (SLM) or a stronger cloud large language model (LLM). Existing hybrid approaches primarily select models to ensure the same output quality to stronger LLM in each step or preserve trajectories from LLMs, without jointly considering the recoverability of an SLM's action in the future trajectory and its long-term impact on end-to-end latency and cloud cost. An imperfect SLM action may remain recoverable by future LLM actions, while a seemingly faster SLM step may induce additional steps and ultimately incur higher end-to-end latency than escalating to the LLM immediately. Moreover, using LLM now may leave insufficient budget for more important future escalations. To address the problems, we present a lookahead Safe and budget-aware Action Gating system for Edge-cloud agents (SAGE), that accounts for these long-term effects on the future trajectory under end-to-end latency and cloud-cost budgets. First, it estimates whether accepting SLM preserves the agent's ability to successfully complete the request. Second, it jointly considers this safety estimate, the remaining budgets, and anticipated future resource needs to decide whether to execute the local action or escalate to the cloud LLM. Across two SLM/LLM pairs and six benchmarks, SAGE preserves 96.87–98.06% of LLM-Only accuracy while reducing normalized cloud cost to 0.41–0.45, and reducing mean latency by 25.9–28.6%. Compared with a state-of-the-art method, SAGE improves accuracy by 3.21–3.26 percentage points, reduces mean latency by 35.6%, and reduces normalized cloud cost by 37.5–40.6%.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
LLM Agents, Small Language Models, Dynamic Model Selection, Edge AI, Efficient AI Inference

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_QyBXOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents SAGE (Lookahead Safe and Budget-Aware Action Gating for Edge-Cloud Agents), a hybrid routing framework designed to optimize multi-step AI agent trajectories by dynamically deciding whether to execute an action using a fast, local Edge Small Language Model (SLM) or escalate to a powerful Cloud Large Language Model (LLM). Its main contributions include formulating hybrid agent routing as an action-level safe acceptance problem under end-to-end latency and cloud-cost budgets, combining model confidence, stochastic sample agreement, deterministic runtime checks, and an impact model into a calibrated safe-accept probability threshold that accounts for future trajectory recoverability and reserved cloud budgets. SAGE was evaluated across six benchmarks (HotPotQA, GSM8K, DROP, HumanEval, WebShop, MATH) using two SLM/LLM pairs (Qwen3-4B/GPT-5 and Gemma 3 4B/Claude Sonnet 4.5), demonstrating that it preserves 96.87–98.06% of LLM-Only accuracy while reducing normalized cloud cost to 0.41–0.45 and mean latency by 25.9–28.6%, significantly outperforming state-of-the-art baselines like Router-R1.

Critical strengths

(1) The main conceptual strength of the paper is moving beyond traditional single-step routing, which evaluates model choices solely on immediate confidence or output similarity. By recognizing that an imperfect local action can remain recoverable through future cloud steps, SAGE formulates gating around trajectory-level safety, impact scoring, and resource reservation for upcoming critical steps. This lookahead framing directly addresses the multi-step nature of agentic tasks and fundamentally alters how edge–cloud model collaboration is approached.

(2) The paper demonstrates strong empirical validation across six benchmarks and two distinct edge/cloud model pairs. SAGE retains 96.87–98.06% of LLM-Only accuracy while reducing normalized cloud cost to 0.41–0.45 and mean latency by 25.9–28.6%. Crucially, it decisively outperforms competitive SOTA baselines like Router-R1 by delivering higher accuracy (+3.21–3.26 percentage points) along with significantly lower latency (~35.6%) and lower cloud spend (~37.5–40.6%), proving its practical efficiency for real-world agent deployments.

Critical weaknesses

(1) SAGE relies on generating KK additional stochastic local drafts and evaluating token-level confidence metrics at every step to compute safety estimates. This multi-draft probing mechanism introduces non-trivial local computation and memory overhead on edge devices. The evaluation lacks rigorous comparisons against equal-budget or simpler edge-cloud baselines—such as direct local retries, single-pass speculative decoding, or lightweight cascading heuristics—making it difficult to determine whether the performance gains stem from the lookahead policy itself or simply from spending extra local compute during the probing phase.

(2) The framework's runtime verifier depends heavily on domain-tailored adapters and eight manually crafted deterministic signals, including symbolic state transitions and explicit tracking of unresolved objective sets (UtU_t). While structured benchmarks provide clear schemas for state tracking, unstructured and open-ended real-world environments rarely allow for clean symbolic modeling. This tight coupling to hand-engineered domain logic raises concerns about SAGE's generalizability and plug-and-play scalability across unseen tools and complex real-world workflows.

Questions for authors

(1) Clarification RequestsEqual-Compute Baseline Comparison: In the local probing phase, SAGE generates KK stochastic drafts (default K=3K=3) and computes token-level log probabilities. If baseline methods (e.g., Hybrid or Router-R1) are granted the same compute or latency budget—such as allowing KK local retries, single-pass speculative decoding, or self-correction turns—does SAGE still maintain a statistically significant Pareto advantage in accuracy, cost, and latency?

(2) SAGE’s runtime verification relies on domain-tailored adapters, deterministic schema checks, and symbolic state transition functions to compute unresolved objectives (Ut(a)U_t^{(a)}). How does SAGE perform on entirely unseen domains or dynamic tool APIs where no hand-engineered adapter or symbolic state transition rules exist?

(3) The empirical evaluation was conducted on a high-end host featuring an NVIDIA RTX 5090 GPU. On resource-constrained edge hardware (e.g., embedded NPUs or mobile chips), does the compute overhead of generating K=3K=3 verification samples locally negate the latency saved by avoiding cloud escalation?

Reviewer_ysq8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces SAGE, an action-level routing method for edge-cloud AI agents. By estimating how a local action generated by an on-device small language model affects the recoverability of the remaining execution trajectory, and by incorporating factors such as action risk, latency budgets, and cloud-cost constraints, SAGE dynamically decides whether to execute the local action or escalate to a cloud-based LLM. Experimental results show that this approach achieves accuracy close to that of a cloud-only model while reducing end-to-end latency and cloud invocation costs.

Critical strengths
  1. SAGE accounts for the impact of local actions on the recoverability of the remaining execution trajectory, which aligns well with the practical execution scenarios of multi-step agents.

  2. It outperforms the adapted Router-R1 baseline across two different model pairs, achieving improvements in accuracy, latency, and cloud costs.

Critical weaknesses
  1. The training labels are derived from replay trajectories where a cloud-based LLM is used for all subsequent steps after the local action. This creates a discrepancy with the actual dynamic routing policy, which may affect the reliability of the predicted safety probabilities.

  2. The estimated number of future critical escalations depends on the statistical distribution of the training trajectories. The paper does not fully validate the robustness of this budget-reservation mechanism under distribution shifts.

  3. The proposed method involves multiple prediction and decision-making modules, yet it lacks a clear, high-level architecture diagram. Including a flowchart illustrating the entire process from action generation and verification to budget assessment and final routing decisions would improve the paper's readability and clarity.

Questions for authors
  1. Does the safety predictor remain well calibrated when the continuation policy changes from cloud-only execution to the actual dynamic hybrid routing policy?

  2. Can the budget-reservation mechanism effectively prevent budget exhaustion when the number of critical escalations at test time exceeds the estimates derived from the training trajectories?

Reviewer_bFtJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SAGE decides whether to accept an already-generated local SLM action or escalate a multi-step agent to a cloud LLM. It combines sampling agreement, token confidence, model self-assessment, and deterministic runtime checks in an XGBoost predictor with isotonic calibration. Its safe-accept labels force the local action and then continue exclusively with the cloud model under the budgets. An impact predictor, future-escalation reserves, and estimated escalation burden determine a dynamic acceptance threshold. Across six benchmarks and two model pairs, reported macro-average accuracy retains 96.87–98.06% of LLM-Only accuracy, normalized cloud spend is 0.41–0.45, and mean latency decreases by 25.9–28.6%.

Critical strengths
  1. The routing unit fits multi-step agents well. Evaluating an actual local action exposes arguments, tool availability, loops, and side effects that a request-only router cannot inspect. The paper explicitly defines safe acceptance relative to a continuation policy and budgets, giving the predictor an interpretable target (Sections 3.1–3.4). The more specific contribution is the integration of realized-action conditioning, reference cloud continuation, and latency/spending reserves.
  2. The reported quality/latency/cost tradeoff is useful. Across six benchmarks and two model pairs, SAGE improves over Router-R1 by 3.26 and 3.21 accuracy points while reducing both mean latency and cloud spend. Including local drafting, verification, network, and tool time in latency is appropriate. Component ablations offer additional evidence for the controller's design (Tables 3–7).
Critical weaknesses

1.The main experiments do not report the outcomes needed to evaluate the central request-level budget objective. Sections 3.1–3.3 already deduct probing time, reserve resources, check cloud-escalation feasibility, and define SafeFallback. Pilot Tables 1–2 report latency-SLO satisfaction under their own SLO setting, but the main experiments instead use an LLM-Only median-latency budget and a 0.50 normalized-spend budget. Tables 5–6 give mean latency and aggregate normalized spend, not per-request latency/cost compliance, within-budget success, or fallback rates under those main settings. Consequently, the reported averages do not establish how often the request-level constraints are met. 2. The main evaluation and the attribution of action-derived information remain incompletely specified. Section 3.4 gives request-level splits and predictor settings, and Section 4 states that prompts, step caps, and stopping criteria are fixed across methods. However, the manuscript does not provide the benchmark-specific evaluated counts/splits, concrete prompts/adapters and cap values, the EM-versus-F1 choice behind the HotPotQA/DROP table entries, or repeated-run uncertainty. Equation (8) also needs an operational account of a corrective remote step and failure within h=2, since the ordinary reference continuation is already cloud-only. For contribution attribution, Section 2 already distinguishes SAGE from MTRouter, and Table 7 contains useful component ablations. These do not fully isolate the value of realized-action-derived features relative to a history/state-only estimator trained on the same replay labels.

Questions for authors
  1. Please report, from the default main-experiment logs if available, request-level latency and cost compliance, within-budget task success, and fallback/termination rates. How is the 0.50 normalized-spend target mapped to each request's initial cost budget, and is the median-latency budget computed separately by benchmark? Equation (6) defines SafeFallback, but does a request terminate whenever probing has exhausted its latency budget, including the branch that would otherwise accept a valid local draft? How are tool-time overruns counted?
  2. Please identify evaluated counts/splits, exact prompts/adapters, numerical step caps, and the EM or F1 metric used for each HotPotQA/DROP entry. For Equation (8), what distinguishes the corrective remote step from the cloud-only reference continuation, and does failure within h=2 mean an explicit terminal failure, budget exhaustion, or lack of completion within two steps? Please also specify the held-out samples and binning procedure used to compute the reference-target ECE.
  3. Is a history/state-only predictor available using the same replay labels and comparable capacity/training resources, with the realized-action-derived features removed? Alternatively, please explain how the existing ablations isolate that information. Comparisons at similar actual spend/latency and repeated-run intervals would help assess the additional benefit.
Reviewer_9uoSOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes SAGE, a controller that routes individual agent actions between a local small language model (SLM) and a cloud large language model (LLM). SAGE combines model confidence, repeated-sampling agreement, runtime checks, and execution-state features to estimate whether accepting an SLM action permits successful task completion. Experiments cover six benchmarks and two model pairs. The paper reports retaining approximately 97–98% of LLM-only accuracy while reducing cloud spending and mean latency, supported by component ablations and sensitivity analyses.

Critical strengths
  1. The paper addresses a practical routing problem. An action’s immediate quality or latency does not fully capture its effect on eventual task success and resource consumption. Considering recoverability and future resource needs is a useful motivation for routing within multi-step agent trajectories.

  2. The reported evaluation covers several task types. The breadth of benchmarks and model pairs could provide useful evidence, although the reporting limitations below substantially reduce its persuasiveness.

Critical weaknesses
  1. The experimental reporting contains inconsistencies and missing details that undermine confidence in the conclusions. Here are some examples. See more in Questions For Authors section. Figure 1(b)’s blue HotPotQA bar appears above 60%, whereas the paper report 53.76%. Its vertical-axis label, “Success Portion,” also does not clearly identify the local-assignment metric described in the text. Figure 1(a) presents bars with unexplained error bars, while the accompanying discussion claims overlapping distributions. Without sample counts and definitions of the displayed statistics, the evidence does not adequately support the stated conclusion about the limitations of local similarity. Figure 3(b) similarly omits the number of LLM calls and the procedure used to compare equivalent workloads.
Questions for authors
  1. How to estimate whether accepting SLM preserves the agent's ability to successfully complete the request?

  2. Why Hybrid-R doesn't equal to SLM-only or LLM-only if all steps use one model selected by HybridLLM for each request at the beginning?

  3. How are the normalized step index, step_ratio, progress deciles, and the early/middle/late categories computed without knowing the completed trajectory length? Are these quantities based exclusively on information available at decision time? How does the definition accommodate tasks requiring only one or two steps versus substantially longer trajectories?

  4. Please reconcile Figure 1(b)’s bar heights with the reported 53.76% and 62.76% local-step ratios, and define the numerator and denominator of “Success Portion.” For Figure 1(a), what statistics and error bars are plotted, and how many FP/FN examples contribute?

  5. Figure 3(b)’s comparison protocol. For each category requiring one, two, or three SLM subtasks, how many LLM calls contribute to the corresponding LLM bar? Are both models completing the same underlying workload to the same success criterion?

  6. Experimental protocol and consistency: Please provide benchmark sample counts, split provenance, repeated-run uncertainty, and the concrete baseline and execution configurations. Please also explain the substantial difference between the pilot and main LLM-only results—for example, HotPotQA accuracy is 45.1% in Table 1 versus 78.62% in Table 3. Are different subsets, prompts, metrics, or execution protocols responsible?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.