- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper introduces RectifyFlow, an edge–cloud collaboration framework for tool-using agents that aims to significantly reduce cloud API cost while preserving most of the quality of cloud-only execution. The key ideas are Dual-View Plan Construction, which builds a dependency-aware DAG and a linear execution view with explicit output references, grounding specifications, and execution criteria to guide a small edge model, and Dependency-Aware Plan Rectification, which localizes deviations via backward tracing and selectively repairs only affected subgraphs (plus optional state rollbacks). Experiments on BFCL and ToolSandbox show RectifyFlow achieves around 92–95% of cloud-only quality with roughly 2.+× cost reduction and fewer cloud calls than nine representative baselines.
Critical strengths
Clear formulation of a dual-view plan that separates a DAG dependency view (for selective rectification) from a linear execution view (for edge execution), with explicit mechanisms for output binding, argument grounding, and execution criteria. Dependency-aware rectification that performs backward localization to find a deviation source and forward impact analysis to bound the repair scope. The additional separation of plan edits and state-consistency repairs is a practical and well-motivated design for stateful tools. The framework explicitly targets the quality-cost frontier in tool-use scenarios, bridging plan-and-execute paradigms with edge or cloud collaboration mechanics. Broad comparisons against nine baselines (routing, hybrid planning, and plan-and-execute variants) across two widely used tool-use benchmarks, reporting both quality and cloud cost. Useful systems-level metrics beyond accuracy: latency, P95 latency, RTT, and communication, plus ablations on the two main components (DPC and DPR) and sensitivity to local recovery budgets. Generalization tests across multiple edge models and a second cloud LLM, suggesting the method’s benefits are architecture-agnostic. The pipeline is presented end-to-end with an overview figure, concrete node schema, and high-level pseudocode (Algorithms 1 and 2), making the components and execution flow understandable. The role of the binding map, execution criteria, and local recovery budget is clearly articulated, as is the rectification context passed to the cloud.
Critical weaknesses
The deviation localization and selective rectification steps rely on the cloud LLM’s qualitative reasoning rather than a principled, auditable algorithm; criteria for selecting the “earliest” deviation source are unspecified, which may affect stability and reproducibility. The semantics and feasibility of “state rollback” in state are under-specified. Many real tools have irreversible or non-idempotent effects, and the framework’s behavior in these cases is unclear. Parallelism is not discussed: the linear execution view may limit throughput in scenarios with independent branches, potentially sacrificing efficiency relative to DAG-parallel methods. Missing comparisons to recent dependency-aware planning or repair frameworks like ATG and SPIN that also use DAGs and minimal subgraph repair or validation, which are highly relevant for judging novelty and effectiveness of selective rectification. Limited analysis of failure modes: e.g., how often backward localization misattributes sources, frequency and cost of state rollbacks, and the distribution of rectification rounds. The evaluation relies on BFCL and ToolSandbox. Recent audits show evaluator brittleness and run-to-run variance on such tool-calling benchmarks. Only “mean over three runs” is reported, with no variance or adjudication measures. Some tables/figures appear with minor formatting artifacts and unlabelled axes in the extracted text. More precise numeric reporting for generalization (e.g., GPT-5.6-Terra) would aid interpretation. The sensitivity table mixes budgets awkwardly, and the “w/o Repair” line appears in a different sub-table context. Clearer separation of analyses would help. Beyond the plan-and-execute baselines included (ReWOO, Plan-and-Act), there is no direct experimental comparison to ATG’s minimal subgraph repair and SPIN’s validated DAG interface, both very close in spirit to the selective rectification and structured planning claims here. Recent works on device-cloud orchestration for agents (e.g., EdgeWise) and device-cloud multi-agent planners (e.g., EcoAgent) could be discussed more explicitly to situate the contribution among designs that reduce cloud interactions via planning and on-device verification.
Questions for authors
See weaknesses.
4 comments
Response to Reviewer 2NSW
W3. RectifyFlow vs. DAG-Based Parallel Execution
We further compare RectifyFlow with the parallel execution method LLMCompiler (Tab. 3). LLMCompiler achieves lower P95 latency but incurs substantially higher cloud overhead and lower task quality. One possible explanation is that parallel execution allows multiple branches to progress simultaneously, potentially accumulating errors before they are detected. This may require the cloud model to handle multiple errors and their downstream effects concurrently, increasing diagnosis and recovery costs and offsetting much of the expected speedup. Consequently, despite achieving lower P95 latency, LLMCompiler has slightly higher average latency (14.33 vs. 14.08 s/task), while incurring substantially higher cloud overhead and lower task quality. In contrast, RectifyFlow's sequential execution enables deviations to be detected and rectified before further propagation, yielding a more favorable quality–cost trade-off.
Table 3. RectifyFlow vs. LLMCompiler
| Method | Similarity ↑ | Avg. E2E (s/task) ↓ | P95 E2E (s/task) ↓ | Cloud Traffic (MiB/task) ↓ | API Cost ( USD/task) ↓ |
|---|---|---|---|---|---|
| LLMCompiler | 0.6146 | 14.33 | 35.77 | 0.0671 | 1.519 |
| RectifyFlow | 0.7188 | 14.08 | 42.06 | 0.0288 | 0.542 |
W4. Comparison of RectifyFlow with ATG and SPIN
We thank the reviewer for pointing out these relevant works. ATG and SPIN focus on improving agent planning and execution efficiency, whereas RectifyFlow targets cost-efficient edge–cloud collaboration. ATG reduces execution overhead through DAG-based task decomposition, parallel execution, and minimal subgraph repair. SPIN reduces redundant tool execution through DAG validation and simulator–critic-based early stopping.
More importantly, the underlying problem formulations differ. ATG and SPIN primarily model task and tool dependencies within an LLM-agent workflow, without explicitly considering heterogeneous edge–cloud collaboration. In contrast, RectifyFlow coordinates cloud-side planning and rectification with continuous edge-side execution, introducing cross-tier dependencies and interactions among models, tool operations, and execution states. Managing these dependencies while minimizing cloud inference costs is a central focus of our work.
Our initial baseline selection primarily focused on methods targeting edge–cloud collaboration and cost optimization. Nevertheless, given the methodological relevance of ATG and SPIN, we have added direct comparisons on ToolSandbox.
Table 4. RectifyFlow vs. ATG and SPIN
| Method | Avg. Perf. ↑ | Avg. E2E (s/task) ↓ | P95 E2E (s/task) ↓ | API Cost ( USD/task) ↓ | Traffic (MiB/task) ↓ |
|---|---|---|---|---|---|
| ATG | 0.4038 | 14.50 | 42.39 | 1.975 | 0.0526 |
| SPIN | 0.5261 | 22.49 | 49.94 | 2.057 | 0.1004 |
| RectifyFlow | 0.7188 | 14.08 | 42.06 | 0.542 | 0.0288 |
As shown in Tab. 4, RectifyFlow achieves the highest task performance (0.7188) and the lowest overhead across all reported metrics. These results demonstrate the advantage of jointly coordinating cross-tier execution and dependency-aware rectification to optimize task quality and cloud inference cost.
Response to Reviewer 2NSW
W5 Analysis of failure modes
We further analyze three aspects of RectifyFlow's failure modes and recovery overhead:
(1) Backward localization errors. As reported in W1, under the DeepSeek-Flash configuration, RectifyFlow achieves 91.23% accuracy in backward deviation localization, corresponding to an 8.77% localization error rate. This indicates that most deviation sources are correctly identified, providing a reliable basis for selective rectification.
(2) State rollback frequency and overhead. On ToolSandbox, state rollback is triggered 0.314 times per task on average, with an average cumulative time of 0.458 s/task. This accounts for 4.23% of average task latency, while its associated API cost accounts for 16.2% of RectifyFlow's average API cost, indicating relatively limited recovery overhead.
(3) Rectification round distribution. Tab. 5 reports the distribution of cloud repair rounds. Specifically, 54.93% of tasks require no repair, 39.41% require one round, and only 5.66% require two or more rounds. Thus, 94.34% of tasks complete with at most one repair round, suggesting that RectifyFlow generally avoids repeated rectification. Table 5. Cloud Repair Round Distribution on ToolSandbox
| Repair Rounds | Percentage |
|---|---|
| 0 | 54.93% |
| 1 | 39.41% |
| 2 | 5.16% |
| 3 | 0.25% |
| 4 | 0.25% |
| ≥5 | 0.00% |
W6. Reliability of the Evaluation
We thank the reviewer for highlighting the reliability of benchmark evaluation. We address this concern from two aspects:
(1) Evaluator reliability. We re-examined the evaluation results against recorded execution traces, achieving 96% evaluation accuracy. We further manually adjudicated discrepant cases and found that disagreements mainly stemmed from hallucinations by the ToolSandbox user-simulator LLM and scoring discrepancies caused by state and field matching in BFCL. After excluding these cases, RectifyFlow's advantages in task quality and overhead remain consistent, indicating that these evaluation discrepancies do not alter our overall conclusions.
(2) Run-to-run stability. Beyond reporting mean performance, we provide the variance across three repeated runs on both benchmarks (Tab. 6). The results show that RectifyFlow maintains reasonably stable performance across runs, with variances of 6.2500 × 10⁻⁴ on BFCL-V4 and 11.0217 × 10⁻⁴ on ToolSandbox, providing additional evidence of evaluation stability.
Table 6. Average Performance and Variance Across Three Runs
| Method | BFCL V4 Avg. Perf. ↑ (×10⁻²) | BFCL V4 Variance (×10⁻⁴) | ToolSandbox Avg. Perf. ↑ (×10⁻²) | ToolSandbox Variance (×10⁻⁴) |
|---|---|---|---|---|
| Cloud-only | 81.09 | 22.3668 | 70.65 | 10.8187 |
| Local-only | 6.88 | 4.4092 | 38.26 | 24.2544 |
| MiniCPM5-2B | 4.38 | 3.2467 | 38.20 | 16.6288 |
| GAPG | 45.63 | 14.0625 | 59.95 | 22.0897 |
| AIMS | 37.03 | 13.4881 | 53.90 | 3.1759 |
| Hera | 37.03 | 16.5901 | 58.78 | 13.6848 |
| DoT | 16.72 | 5.9461 | 66.70 | 10.1091 |
| Minions | 48.44 | 10.2592 | 44.59 | 13.3567 |
| HybridFlow | 53.13 | 5.3116 | 55.85 | 7.7719 |
| RectifyFlow | 76.72 | 6.2500 | 64.83 | 11.0217 |
W7. Figure Formatting and Axis Labels
We thank the reviewer for pointing out these presentation issues. We will correct the formatting artifacts in the relevant tables and figures and ensure that all figure axes are clearly labeled in the revised manuscript.
W8. Detailed Results for GPT-5.6-Terra
We provide detailed results for GPT-5.6-Terra on BFCL-V4 in Tab. 7, reporting both task performance and estimated API costs to enable a more precise comparison across methods. RectifyFlow retains 96.42% of Cloud-only performance while incurring only 62.75% of its API cost, further demonstrating its favorable quality–cost trade-off with GPT-5.6-Terra.
Table 7. performance on BFCL V4 with GPT-5.6-Terra
| Method | Approx. Avg. Perf. ↑ () | Est. API Cost ↓ ( USD/task) | vs. Cloud-only (Perf. ↑ / Cost ↓) |
|---|---|---|---|
| Cloud-only | 82.97 | 35.650 | 100.00% / 100.00% |
| Local-only | 5.00 | 0.000 | 6.03% / 0.00% |
| GAPG | 60.00 | 27.580 | 72.32% / 77.36% |
| AIMS | 44.06 | 26.182 | 53.11% / 73.44% |
| Hera | 54.06 | 27.120 | 65.16% / 76.07% |
| DoT | 29.06 | 29.544 | 35.03% / 82.87% |
| Minions | 39.06 | 25.275 | 47.08% / 70.90% |
| HybridFlow | 55.94 | 30.558 | 67.42% / 85.72% |
| RectifyFlow | 80.00 | 22.370 | 96.42% / 62.75% |
Response to Reviewer 2NSW
W9. Revision of Table Structure
We thank the reviewer for pointing out the ambiguity in the table organization. We will separate Table 4 into two independent tables: (1) sensitivity analysis on BFCL, reporting performance under different local recovery budgets (0–5), with "w/o Repair" relabeled as "No Local Recovery (Budget = 0)"; and (2) ablation study on ToolSandbox, comparing the full framework with variants without DPC and DPR. This separation clarifies the distinction between recovery-budget sensitivity and component contributions, making the experimental results easier to interpret and compare.
W10. Comparison with ATG, SPIN, EdgeWise, and EcoAgent
We thank the reviewer for highlighting these relevant works. As discussed in W4 (Comparison with ATG and SPIN), ATG and SPIN address different problem formulations and optimization objectives. EdgeWise was released after our submission deadline and primarily evaluates cloud delegation on coding and terminal-agent benchmarks. EcoAgent targets mobile GUI automation, where execution relies on screen observations and UI interactions, with efficiency gains primarily achieved through on-device verification and compressed visual feedback.
Although EcoAgent also optimizes the quality–cost trade-off in device–cloud collaboration, its task setting and primary sources of overhead differ from ours. RectifyFlow focuses on stateful API-based tool use, where structured tool outputs and state changes create interdependent operations, and errors may propagate to downstream steps, incurring additional cloud recovery costs. EcoAgent is not specifically designed or evaluated for this dependency-driven recovery setting, which informed our initial baseline selection. Nevertheless, we acknowledge the relevance of these works and have added direct comparisons with all four methods on ToolSandbox.
Table 8. RectifyFlow vs. All Baselines on ToolSandbox
| Method | Avg. Perf. ↑ | Avg. E2E (s/task) ↓ | P95 E2E (s/task) ↓ | API Cost ( USD/task) ↓ | Traffic (MiB/task) ↓ |
|---|---|---|---|---|---|
| ATG | 0.4038 | 14.50 | 42.39 | 1.975 | 0.0526 |
| SPIN | 0.5261 | 22.49 | 49.94 | 2.057 | 0.1004 |
| EdgeWise | 0.4782 | 21.36 | 56.31 | 0.969 | 0.0605 |
| EcoAgent | 0.5198 | 39.79 | 91.76 | 1.445 | 0.0323 |
| RectifyFlow | 0.7188 | 14.08 | 42.06 | 0.542 | 0.0288 |
As shown in Tab. 8, RectifyFlow achieves the highest task performance (0.7188) and the lowest API cost (0.542 × 10⁻³ USD/task), latency (14.08 s/task) among the compared methods, demonstrating its effectiveness in jointly maintaining execution quality and controlling cloud recovery overhead.
Response to Reviewer 2NSW
We thank the reviewer for the thoughtful and constructive feedback. Below, we comprehensively address the raised concerns through clarifications and additional experiments, and welcome further discussion. Please note that a recent update to the underlying DeepSeek API yields higher scores in some newly added experiments than those reported in the original manuscript, without changing our main conclusions.
W1. Auditability and Reliability of the RectifyFlow
We thank the reviewer for raising this important concern. RectifyFlow does not rely on unconstrained qualitative reasoning; instead, it adopts an evidence-constrained, graph-based localization procedure, where the LLM interprets execution evidence under explicit DAG constraints.
Specifically, RectifyFlow (1) identifies the interrupted node from runtime feedback, (2) traces its predecessors in the DAG to constrain candidate sources, (3) compares their planned behavior against actual outputs, execution history, and state changes, and (4) selects the earliest evidence-supported erroneous node, i.e., one without an earlier erroneous ancestor. The downstream impact scope is then determined through graph reachability.
The process is auditable: the DAG explicitly records node IDs, typed dependencies, output references, and execution conditions, while runtime bindings, interaction history, and interruption evidence are retained. Structured diagnosis records the identified source, supporting evidence, affected nodes, and rectification operations, making the reasoning process and resulting decisions inspectable.
Table 1. Deviation Source Localization Accuracy
Table 2. Rectification Performance of RectifyFlow
To evaluate reliability, we conduct ground-truth-based localization across three cloud models. As shown in Tab. 1, localization accuracy reaches 91.23%, 90.17%, and 94.43%, respectively, demonstrating cross-model robustness. Together with the post-rectification accuracy and rectification-time results reported in Tab. 2, these findings further demonstrate the effectiveness and reliability of RectifyFlow's localization and rectification mechanism.
W2. Clarifying the Semantics and Scope of State Rollback
We thank the reviewer for raising this important concern and prompting us to clarify the semantics of state rollback.
In RectifyFlow, rollback refers to restoring consistency in the states and results affected by executed tool operations. It includes correcting reversible states through compensating tool operations (e.g., toggling feature switches) and invalidating erroneous result bindings followed by re-querying. However, rollback does not imply transactional undo: its applicability depends on tool-specific recovery capabilities. It cannot reverse irreversible actions (e.g., sent emails) or guarantee safe re-execution of non-idempotent operations. We will clarify these semantics in the revised manuscript.
Furthermore, we inspected execution traces involving irreversible operations and found that 9% of evaluated samples involved erroneous irreversible actions. We emphasize that pre-execution safety and RectifyFlow's runtime rectification address two distinct yet complementary problems: the former aims to prevent irreversible actions before execution, whereas RectifyFlow focuses on localizing execution deviations and recovering from them. Recent studies, such as MOSAIC (ICML 2026) and StepGuard (EMNLP 2026), investigate pre-execution safeguards. These mechanisms can be integrated with RectifyFlow to enable safer execution.