acceptodds
Under review as a conference paper at ICLR 2027

ProSpect: Proactive Speculation for Lossless Acceleration of Multi-Agent Systems

Abstract

Multi-agent LLM systems are increasingly used for complex, long-horizon tasks, but their sequential, dependent calls create high end-to-end latency. Speculative execution reduces latency losslessly, by pre-executing likely future calls and committing only those the system would have made anyway. Extending it from a single agent to a multi-agent system is hard because a speculation must predict which agents are invoked next in an execution trace that is a DAG instead of a chain, and what each predicted agent receives. An agent starts early only if every incoming edge is predicted correctly. We present ProSpect, which separates these two predictions. Choosing the next agent is cast as a link prediction task over a graph of agents assembled from past traces and answered by a recurrence folded along the execution DAG; the Speculator LLM is left with the task of message generation alone for the predicted agent, and boosted further through retrieval-augmented in-context learning on Actor's past traces. The speculation budget between these two prediction heads is split between them adaptively, by the predictor's confidence. Across four multi-agent systems and spanning the Qwen, Gemma and DeepSeek LLM families, ProSpect saves up to of wall-clock time and outperforms the state of the art speculative acceleration method on every system, while also reducing token cost and compute flops.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Multi-Agent Systems, LLM Agents, Speculative Execution

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_yB31OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ProSpect. It expands lossless speculative executions to multi-agent system. ProSpect separates routing modules and context generation. It uses DAG-aware predictor to choose the next agent, and use Speculator LLM to generate message augmented by ICL. It also introduces adaptive speculation budget between these two heads. ProSpect achieve better performance compared to Speculative Actions across tested models and benchmarks.

Critical strengths
  1. This paper expand lossless speculative execution from single agent to multi-agent system.
  2. ProSpect isolates routing prediction from message generation. This decomposition is supported by an ablation experiment. It reduces the Speculator from 9B to 2B, and the results shows that content matching is substantially hurt while next-agent prediction remains nearly unchanged.
  3. The empirical results are strong and consistent across multiple multi-agent systems and model configurations.
Critical weaknesses
  1. The contribution of DAG-aware routing design is insufficiently ablated. The paper claims that DAG-aware component predictor as a key part of the ProSpect, but it didn't clearly isolate the DAG-aware components. Current experiments only show that the full predictor are very strong, while it combines several design choices. I think an ablation that isolates the effect of DAG recurrence would be very necessary for the paper's core contribution.
  2. The paper also introduce adaptive width. In the appendix, the paper ablates different speculation budgets to see how the performance will change. However, it does not compare adaptive width allocation with fixed allocation strategies under the same budget. This comparison would be critical to demonstrate the motivation of adaptive width.
  3. Related works and positioning should be more complete. There has been several works expanding sepculatve execution to multi-agent system orchestration, such as "Accelerating Multi-Agent Orchestration with Speculative Dispatching". Discussion of such works would be more helpful for better positioning this paper.

[1] Ye et al. Accelerating Multi-Agent Orchestration with Speculative Dispatching. GCASR, 2026.

Questions for authors

See weakness.

Reviewer_SXQ3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ProSpect, a method to reduce the latency of multi-agent systems through speculative execution. Instead of waiting for each agent to finish, ProSpect predicts which agents will be called next and what messages they will receive, allowing future calls to start early.

The method separates these predictions into two parts: a learned routing model predicts the next agents, while a smaller LLM generates their messages using examples from past executions. It also adjusts how many predictions to make based on confidence. The paper provides a theoretical analysis of lossless execution and latency reduction.

Experiments cover four multi-agent systems with different model combinations. ProSpect reports 17.25–37.86% latency savings based on replayed execution traces, while also measuring additional token and compute costs.

Critical strengths

Separating agent routing from message prediction is a useful idea. The learned routing model achieves much better prediction accuracy than the baseline, especially on scientific workflows.

The experiments cover multiple multi-agent systems and model combinations, with useful analysis of prediction accuracy, latency, and compute costs. The theoretical discussion of speculative execution on DAGs is also helpful.

Critical weaknesses

It is unclear whether the implementation fully supports the claimed losslessness guarantee. The theory requires exact input matching, but the released code appears to allow semantic matching and reuse of predicted history, which may change downstream agent outputs.

The reported latency savings may be overestimated. The evaluation appears to count speculative time savings without fully accounting for when inputs are confirmed and how long the next call takes. It is also unclear which routing and budget allocation settings were used for the main results.

Questions for authors

Can the authors clarify which implementation and settings produced Tables 2–3? Do the reported gains still hold with exact input matching and correct accounting of confirmation time?

Can the authors provide a multi-agent example with multiple parent dependencies and compare adaptive budget allocation against simpler fixed allocation methods under the same compute budget?

Reviewer_YiiBOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

ProSpect proposes speculative execution to reduce the latency of multi-agent systems. It separates prediction into two tasks: a graph-based predictor identifies the next components to invoke, and a smaller language model generates their input messages using retrieved examples from previous executions. Experiments replay recorded Actor traces from four multi-agent systems. Across several model pairings, the paper reports approximately 17–38% latency reduction over non-speculative execution and generally lower speculative overhead than the Speculative Actions baseline.

Critical strengths

The paper addresses a practical and important problem: reducing the latency of multi-agent systems while preserving their behavior. Separating component prediction from message generation is a reasonable approach.

Critical weaknesses
  1. Poor presentation. The paper’s structure and formatting make it difficult to read. Notation is repetitive and sometimes inconsistent, and too many definitions interrupt the narrative. Some equations and proofs should be moved to the appendix so the main text can explain the method more clearly. Absolute statements such as “outperforms … on every system” should be qualified as “on all evaluated systems.”
  2. Unclear handling of multiple-parent dependencies. The paper does not clearly explain how candidate messages from multiple parents are combined. Correct predictions on individual edges do not guarantee that the correct combined invocation was launched.
  3. Unclear applicability of the losslessness guarantee to stateful tools. Discarding a speculative output does not undo changes to files, browser state, or external systems. The paper should clarify how speculative branches are isolated or rolled back and which tool operations satisfy its assumptions.
Questions for authors
  1. How are candidate messages from multiple parents combined?
  2. How are speculative tool calls isolated or rolled back to preserve losslessness when they modify shared state?
  3. What does Hit rate mean and how the author measure content match in Table 2?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.