- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper presents SpecRoute, a multi-step speculative framework for LLM agents. It uses a draft agent to execute future tool calls and allows the target model to verify the observations, reducing intermediate reasoning steps. A slack-aware scheduler coordinates draft and target execution on a shared GPU. Experiments on six benchmarks show 1.28–1.56× speedups over vLLM with comparable accuracy.
Critical strengths
1.The paper is well-written and easy to follow, with clear motivation: the challenges of multi-step speculative execution. 2.The evaluation demonstrate end-to-end improvements, achieving up to 1.56× speedup over vLLM.
Critical weaknesses
- Limited novelty. Most components of SpecRoute have already been explored in prior work, including multi-step speculative execution, semantic verification, and shared-GPU scheduling. In the Related Work section, the authors briefly describe existing approaches without clearly explaining the fundamental differences and challenges addressed by SpecRoute. I believe the paper will benefit from sufficient comparisons or evaluations against closely related work, such as DSP, DualSpec, and SPAgent. Furthermore, the Motivation section should explain why existing approaches cannot adequately address these challenges.
- Concerns about speculation accuracy and effectiveness. The target LLM directly judges whether to accept speculative observations. This raises the following concerns. First, the capability gap between the small drafter and the target model may lead to low acceptance rates on harder tasks when involving multiple steps. Second, LLM-based verification may introduce incorrect acceptance decisions due to hallucinations. However, the paper does not sufficiently evaluate this part. All six benchmarks focus on relatively simple information-retrieval tasks. It remains unclear whether the approach generalizes to more complex agentic workloads (e.g. coding, scientific benchmarks).
- Concerns about experimental results. Table 1 shows that SpecRoute even improves answer accuracy on some benchmarks, which is counterintuitive. The author didnot give a sufficient explanation. It is unclear whether these improvements are simply due to experimental variability. Moreover, the paper lacks a detailed evaluation of acceptance rates across different benchmarks and their impact on accuracy and speedup, which I consider one of the most important evaluations for the proposed method. Figure 5(b) is somewhat unclear, and the results for two benchmarks appear to be missing.
- Table 1 also only compares accuracy against vLLM. It is unclear whether SpecRoute achieves a better accuracy-efficiency trade-off than other baselines.
Questions for authors
Unless the concerns are fundamentally addressed, I am unlikely to change my rating.
- The authors should more explicitly discuss how SpecRoute differs from existing work and how it addresses the challenges identified in the Motivation part.
- I suggest strengthening the baseline comparisons. New baselines should including standard speculative decoding in vLLM, existing general speculative execution frameworks, DSP [1], and DualSpec [2]. At a minimum, the paper should clearly explain its qualitative contributions over existing work.
- I suggest evaluating more diverse and challenging workloads, such as coding agents or complex deep-research tasks. This would help identify whether increasing task difficulty reduces the speedup.
- More analysis of task accuracy and draft accuracy is needed. I think a more interesting report should be when and where speculation fails, how these failures affect accuracy and latency, where may the large model gives wrong verification, and how they influence the choice of speculation depth and program behavior.
- Table 1 should include the accuracy of other baselines and provide a clearer accuracy-efficiency comparison. The authors should also explain why speculative execution sometimes improves answer accuracy.
- Could the authors also provide a direct comparison with the standalone small model (2B) on these benchmarks? I am concerned that the smaller model alone may already achieve sufficient accuracy, which would weaken the contribution for the proposed method.
[1] Dynamic Speculative Agent Planning
[2] DualSpec: Accelerating Deep Research Agents via Dual-Process Action Speculation
Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts. The new results address several of my concerns.
However, two major concerns still remain.
Speculative draft correctness. My primary concern is the correctness of the speculative drafts generated by the 2B model. The authors only present its standalone end-to-end task accuracy. The reported acceptance rates do not directly establish draft correctness or verification reliability. I understand that time is limited, but I believe this issue deserves at least a clear explanation and discussion, as it is central to the proposed method.
Novelty. I consider feedback-driven drafting and shared-GPU scheduling primarily system design choices, and are relatively straightforward. The key potential contribution is observation-level verification. However, its differences from existing methods, such as SPAgent, is unclear. The key innovation of technical mechanisms should be sufficiently explained.
I am therefore inclined to maintain my original rating.