- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DualSpec, a framework for reducing the inference latency of deep research agents. The authors observe that different tool actions have different reasoning requirements: Search actions benefit more from explicit reasoning, while Visit actions can often be generated directly by a larger model without reasoning. Based on this observation, DualSpec generates two candidate actions in parallel, using a small reasoning model and a large non-reasoning model. It then selects a candidate and uses the large model as a semantic verifier to decide whether to accept the action or fall back to full reasoning. The method is evaluated using MiroThinker and Qwen3 model pairs on three deep research benchmarks, reporting latency reductions while maintaining accuracy comparable to the base model.
Critical strengths
1. A useful observation about different agent actions. I think distinguishing the reasoning requirements of Search and Visit actions is a reasonable and practically motivated idea. The analysis suggests that applying the same inference strategy to every tool call may not be the most efficient choice.
2. Considering semantic verification instead of exact action matching. I appreciate the attempt to move beyond strict action matching, since different search queries can be semantically similar without being identical. Using a lightweight verification step to avoid unnecessary full reasoning is also a potentially useful direction for improving agent efficiency.
Critical weaknesses
1. Limited methodological novelty and questionable efficiency of the proposed design.
My main concern is that DualSpec does not seem to introduce a substantially new speculation mechanism. The framework mainly combines two existing ideas: generating candidate actions with different models and using a larger model to verify them. Similar model cascading and verification strategies have already been studied in previous work, such as SpecReason and other speculative agent frameworks.
More importantly, I find the actual efficiency of the design somewhat questionable. As described in Section 5.2, the system selects between the two drafts based on the action type and reasoning length produced by the small model. This means that even when the non-reasoning large-model draft is selected, the system still needs to wait for the small model to finish its reasoning before making that decision. I am not convinced that this is an efficient way to implement the proposed System 1 and System 2 distinction.
Also, unlike previous speculative agent approaches that overlap tool execution with model reasoning, DualSpec appears to execute tools only after the draft has been verified. In this sense, the framework looks more like a model routing and verification pipeline than speculative execution. I think the authors should better clarify the distinction and explain where the actual latency savings come from.
The theoretical analysis is also relatively weak. It largely relies on the standard observation that conditioning reduces uncertainty. Moreover, the lower token uncertainty of Visit actions may partly result from selecting URLs already present in the context, rather than reflecting a fundamentally different reasoning process. I do not find the theoretical contribution particularly convincing.
2. Insufficiently convincing experimental comparisons and unclear practical benefits.
Although the reported speedups are promising, I have several concerns about the fairness and interpretation of the results.
First, DualSpec uses two models running on separate GPUs, while the base model is evaluated on a single GPU. Since the main contribution is inference acceleration, I think a hardware-matched comparison is important. For example, the authors should compare against a base model deployed with the same GPU resources before concluding that the framework provides a substantial efficiency advantage.
Second, some of the simpler baselines already achieve competitive results. In Table 1, generating Visit actions with the large model without reasoning achieves 64.08% accuracy on GAIA, slightly higher than the original base model and the reported DualSpec result. In Table 2, LLM-direct is actually faster than DualSpec on Miro/GAIA, although with lower accuracy. On Miro/XBench, LLM-direct achieves 65% accuracy in 492 seconds, compared with DualSpec's 66% in 480 seconds. This is a rather small difference considering that DualSpec requires an additional model and verification procedure.
Finally, the cost results are not uniformly favorable. For example, Table 3 shows that DualSpec is approximately 11% more expensive than SPAgent for the MiroThinker 72B + 30B-A3B configuration under the authors' pricing assumptions. I think this limitation deserves more discussion.
Additional concerns about presentation.
The presentation needs improvement. Several figures are difficult to understand, particularly Figure 6, where the verification workflow is not clearly illustrated. There are also noticeable citation formatting and spacing issues throughout the paper.
I also found some important implementation details unclear. For example, the exact thresholds used for draft selection and verification are not reported, and the procedure for constructing the development split is insufficiently explained.
In addition, Section 5.3 describes a verifier that evaluates reasoning coherence and action usefulness, while the actual prompt in Appendix A.1 mainly checks whether the trajectory is progressing and includes a criterion about final-answer grounding. I find this inconsistency confusing, especially since the verifier operates on intermediate tool actions.
Questions for authors
-
Method design and actual speculation: Since the routing decision depends on the small model's generated action type and reasoning length, does DualSpec need to wait for the small model to complete its reasoning even when selecting the non-reasoning large-model draft? Also, how does the proposed framework differ from standard model cascading when tool execution happens only after verification?
-
Fairness of latency comparisons: Since DualSpec uses two GPUs while the base model uses one, could the authors provide comparisons under the same hardware budget, such as running the base model with two GPUs? How much of the speedup comes from the proposed method rather than additional hardware resources?
-
Effectiveness compared with simpler alternatives: Table 1 shows that using the large model without reasoning for Visit already achieves strong results, while Table 2 shows that LLM-direct performs competitively in both accuracy and latency. What additional benefit does the full DualSpec framework provide over these simpler configurations under matched resource budgets?
4 comments
W2 / Q2
Our main setup placed the two models on separate GPUs to model cloud API deployments such as DSP, where independent endpoints serve multi-batch traffic at stable throughput and are evaluated via latency and metered token cost in Table 3; DSP and SPAgent use the same two-GPU setup in Figure 7. For local single-GPU deployment, where interactive agents typically serve single-request (batch = 1) workloads, we co-located both models on one 80GB A100 GPU under batch = 1 on GAIA-Text-103 (added in Appendix D).
| Model pair | Deployment | GPUs | Mean latency (s/query) | Speedup vs. 1-GPU Base |
|---|---|---|---|---|
| MiroThinker 72B | Base | 1 | 1041 | 1.00× |
| MiroThinker 72B + 8B | DualSpec co-located | 1 | 623 | 1.67× |
| MiroThinker 72B + 8B | DualSpec separate endpoints | 2 | 605 | 1.72× |
| Qwen3 32B | Base | 1 | 80 | 1.00× |
| Qwen3 32B + 4B | DualSpec co-located | 1 | 48 | 1.67× |
| Qwen3 32B + 4B | DualSpec separate endpoints | 2 | 46 | 1.74× |
Single-GPU co-location retains a 1.67× speedup on both model pairs, closely matching separate endpoints. Together with the super-linear 2.6× average speedup on 72B + 30B-A3B and the 49.5%–66.6% token cost reduction in Table 3, this confirms that the gains come from removing roughly 80% of base-model reasoning tokens rather than extra hardware.
W2 / Q3
Table 1 is an offline diagnostic study with Oracle action-type knowledge (clarified in Section 3.1): full base-model reasoning first determines each step's action type before selectively replacing Visit steps. At actual inference time, the next action type is unknown prior to reasoning, so DualSpec runs the small reasoning model to predict the action type online while simultaneously drafting Search.
Moreover, LLM-direct in Table 2 is not a verifier-free baseline—all variants in Table 2 already include our semantic verifier and full-reasoning fallback, a clarification restored in Section 6.3. Even with identical verification and fallback, LLM-direct loses 3.9 percentage points on Miro/GAIA and is both less accurate and slower than DualSpec across all other three settings:
| Pair / Dataset | LLM-direct (+ Verifier): Pass@1 / Latency | DualSpec: Pass@1 / Latency |
|---|---|---|
| Miro 72B+8B / GAIA | 59.2% / 575s | 63.1% / 605s |
| Miro 72B+8B / XBench | 65.0% / 492s | 66.0% / 480s |
| Qwen 32B+4B / GAIA | 27.1% / 67s | 30.1% / 46s |
| Qwen 32B+4B / XBench | 25.0% / 46s | 27.0% / 41s |
In Table 3, DualSpec costs 16.9% and 17.6% less than SPAgent on 72B + 8B and 32B + 4B. On 72B + 30B-A3B, SPAgent lowers cost by 11.0% only by skipping verification entirely in early turns, which degrades accuracy and increases latency in Figure 7b, whereas DualSpec still cuts base-model cost by 49.5% while preserving accuracy at a 2.6× speedup (discussed in Section 6.2 and Appendix B).
Presentation and Implementation Details
- Verifier prompt alignment: Because each ReAct step emits either a tool call or a Final Answer and drafts may conclude prematurely or hesitate, the prompt in Appendix A.1 audits all steps via stagnation and drift rules for tool calls (Criteria 1 and 3) and grounding for final answers (Criterion 2). Section 5.3 now states all three criteria explicitly.
- Thresholds and calibration split: Using 20 GAIA-Text-103 questions without inspecting ground-truth answers, we calibrated solely to match an unlabeled roughly 20% step-fallback rate ( for both MiroThinker-72B pairs, for Qwen3-32B, and shared tokens; Section 6.1), and reused these exact values zero-shot on XBench-DeepSearch and Seal-0.
- Figure 6 and citations: We expanded the Figure 6 caption to clarify parallel drafting, routing, verification, and fallback step by step, and fixed all citation and spacing issues.
Dear Reviewer AQmc,
Thank you again for your review. Our responses and the revised manuscript (with changes highlighted in blue) are posted above. W1/Q1 provides single-step drafting latencies and stage-wise runtime breakdowns showing why waiting for the small model adds zero or negligible overhead, why tools execute after single-token prefill verification, and how DualSpec differs from sequential cascading, while reframing Section 4 as empirical observations. W2/Q2 adds hardware-normalized single-GPU co-location experiments demonstrating a 1.67× speedup on both model pairs on one A100 GPU (Appendix D). W2/Q3 clarifies that Table 1 is an offline Oracle action-type diagnostic (Section 3.1), confirms that LLM-direct in Table 2 already uses our verifier and fallback, and discusses the Table 3 cost–accuracy trade-off (Section 6.2 and Appendix B). We also aligned Section 5.3 with the Appendix A.1 prompt, documented all thresholds, expanded the Figure 6 caption, and fixed citation formatting.
Could you let us know whether these additions address your concerns, or which points remain open? We remain available to answer any further questions during the discussion. If your concerns are resolved, we would be grateful if you could reflect this in your assessment.
Best regards,
The authors
I thank the authors for their thorough response and want to confirm that I have read it. I will keep my score.
We thank the reviewer for their careful review and comments on the drafting-verification workflow, hardware-matched comparisons, simpler baselines, cost trade-offs, and presentation clarity. We have revised the paper in light of this feedback.
W1 / Q1
Waiting for the small model's routing signal introduces zero or negligible idle waiting on Visit steps. Average single-step generation times on GAIA-Text-103 are reported below.
On MiroThinker, SLM w/ reasoning finishes faster than LLM w/o reasoning: Search steps proceed immediately at 2.37s without waiting for the 72B model, while Visit steps incur zero extra waiting since the concurrent 72B draft is already running. On Qwen3, the per-step gap is under 2.8s while keeping long syntheses above τthink=400 tokens immediately available.
Stage-wise runtime breakdowns further show where the end-to-end savings come from, reported in mean seconds per query and shares of total latency.
Executing tools after verification is a deliberate choice: computing the Yes/No log-odds requires a single forward prefill taking only 2%–8.9% of runtime, so speculating web calls before this check saves negligible time while wasting live API quota on the roughly 20% rejected steps.
Unlike sequential model cascading, which runs a small model first and invokes a larger model upon failure along one axis, DualSpec concurrently runs two complementary pathways and reuses the base model for single-token prefill verification. We have clarified this in Section 5.1 and revised Section 4 to present the entropy proxy results as empirical observations, noting the impact of URL copying while relying on discrete URL hit rates in Figure 4b and interventions in Table 1.