- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
SpecSplice jointly selects parametric and retrieved draft candidates under a shared node budget. Prefix-aware scores enable best-first construction with on-demand retrieval, with optimality guaranteed for the defined surrogate objective. Experiments show 12.8–37.1% higher decoding throughput than the strongest baseline across 12 agentic model–workload pairs.
Critical strengths
-
The method addresses a concrete limitation of existing hybrids: parametric prefixes can expose useful retrieval matches within a decoding round. Unified scoring and dynamic splicing provide a coherent solution with a sound surrogate-optimality argument.
-
Throughput gains are consistent across models and agentic workloads. Component ablations, alternative drafting backends, official harnesses, and batching experiments strengthen the empirical evidence.
Critical weaknesses
-
Moderate novelty. The method builds on established ideas in hybrid drafting, path-based scoring, and best-first tree construction. Its main advance is a unified, prefix-aware selection framework with dynamic retrieval at parametric nodes, making the methodological contribution useful but incremental.
-
Parameters and node budgets are optimized per model–workload pair, but the manuscript does not clarify whether tuning uses held-out data. This limits confidence in performance on unseen tasks.
-
Important evaluation details are missing, including sample counts, selected budgets, measurement variability, retrieval-index initialization, and precise timing boundaries. These omissions hinder reproducibility.
Questions for authors
-
Were hyperparameters selected independently of the reported evaluation tasks? How well do fixed settings transfer to unseen tasks?
-
Does throughput measurement include retrieval, tree construction, and index updates? Please clarify sample counts, repeated measurements, and index-reset policies.
3 comments
Response (2/2): Tuning protocol, transfer, and evaluation details
W2 / Q1. Tuning and transfer
Yes, the original hyperparameters were selected on the evaluation tasks. For each model–workload pair, every method's node budget and any method-specific parameter were selected to maximize throughput (Appendix C, Table 3). We should have been more explicit about this protocol, and we agree that it alone does not establish transfer to unseen tasks.
Following your suggestion, we added a transfer study (Section 4.5; Appendix I, Table 10). Tuning separately on Spider2-DBT and BFCL-v4 with Qwen3.5-9B selects the same value, λ = 0.1, which we reuse unchanged on the remaining agentic pairs. Speedup and MAT decrease by at most 2.0% and 1.0%, and SpecSplice remains 12–37% faster than the strongest baseline in Table 1 on all 12 agentic pairs. The original budgets are unchanged, so this study isolates the transfer of λ.
Budgets are examined separately: on Qwen3.5-9B/Spider2-DBT at batch size 1, SpecSplice has higher throughput than Selection at every tested common budget from 16 to 256, on both GPUs (Appendix F, Figure 9(a)).
The transfer holds for the agentic workloads. On Spec-Bench, λ = 0.1 lowers the two Qwen models' speedups by 7.7–9.1% relative to their tuned weights, so a smaller weight works better there (Table 10).
W3 / Q2. Timing and reproducibility details
We agree that these details should have been clearer, and they are now stated in the paper.
The metric is decode throughput. Measured decoding time includes drafting, retrieval, suffix-tree updates, tree construction, and verification (Section 4.1, "Metrics"). Table 2 separates drafting, meaning the DFlash forward pass plus construction including retrieval, from the remaining work, which is mainly target verification (Section 4.4).
The global suffix tree starts empty for each model–workload–method run and grows as each request's output is generated (Appendix C, "Retrieval index"). A separate per-request tree covers the current prompt and output (Appendix B.2).
Appendix C ("Workloads") now reports the evaluated sizes: 300 SWE-bench Lite instances, 68 Spider2-DBT tasks, 665 BFCL-v4 memory and web-search entries, 480 Spec-Bench questions, and 160 Gaia2 mini scenarios. The runtime breakdown separately averages over 100 Gaia2 tasks (Tables 2 and 7).
For variability, Appendix D.1 (Table 5) reports five sampling runs each for SpecSplice and Selection on Qwen3.5-9B/Gaia2. Mean speedup is 4.91 ± 0.09× for SpecSplice and 3.53 ± 0.07× for Selection, and the geometric-mean gain is 39.4% (95% CI 35.5–43.3%).
Recommendation After the Author Response
Thank you for your response. It addressed my questions, and I appreciate the clarifications. My original assessment remains unchanged, and I continue to lean toward acceptance.
Response (1/2): Methodological contribution
We sincerely thank the reviewer for the careful and balanced review. Your comments on tuning and evaluation details helped us identify where clearer reporting and additional evidence were needed.
W1. Methodological contribution
We agree that hybrid drafting, path scoring, and best-first construction are established. Our contribution is to formulate hybrid draft construction as a joint optimization over individual parametric and retrieved nodes, under a shared verification budget and an explicit accepted-length surrogate.
The hybrid schemes compared in the paper combine drafts through source selection or prescribed composition rules. Such rules can leave useful combinations of individual candidates unexplored. In our formulation, candidate inclusion is the optimization decision: the selected nodes jointly determine the source allocation and splice structure. The candidate space includes continuations retrievable at the root and at every eligible parametric node (Section 2.3).
This joint optimization requires a comparable contribution estimate for the different evidence of the two sources. Parametric probabilities provide predictive confidence, and retrieval frequencies provide historical support. A retrieved node contributes only if its preceding parametric prefix is accepted, so its retrieval support must be evaluated together with prefix survival. Eq. (4) makes this dependency explicit on a common contribution scale, and Eq. (5) maximizes the sum of these contributions over the combined candidate space (Sections 3.1–3.2).
The controlled ablations show the value of treating hybrid composition as joint node selection. At a common B = 64 with retrieval fixed at the root, unified scoring raises MAT by 17.9% over Selection and 26% over Graft. Extending retrieval to all eligible parametric nodes, with scoring unchanged, adds a further 10.9% (Section 4.3, Figure 5(a–b)). These comparisons isolate the benefit of joint budget allocation and of the expanded candidate space.
Appendix J makes the resulting composition concrete: SpecSplice combines both sources within a verification step, which reduces the verification steps for the identical output from 12 to 9 (Figure 10(b)).
The same contribution score also supplies the path monotonicity and query bounds for efficient construction: SpecSplice skips queries that cannot affect selection while retaining the defined surrogate optimum over the supplied candidate space (Section 3.2, Propositions 1–2; Appendix A.2).