- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper extends single-chain diffusion drafters (DFlash) to tree-based verification. It proposes a budget-optimal candidate tree that keeps the top-N candidates ranked by the product of the drafter’s position-wise marginals, a cost-aware rule for selecting the node budget from measured verification latency, and global best-first allocation of a shared verification budget across concurrent requests.
Critical strengths
None
Critical weaknesses
The paper’s main contributions have already been proposed in prior work that the paper neither cites nor compares against.
-
The core method and the cost-aware budget rule reproduce DDTree and CaDDTree. Building a candidate tree from the per-position marginals of a one-pass block diffusion drafter, expanded best-first by the product of marginals, is the method of DDTree (arXiv:2604.12989), the standard tree builder for DFlash. Selecting the node budget by directly maximizing throughput under explicit draft and verification costs, proving that throughput is unimodal in the budget, and stopping greedily without an offline budget search is the method of CaDDTree (arXiv:2606.01813), which also evaluates DFlash on Qwen3-4B/8B and reports matching the oracle budget. Theorems 1 and 2 essentially restate these results. None of these works is cited.
-
The experiments do not establish a contribution beyond these methods. Main results are compared against the DFlash chain and EAGLE-3, while DDTree, the direct baseline for trees built from DFlash marginals, is absent. The reported gains over the chain therefore largely reflect the known benefit of tree verification.
What remains is the shared-budget allocation under concurrent serving, which applies global best-first allocation with a variable-length kernel. These issues concern the paper’s central claims and cannot be resolved without repositioning the work against DDTree and CaDDTree.
Questions for authors
Please explain the methodological and theoretical differences between this paper and prior art such as DDTree (tree construction from one-pass drafter marginals) and CaDDTree (throughput-optimal node budget with a unimodality guarantee and greedy stopping rule).
20 comments
Response to Reviewer HuaJ (2/13)
W1. CAST's additions to DDTree.
"Building a candidate tree from the per-position marginals of a one-pass block diffusion drafter, expanded best-first by the product of marginals, is the method of DDTree (arXiv:2604.12989), the standard tree builder for DFlash."
Answer. CAST shares DDTree's best-first order by the product of marginals and cites it, and it differs from DDTree in four decisions that this order leaves open. DDTree takes the node budget as an input and CaDDTree stops each round's tree with a fitted latency cost, while CAST sets one fixed width for each deployment (Theorem 2). DDTree builds each request's tree from its own budget, while CAST splits one budget across concurrent requests in one best-first pass (Propositions 2 and 3). DDTree's optimality result covers the full-rank tree of one request, while Theorem 1 covers any finite prefix-closed universe, such as the top 8 tokens of each position or the union of the requests' universes. DDTree's tree is optimal for the drafter's factorized surrogate, while CAST bounds the loss of its tree against the tree that is optimal under the round's target prefix law (Proposition 1). Theorems 1 and 2 therefore do not restate DDTree or CaDDTree.
The table lists where CAST differs from the two methods, and the paragraphs below take the differences one at a time.
| Question | DDTree and CaDDTree | CAST |
|---|---|---|
| Order in which nodes enter the tree | DDTree expands best-first by the product of marginals, and CaDDTree picks the node budget of DDTree's tree | The same best-first order (Section 3.2, p. 4) |
| Size of the tree | DDTree takes the node budget as an input and reports the best of a sweep. CaDDTree stops each round's tree with a greedy rule on a convex cost fitted to a latency profile measured offline for each model and GPU | One fixed width for each deployment, the width with the largest long-run throughput under the drafter's scores, set from offline scores and one short latency probe (Theorem 2) |
| Several requests verified together | DDTree builds each request's tree from its own node budget, and CaDDTree does not address a shared budget | One best-first pass splits one budget across the requests (Propositions 2 and 3) |
| Candidate set of the optimality result | DDTree's result covers the full-rank tree, which expands over every rank at each position | Any finite prefix-closed universe, which covers the top K=8 tokens of each position and the union of the requests' universes (Theorem 1) |
| Distribution the guarantee refers to | DDTree's tree is optimal for the drafter's factorized surrogate, and its Remark 1 states that this is exact for the surrogate, not for the target distribution | A bound on the loss against the tree that is optimal under the round's target prefix law (Proposition 1) |
The shared order is DDTree's. The revision cites DDTree at Theorem 1 (Section 3.2, p. 4), in Section 1 (p. 1) and in the first paragraph of Section 6 (p. 10), and DDTree had not yet been released when this study was designed. The order fixes which nodes enter first. How many nodes a deployment verifies, how concurrent requests share them and what the resulting tree guarantees under the target law are separate decisions, and they are the four lower rows of the table.
Response to Reviewer HuaJ (3/13)
W1. Target-law guarantees, continued.
"expanded best-first by the product of marginals, is the method of DDTree"
The first theoretical difference is the distribution that the guarantee refers to. DDTree's Proposition 2, which its Algorithm 1 attains (its Proposition 3), gives the optimal tree for the surrogate objective of the drafter's factorized distribution, and its Remark 1 states that this result is exact for that surrogate, not for the target-model distribution. CAST's Proposition 1 bounds the loss of its plug-in tree against the tree that is optimal under the round's target prefix law inside the same candidate universe, by the score errors on the nodes where the two trees differ. The new Appendix G.3 measures these errors at every depth of the deployed tree. The score's mean is within 0.010 of the realized hit rate in absolute terms, and a correction fit for each depth group and branching pattern on one half of the prompts changes the accepted length of the tree rebuilt on the other half by at most 0.19% at N=63.
The second theoretical difference is the candidate set that optimality covers. DDTree expands over every rank at each position, and its Proposition 2 is stated for that full-rank tree of one request. Theorem 1 holds for any finite prefix-closed universe and any prefix belief that does not increase from ancestor to descendant. It therefore shows that CAST's tree inside the top K=8 tokens of each position (Section 3.2, p. 4) is the optimal tree of its universe under the drafter's scores, and the proof of Proposition 2 applies it to the union of the requests' candidate universes (Appendix B.4). The capped tree matches the full-rank tree's accepted length to within 1.1% at the evaluated budgets while scoring 8 candidates for each position (Appendix G.1).
W1. CaDDTree and Theorem 2, continued.
"Selecting the node budget by directly maximizing throughput under explicit draft and verification costs, proving that throughput is unimodal in the budget, and stopping greedily without an offline budget search is the method of CaDDTree (arXiv:2606.01813), which also evaluates DFlash on Qwen3-4B/8B and reports matching the oracle budget. Theorems 1 and 2 essentially restate these results."
The third difference is the decision about tree size, where Theorem 2 decides something different from CaDDTree's stop. CaDDTree decides in every round when to stop growing that round's tree, with a convex cost fitted to a latency profile measured offline for each model and GPU, and it maximizes the ratio of each round as a tractable surrogate for the long-run ratio that it also states (its Appendix A.1). Its experiments use one GPU type (A800), where it reports matching the oracle budget. Theorem 2 instead gives one fixed width for each deployment, the width with the largest long-run throughput under the drafter's scores. Its stopping condition is stated on the order statistics of the prefix masses, which CAST reads from the drafter's offline scores, and on the round cost, whose slope comes from one latency probe of at most 79 seconds with model loading on the RTX A6000. Averaged over offline rounds, the ratio is the expected committed length over the expected round time of a fixed width, so CAST targets long-run throughput directly and decides it once for each deployment. Theorem 2 uses a unimodality argument like CaDDTree's, but on the long-run throughput of a fixed width rather than on the ratio of one round.
A width for each deployment is needed because the best width changes with the GPU and the drafter. The predicted width is 127 on the H100 SXM, 63 on Blackwell with Qwen3-8B, and 63 and 95 on the RTX A6000 with DART's and PARD's drafters. In all twelve settings of Tables 2 and 18 it is within 1.7 percentage points of the best swept width in domain-mean gain, while each single width from 15 to 127 is at least 4.5 points lower in some setting. In every round the drafter's scores still choose which N nodes form the tree, so the deployment width and each round's tree are separate choices.
Theorem 1 is the general result over capped and pooled candidate sets described above, and Theorem 2 decides a fixed deployment width rather than a stop in each round, so neither restates CaDDTree's results.
Response to Reviewer HuaJ (1/13)
We thank the reviewer for raising DDTree and CaDDTree.
Before writing this response, we ran a number of new experiments for the reviews and restructured the paper around their results, which took much of the discussion period. We realize that little time remains, and we respectfully ask the reviewer to read our response and the new experiments.
Response overview.
Answer. CAST studies deployment width and shared budgets, with guarantees beyond the common ordering and measured gains against DDTree's released code.
The review states that its concerns "cannot be resolved without repositioning the work against DDTree and CaDDTree". New and changed text in the revision is shown in blue.
Our answers by concern.
W1. Overlap with DDTree and CaDDTree. CAST shares DDTree's best-first order by the product of marginals and cites it (Section 6), and it differs in what that order leaves open. DDTree takes the node budget as an input and reports the best of a sweep, while CAST predicts one width for each deployment from the drafter's scores and one latency probe. DDTree builds each request's tree from its own budget, while CAST splits one budget across concurrent requests. DDTree's tree is optimal for the drafter's factorized surrogate, while CAST also bounds the loss of its tree under the target law (Proposition 1). CaDDTree stops each round's tree with a greedy rule on a latency cost fitted offline for each model and GPU, while CAST sets one fixed width for each deployment by its long-run throughput (Theorem 2).
W2. Gains beyond tree verification. Measured against DDTree's released code, CAST commits more tokens under load and reaches a higher speedup over the autoregressive decoder at every batch-one budget pairing of Table R7. Equal-budget allocation gains and direct decoder timings separate the contribution from the general benefit of trees (Tables R1 and R7).
Q1. Methodological and theoretical differences. General optimality supports the rank cap (Theorem 1), and a fixed deployment width follows the stopping rule (Theorem 2). CAST shares one budget across concurrent requests (Propositions 2 and 3) and bounds the loss of its tree under the target law (Proposition 1). CaDDTree's experiments use one GPU type (A800), and CAST predicts the width for each deployment, so the budget decision and guarantees extend beyond the common ordering (Section 3.3 and Appendix B.4).
The revision does this repositioning in Section 1 (p. 1), Section 3.2 (p. 4), Section 3.3 (p. 5), Section 5 (p. 9), Section 6 (p. 10) and Appendices F and J, and it adds measured comparisons with DDTree's released code.
Conclusion. The revision identifies CAST's added width, allocation and target-law guarantees and measures its gains against DDTree.