- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes FACT, a framework that uses historical target verification feedback to improve candidate prioritization in block-diffusion speculative decoding. It also introduces cost-aware budget control to dynamically select the verification tree size and improve decoding efficiency.
Critical strengths
-
The paper identifies a reasonable limitation of existing tree-based speculative decoding methods and proposes using historical target feedback to improve candidate prioritization.
-
FACT is training-free and introduces no additional inference-time model calls. Its cost-aware budget control also considers the trade-off between acceptance length and decoding latency.
Critical weaknesses
-
The paper does not sufficiently justify the necessity of the proposed method. In practice, target tokens are often among the top-ranked draft candidates (e.g., Top-3), which existing tree-based methods already exploit. The authors should analyze how frequently correct paths are missed by existing methods and how much improvement is possible.
-
The reported latency improvements over DDTree are modest (0.27%–3.05%), and the ablation studies are insufficient. The paper does not isolate the contributions of Feedback Calibration and Cost-aware Budget Control. More detailed experiments are needed to explain the performance gains and justify both components.
-
The paper lacks a comparison with DARTree, which also improves candidate prioritization and verification budget utilization through score correction. Since DARTree reports substantial speedups and shares similar objectives, this comparison is important to evaluate FACT's competitiveness.
Questions for authors
-
How often do target tokens fall outside the top-ranked draft candidates (e.g., Top-3, Top-5, or Top-10)? How frequently does DDTree miss correct paths, and what is the potential upper bound of improvement under the same verification budget?
-
Does the frequency of these missed candidates depend on the capability gap between the draft and target models? How does this gap affect the effectiveness of FACT?
-
Could the authors provide separate ablation studies for Feedback Calibration and Cost-aware Budget Control? How much does each component contribute to the reported speedup?
-
Could the authors compare FACT with DARTree under the same experimental settings? Both methods share similar objectives, and such a comparison would help establish FACT's competitiveness.
7 comments
Response to Reviewer LwQL — Part 2/4: Component Contributions and Decoding Gains (W2/Q3)
We thank the reviewer for suggesting additional component ablations.
Q3. Could the authors provide separate ablation studies for Feedback Calibration and Cost-aware Budget Control? How much does each component contribute to the reported speedup?
W2. The reported latency improvements over DDTree are modest (0.27%–3.05%), and the ablation studies are insufficient. The paper does not isolate the contributions of Feedback Calibration and Cost-aware Budget Control. More detailed experiments are needed to explain the performance gains and justify both components.
To quantify the independent contributions of Feedback Calibration and Cost-aware Budget Control, we have conducted a controlled study under the same C++ implementation by enabling or disabling the two components separately. We have extended the original two timing repeats to six timing blocks. The results are presented below.
Table 3. Component ablation
| Configuration | Time (s) ↓ | Speedup ↑ | Time Saved (%) ↑ |
|---|---|---|---|
| Raw fixed 128 | 2.3270 | 5.207× | 0.000 |
| Calibrated fixed 128 | 2.3136 | 5.237× | 0.576 |
| Raw adaptive | 2.2277 | 5.439× | 4.266 |
| Full FACT | 2.2259 | 5.443× | 4.344 |
As shown in the table, Feedback Calibration independently reduces decoding latency by 0.58% at a fixed budget, while adaptive budgeting achieves a 4.27% reduction without calibration. Combining both components yields the lowest latency, achieving a 4.34% reduction over the baseline.
These results demonstrate the complementary functions of the two components. Feedback Calibration refines which candidate paths receive verification resources, whereas Cost-aware Budget Control determines the appropriate verification budget. Their integration enables FACT to optimize candidate selection and verification cost within a unified framework.
We have further evaluated FACT against CaDDTree and BASTION under matched Qwen3-8B/DFlash checkpoints, a single RTX 4090, and six timing blocks across 182 prompts. The results are summarized below.
Table 4. Six-domain speedups
| Domain | CaDDTree | BASTION | FACT |
|---|---|---|---|
| HumanEval | 6.472× | 6.201× | 6.541× |
| GSM8K | 5.811× | 5.661× | 5.827× |
| MBPP | 5.823× | 5.647× | 5.898× |
| MATH-500 | 6.628× | 6.286× | 6.677× |
| Alpaca | 3.179× | 3.138× | 3.228× |
| AIME25 | 6.029× | 5.618× | 6.154× |
| Geometric Mean | 5.505× | 5.292× | 5.569× |
All values represent speedups over matched autoregressive decoding. FACT achieves the highest speedup across all six domains, with a geometric mean of 5.569×, compared with 5.505× for CaDDTree and 5.292× for BASTION. These results further support the effectiveness of FACT's feedback-informed allocation and cost-aware budget selection across diverse workloads.
Response to Reviewer LwQL — Part 3/4: Draft–Target Agreement and Calibration Gains (Q2)
We thank the reviewer for the insightful question regarding the relationship between draft–target agreement and FACT's effectiveness.
Q2. Does the frequency of these missed candidates depend on the capability gap between the draft and target models? How does this gap affect the effectiveness of FACT?
To investigate this relationship, we have conducted a controlled experiment by fixing the Qwen3-8B target, draft architecture, and verification budget of 128, while varying the draft training checkpoints at steps 0, 50, and 200.
The evaluation uses 40 development prompts, 160 test prompts, and six timing blocks, covering 8,640 requests across the evaluated configurations. Training here is used to construct different draft states for this controlled experiment; FACT itself requires no additional neural-network training. The results are summarized below.
Table 5. Draft–target agreement
| Draft Step | Top-1 Agreement (%) | Raw Coverage | Original Table Coverage | Refitted Table Coverage | Original Table Saved (%) | Refitted Table Saved (%) |
|---|---|---|---|---|---|---|
| 0 | 40.75 | 1,006 | 1,010 | 1,009 | 0.678 | 0.648 |
| 50 | 41.55 | 1,023 | 1,025 | 1,022 | 0.096 | -0.422 |
| 200 | 46.75 | 1,092 | 1,088 | 1,094 | 0.751 | -0.075 |
Top-1 agreement measures how often the AR reference token has the maximal draft score, including ties, over 2,385 common reference positions. Coverage counts contiguous reference tokens across the 160 test prompts. Original and refitted refer to the historical table and a table refitted on development prompts, respectively. Time saved is 100 × (1 − method time / raw time), calculated within each checkpoint under greedy EOS/512 stopping, with native output lengths retained.
As shown in the table, Top-1 agreement increases from 40.75% to 46.75%, accompanied by an increase in raw reference-path coverage from 1,006 to 1,092. This indicates that improved draft–target agreement is associated with better baseline path coverage in the evaluated checkpoints.
Notably, the original calibration table achieves positive latency reductions at all three checkpoints, while its benefits vary non-monotonically with agreement. These results suggest that FACT's effectiveness depends not only on draft–target agreement but also on the distribution of candidate-ranking errors and the paths already retained by the uncalibrated tree. This further highlights the importance of feedback-informed candidate allocation across different draft states.
Response to Reviewer LwQL — Part 1/4: Candidate Ranks and Path Coverage (W1/Q1)
We thank the reviewer for the valuable questions regarding candidate rankings and path coverage.
To quantify the limitations of existing draft-tree allocation, we have conducted additional analyses of target-token rankings across different draft depths.
Table 1. Draft-token ranks
As shown in the table, the Top-3 omission rate increases substantially with draft depth, reaching 55.77% at depth 15. Across 8,513 reference positions, the overall omission rates for Top-3, Top-5, and Top-10 are 31.81%, 25.35%, and 17.47%, respectively. These results reveal the increasing discrepancy between draft candidate rankings and target-accepted continuations at deeper positions.
Furthermore, we have evaluated reference-path coverage under the same verification budget of 128 nodes. The results are summarized below.
Table 2. Reference-path coverage
The reference-chain oracle incorporates the available target-reference prefix, limited to 15 unknown positions, and achieves longer coverage than DDTree on 464 of 571 prefixes. This demonstrates substantial untapped potential in allocating a fixed verification budget to complete reference paths.
Under the same budget, Feedback Calibration increases coverage from 4,330 to 4,349, demonstrating that historical verification feedback can improve candidate selection without increasing the tree size. Together, these findings establish the practical motivation for FACT's feedback-informed allocation mechanism.