- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
TraceQuery learns which unresolved positions to recompute during cached diffusion-language-model inference. A lightweight router is trained on full-denoising trajectories to rank near-future unmasking positions. At inference, it selects a small candidate set, and the backbone commits the highest-confidence freshly evaluated candidate. Periodic refreshes control cache error.
Across four LLaDA/Dream variants and five benchmarks, the paper reports approximately 14× fewer TFLOPs and 2.1× higher throughput at batch size 16, retaining approximately 91–98% of the full-denoising macro-average score. Additional experiments study quality–compute trade-offs, batching, and feature aging.
Critical strengths
-
Learning candidate relevance from the frozen backbone's trajectories requires neither external annotations nor online search. Reporting a measured throughput improvement alongside FLOPs makes the proposal practically relevant, while the four-backbone evaluation provides useful breadth.
-
Section 5.4 shows that confidence features improve offline ranking but that random-aged and policy-aged variants hurt downstream accuracy during cached decoding. This supports evaluating learned inference policies in closed-loop deployment rather than selecting them solely on teacher-state prediction metrics.
Critical weaknesses
-
The benefit of trajectory supervision is not isolated from the underlying sparse-query design. d²Cache already anticipates positions likely to be decoded soon, selects masked candidates using confidence and positional density, and recomputes selected positions against cached states. Its public code also seems to restrict eligible commitments to queried positions. TraceQuery's promising distinction is therefore learning this allocation from full-denoising trajectories. However, the end-to-end ablation studies feature aging, not whether the learned selector outperforms a heuristic selector with the same query budget, refresh schedule, and commitment rule. Consequently, the reported gains cannot yet be attributed specifically to the paper's central contribution. A controlled selector replacement would directly address this concern.
-
The evidence for a superior quality–efficiency trade-off is not yet sufficiently established. Section 5.3 explains the batching advantage through growth of the union of different samples' query positions. This is an implementation choice: the inspected public d²Cache code gathers positions separately for each sample and equalizes query counts without taking a position union. This does not invalidate the reported timings or establish which implementation the authors used, but it leaves open whether the claimed advantage persists against an appropriately batched baseline.
-
The matched-quality comparison also needs a clearly defined evaluation protocol. I am not sure whether I missed some details, but it seems that Figure 3(a)'s full-denoising score is around 65, whereas the four backbones' five-task means in Table 1 range from 43.60 to 54.38; the paper does not identify the different model, subset, or aggregation that explains this. Table 1 alone shows a faster operating point with sometimes substantial quality losses—for example, Dream-Base HumanEval falls from 37.8 to 30.5. Such a trade-off can be worthwhile, but the central superiority claim depends on a reproducible matched-quality comparison. The undisclosed prompt-level separation between router training, configuration selection, and final testing further limits confidence in that comparison.
Questions for authors
-
Under the same sparse execution path, query budget, refresh schedule, and commitment rule, how does the trained router compare with a d²Cache-style or simple locality-based selector in downstream quality and runtime?
-
Which baseline implementation produced the batching results, and does TraceQuery retain its advantage when the baseline uses per-sample query gathering rather than a position union, at comparable output quality?
-
What models, data, aggregation, and operating settings underlie Figure 3, and how do they relate to Table 1? Are router-training and configuration-selection prompts disjoint from final evaluation prompts?
Thank you for identifying where our experiments fell short of our claims. We appreciate your thoughtful critiques and will take them into account as we revise the paper. We agree that the paper needs a controlled test of trajectory supervision, a batching comparison against the official d²Cache implementation, and a clearer account of the matched-quality protocol and data splits.
Since submission, we added age-based requerying and dense warm-up steps on three backbones, fixed issues in the shared evaluation harness, and enabled CUDA graphs for each method’s per-step forward pass. We then reran the full benchmarks at batch size 1.
We thus submitted a new version of the PDF and added all the new experimental results to the appendix. Please download the latest version. Appendix D documents the hardware, software, and method configurations, and Appendix E, Table 4 reports the revised accuracy and latency results.
The controlled selector ablation shows that the learned router outperforms random selection, while its accuracy is statistically indistinguishable from that of the locality and d²Cache-style heuristics under the same sparse execution pipeline. We therefore narrow our claim about the gains attributable to trajectory supervision (Appendix H, Table 8). We also withdraw the reported batching advantage over d²Cache. The original comparison used position unions for both methods, and official d²Cache with per-sample gathering is faster than the TraceQuery implementation timed in the revised comparison (Appendix M, Table 15). The full-set results still show meaningful accuracy losses on some tasks. We scope the matched-quality claim to the measured operating point on a 100-document LLaDA-Instruct GSM8K subset and report a separate full-set analysis that excludes the configuration-selection documents (Appendix G, Table 7 and Appendix K, Table 13).
Q1. How does the learned router compare with heuristic selectors under the same execution path?
We changed only the position score, keeping the query budget, refresh schedule, requery rule, warm-up, and commitment rule fixed. Across six tasks with 100 documents each, the learned, locality, and d²Cache-style selectors scored 49.5%, 49.6%, and 48.2% on LLaDA-Instruct, and 43.1%, 43.3%, and 43.3% on LLaDA-Base. The 1.3-point gap for the d²Cache-style selector on LLaDA-Instruct was not statistically significant (p=0.19). The ablation reports downstream accuracy. Per-selector runtime remains unmeasured in this comparison (Appendix H, Table 8).
Q2. Which implementation produced the batching results, and does the advantage persist with per-sample gathering?
The original results used our reimplementations of both methods, which forwarded the union of the batch’s query positions. At batch size 16 on LLaDA-Instruct GSM8K, official d²Cache with per-sample gathering takes 5.26 seconds per sample, compared with 6.30 seconds for the union-based TraceQuery implementation. We withdraw the original batching claim. The revised TraceQuery implementation adopts per-sample gathering, and a matched-quality timing comparison with both methods using per-sample gathering remains open. The full-set batch-1 results are in Appendix E, Table 4, and the batching measurements are in Appendix M, Table 15.
Q3. What underlies Figure 3(a), and how were prompts separated?
The original Figure 3(a) plots LLaDA-Base accuracy on GSM8K alone. Each point corresponds to a query-budget and refresh-schedule configuration plotted against its measured TFLOPs. The caption and text incorrectly describe the result as an average across benchmarks. This single-task setting explains why the dense reference is around 65%, while Table 1’s five-task dense means range from 43.6% to 54.4%. The revised wall-clock sweep reports settings and numeric results on a 100-document LLaDA-Instruct GSM8K subset at batch size 1. TraceQuery, dense decoding, and dLLM-Cache each score 74.0%, with latencies of 7.53, 38.59, and 10.88 seconds per sample, respectively (Appendix G, Table 7).
Router-training prompts are excluded from all evaluations, and router recall is measured on held-out prompts (Appendix D). The configuration-selection documents are included in the main full-set results and are also used in the 100-document sweep. We separately rescored the full-set runs after excluding the selection documents. Across the tasks covered by this check, accuracy gaps to dense shifted by −0.9 to +1.2 percentage points. We excluded HumanEval from this check because only 14 documents remained after both exclusions (Appendix K, Table 13).