acceptodds
Under review as a conference paper at ICLR 2027

TraceQuery: Learning What to Recompute from Full-Denoising Trajectory in Diffusion Language Models

Abstract

Diffusion large language models (dLLMs) enable flexible generation through iterative masked denoising, but incur substantial inference cost because token representations are repeatedly recomputed across denoising steps. Existing cache-based methods reduce this cost by reusing intermediate results across denoising steps and recomputing only a subset of positions at each step, chosen either on a fixed schedule or by heuristic signals of change. Our trajectory experiments instead point to decoding trajectory as the more relevant quantity. Cached generation remains accurate when it stays close to the full-denoising trajectory, with errors concentrated around unmask-order deviations and reduced as longer reference prefixes are recovered. This suggests a more direct principle: limiting representation error helps, but keeping generation on the full-denoising trajectory is a more direct route to accurate decoding. We therefore formulate efficient dLLM inference as trajectory-guided querying. Rather than broadly repairing representation error, we use full-denoising trajectories to supervise which unresolved positions should receive limited fresh computation. Based on this formulation, we introduce TRACEQUERY, a lightweight sparse decoding framework that predicts a small set of near-future trajectory-relevant positions, queries only those candidates, and lets the backbone select which one to commit from their fresh predictions. TRACEQUERY delivers the strongest quality–computation trade-off among cache-based approaches, sustaining high accuracy with substantially fewer fresh token computations, and generalizes across tasks, model variants, and families.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
language model, diffusion model

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_nViKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TraceQuery learns which unresolved positions to recompute during cached diffusion-language-model inference. A lightweight router is trained on full-denoising trajectories to rank near-future unmasking positions. At inference, it selects a small candidate set, and the backbone commits the highest-confidence freshly evaluated candidate. Periodic refreshes control cache error.

Across four LLaDA/Dream variants and five benchmarks, the paper reports approximately 14× fewer TFLOPs and 2.1× higher throughput at batch size 16, retaining approximately 91–98% of the full-denoising macro-average score. Additional experiments study quality–compute trade-offs, batching, and feature aging.

Critical strengths
  1. Learning candidate relevance from the frozen backbone's trajectories requires neither external annotations nor online search. Reporting a measured throughput improvement alongside FLOPs makes the proposal practically relevant, while the four-backbone evaluation provides useful breadth.

  2. Section 5.4 shows that confidence features improve offline ranking but that random-aged and policy-aged variants hurt downstream accuracy during cached decoding. This supports evaluating learned inference policies in closed-loop deployment rather than selecting them solely on teacher-state prediction metrics.

Critical weaknesses
  1. The benefit of trajectory supervision is not isolated from the underlying sparse-query design. d²Cache already anticipates positions likely to be decoded soon, selects masked candidates using confidence and positional density, and recomputes selected positions against cached states. Its public code also seems to restrict eligible commitments to queried positions. TraceQuery's promising distinction is therefore learning this allocation from full-denoising trajectories. However, the end-to-end ablation studies feature aging, not whether the learned selector outperforms a heuristic selector with the same query budget, refresh schedule, and commitment rule. Consequently, the reported gains cannot yet be attributed specifically to the paper's central contribution. A controlled selector replacement would directly address this concern.

  2. The evidence for a superior quality–efficiency trade-off is not yet sufficiently established. Section 5.3 explains the batching advantage through growth of the union of different samples' query positions. This is an implementation choice: the inspected public d²Cache code gathers positions separately for each sample and equalizes query counts without taking a position union. This does not invalidate the reported timings or establish which implementation the authors used, but it leaves open whether the claimed advantage persists against an appropriately batched baseline.

  3. The matched-quality comparison also needs a clearly defined evaluation protocol. I am not sure whether I missed some details, but it seems that Figure 3(a)'s full-denoising score is around 65, whereas the four backbones' five-task means in Table 1 range from 43.60 to 54.38; the paper does not identify the different model, subset, or aggregation that explains this. Table 1 alone shows a faster operating point with sometimes substantial quality losses—for example, Dream-Base HumanEval falls from 37.8 to 30.5. Such a trade-off can be worthwhile, but the central superiority claim depends on a reproducible matched-quality comparison. The undisclosed prompt-level separation between router training, configuration selection, and final testing further limits confidence in that comparison.

Questions for authors
  • Under the same sparse execution path, query budget, refresh schedule, and commitment rule, how does the trained router compare with a d²Cache-style or simple locality-based selector in downstream quality and runtime?

  • Which baseline implementation produced the batching results, and does TraceQuery retain its advantage when the baseline uses per-sample query gathering rather than a position union, at comparable output quality?

  • What models, data, aggregation, and operating settings underlie Figure 3, and how do they relate to Table 1? Are router-training and configuration-selection prompts disjoint from final evaluation prompts?

1 comment

Thank you for identifying where our experiments fell short of our claims. We appreciate your thoughtful critiques and will take them into account as we revise the paper. We agree that the paper needs a controlled test of trajectory supervision, a batching comparison against the official d²Cache implementation, and a clearer account of the matched-quality protocol and data splits.

Since submission, we added age-based requerying and dense warm-up steps on three backbones, fixed issues in the shared evaluation harness, and enabled CUDA graphs for each method’s per-step forward pass. We then reran the full benchmarks at batch size 1.

We thus submitted a new version of the PDF and added all the new experimental results to the appendix. Please download the latest version. Appendix D documents the hardware, software, and method configurations, and Appendix E, Table 4 reports the revised accuracy and latency results.

The controlled selector ablation shows that the learned router outperforms random selection, while its accuracy is statistically indistinguishable from that of the locality and d²Cache-style heuristics under the same sparse execution pipeline. We therefore narrow our claim about the gains attributable to trajectory supervision (Appendix H, Table 8). We also withdraw the reported batching advantage over d²Cache. The original comparison used position unions for both methods, and official d²Cache with per-sample gathering is faster than the TraceQuery implementation timed in the revised comparison (Appendix M, Table 15). The full-set results still show meaningful accuracy losses on some tasks. We scope the matched-quality claim to the measured operating point on a 100-document LLaDA-Instruct GSM8K subset and report a separate full-set analysis that excludes the configuration-selection documents (Appendix G, Table 7 and Appendix K, Table 13).

Q1. How does the learned router compare with heuristic selectors under the same execution path?

We changed only the position score, keeping the query budget, refresh schedule, requery rule, warm-up, and commitment rule fixed. Across six tasks with 100 documents each, the learned, locality, and d²Cache-style selectors scored 49.5%, 49.6%, and 48.2% on LLaDA-Instruct, and 43.1%, 43.3%, and 43.3% on LLaDA-Base. The 1.3-point gap for the d²Cache-style selector on LLaDA-Instruct was not statistically significant (p=0.19). The ablation reports downstream accuracy. Per-selector runtime remains unmeasured in this comparison (Appendix H, Table 8).

Q2. Which implementation produced the batching results, and does the advantage persist with per-sample gathering?

The original results used our reimplementations of both methods, which forwarded the union of the batch’s query positions. At batch size 16 on LLaDA-Instruct GSM8K, official d²Cache with per-sample gathering takes 5.26 seconds per sample, compared with 6.30 seconds for the union-based TraceQuery implementation. We withdraw the original batching claim. The revised TraceQuery implementation adopts per-sample gathering, and a matched-quality timing comparison with both methods using per-sample gathering remains open. The full-set batch-1 results are in Appendix E, Table 4, and the batching measurements are in Appendix M, Table 15.

Q3. What underlies Figure 3(a), and how were prompts separated?

The original Figure 3(a) plots LLaDA-Base accuracy on GSM8K alone. Each point corresponds to a query-budget and refresh-schedule configuration plotted against its measured TFLOPs. The caption and text incorrectly describe the result as an average across benchmarks. This single-task setting explains why the dense reference is around 65%, while Table 1’s five-task dense means range from 43.6% to 54.4%. The revised wall-clock sweep reports settings and numeric results on a 100-document LLaDA-Instruct GSM8K subset at batch size 1. TraceQuery, dense decoding, and dLLM-Cache each score 74.0%, with latencies of 7.53, 38.59, and 10.88 seconds per sample, respectively (Appendix G, Table 7).

Router-training prompts are excluded from all evaluations, and router recall is measured on held-out prompts (Appendix D). The configuration-selection documents are included in the main full-set results and are also used in the 100-document sweep. We separately rescored the full-set runs after excluding the selection documents. Across the tasks covered by this check, accuracy gaps to dense shifted by −0.9 to +1.2 percentage points. We excluded HumanEval from this check because only 14 documents remained after both exclusions (Appendix K, Table 13).

Reviewer_n4dWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper formulates selective recomputation in cached dLLM decoding as trajectory-guided querying. Full-denoising runs on the frozen backbone provide offline supervision for a two-layer MLP router that ranks unresolved positions from three cheap signals (last-layer attention, local mask density, relative distance). Each sparse step recomputes the top-HH positions (H=5H=5) plus one recently committed position, the backbone chooses what to commit from those fresh predictions, and R=16R=16 periodic full refreshes are retained. Across four LLaDA/Dream backbones and five benchmarks, compute drops from about 5900 to about 400 TFLOPs with 2.1x wall-clock speedup, retaining 91-98% of full-denoising average score.

Critical strengths

I like the reformulation. Existing cache methods aim at limiting representation error; this one aims at anticipating upcoming decoding decisions, and the supervision comes offline from the frozen model with no external annotation or trajectory search. The router is three features and a two-layer MLP, so its inference cost is near zero. The Stage II closed-loop study reports paired bootstrap intervals and honestly overturns the Stage I offline ranking, which is not common in this area.

Critical weaknesses
  1. Both baselines run far below their own level. On the same model and benchmarks, dLLM-Cache reports 4.2-4.5x TPS and 5.5-6.2x FLOPs on LLaDA-8B-Instruct with scores on GSM8K, MBPP and BBH slightly improving; here it gets 1.23x TPS, 3.7x FLOPs, and MBPP drops from 36.8 to 28.4. d2Cache reports 3.5x average throughput over six benchmarks with claimed quality gains; here it is 0.6x, slower than full denoising, and on LLaDA-Instruct BBH falls 58.0 to 40.6 and GSM8K 76.3 to 52.8. Both baselines simultaneously losing their speedup and losing quality they do not lose in their own papers points at configuration, yet no baseline hyperparameter is disclosed (dLLM-Cache's Kp/KrK_p/K_r are given per benchmark in the original and are very sensitive).
  2. The training data is never described. The only statement is "we train on mixed tasks and retrain for each backbone", with no dataset names, no prompt count, and no claim that evaluation prompts are excluded. Stage I's Recall@5 of 0.78 is measured over 17,670 held-out positions; the unit held out is a position, not a prompt, so other steps of the same trajectory and the same prompt may well be in training, and the number says little about generalization to unseen prompts.
  3. The central claim conflicts with the paper's own main table, and there is no ceiling. In Table 1 the relative average is below dLLM-Cache on three of four backbones (91.0 vs 98.1, 93.8 vs 97.7, 98.1 vs 104.5), with Dream-Base losing 19% on HumanEval and LLaDA-Base 14% on MBPP. The stronger claim rests only on the curves of Figure 3(a), with no numeric table or operating-point configurations. The oracle query set is defined but its decoding quality is never reported, so the router has no upper bound and the loss cannot be attributed between selection error and cache drift.
  4. Neither key hyperparameter is justified. H=5H=5 is chosen by Stage I Recall@5, the very metric Stage II shows to disagree with the closed-loop objective, and HH is never re-selected; no curve of HH against end-to-end score is given. RR is never swept at all, although within the 8.45% figure the 16 full refreshes account for 74% of the attention computation. The appendix also gives two mutually exclusive configurations (single block N=256N=256 with R=16R=16, and 16 blocks of 16 positions with no in-block refresh, where six positions per step is 37.5% of the block), and Table 1 does not say which one it uses.
Questions for authors
  1. Please give both baselines' full configurations (dLLM-Cache's KpK_p, KrK_r and update ratio; the d2Cache equivalents) and a batch-1 comparison table. Under their official settings, what TPS and scores do they reach in your environment?
  2. Which datasets do the router's training prompts come from, and do they overlap the five evaluation sets? After retraining with a prompt-level split (or leave-one-task-out), how do Recall@5 and end-to-end scores change?
  3. Please provide the numeric data behind Figure 3(a) (HH, RR, measured TFLOPs, per-benchmark scores). Where in that table does TraceQuery strictly dominate dLLM-Cache?
  4. What quality does decoding with the oracle query set achieve? How do quality and TPS move as RR goes from 4 to 64? Does HH remain 5 when re-selected under closed-loop evaluation? Does Table 1 use the single-block or the 16-block configuration?
  5. With 2-4 tokens committed per step, how is the reference trajectory defined and how do the conclusions change? The contributions claim one router across families while Section 5.4 states per-backbone retraining; what is the zero-transfer drop?
3 comments

Thank you for the detailed review. We appreciate your thoughtful critiques and will take them into account as we revise the paper. We agree that the paper needs a more reproducible baseline comparison, clearer reporting of the router’s training data, and narrower claims about the quality–efficiency trade-off.

Since submission, we added age-based requerying and dense warm-up on three backbones, fixed the shared harness, standardized EOS handling for instruction-tuned models, and enabled CUDA graphs for all methods. We have submitted a new version of the PDF and added all the new experimental results to the appendix.

Appendix D documents the setup and configurations. In the full batch-1 rerun, TraceQuery is faster than dLLM-Cache on average across tasks for all four backbones. Its mean accuracy is higher on both instruction-tuned backbones and lower on both base backbones (Appendix E, Table 4). We also benchmarked both official implementations. Official d2 Cache achieved a higher speedup in its own harness, with a 24-point BBH accuracy drop relative to its dense control on the 100-document subset (Appendix F, Table 6). We document the prompt-level split and report leave-one-task-out results (Appendices D and J, Tables 11–12). We also added closed-loop sweeps, an oracle selector, and replay experiments. (Appendices G–I, Tables 7–10).

Q1. Baseline configurations and official-code results.

Appendix D lists the shared-harness configurations. dLLM-Cache uses a 0.25 update ratio, refreshes the response every 8 steps, and refreshes the prompt every 64 steps. d2 Cache queries 32 masked positions per step with certainty-density (σ=10\sigma=10), also refreshes the just-committed tokens and the attention-rollout nucleus (p=0.1), and uses no periodic refresh. The full-set batch-1 comparison is in Appendix E, Table 4. On LLaDA-Instruct at batch size 1, the official dLLM-Cache and d2 Cache implementations achieve 2.2–2.9× and 4.6–5.0× speedups over their respective no-cache controls. Their GSM8K, BBH, and MBPP accuracy gaps are (+3.0/-5.6/-4.0) and (-3.0/-24.0/-7.0) points, respectively. Table 6 reports wall-clock speedup and accuracy gaps. Absolute TPS and scores for these official-code runs are still missing (Appendix F).

Q2. Training data and prompt-level generalization.

The trajectory data come from 51 prompts each from GSM8K, Minerva-Math, BBH, and TruthfulQA, plus 17 from HumanEval. MBPP and IFEval supply no training prompts. The split is by prompt, and all router-training prompts are excluded from evaluation (Appendix D). The leave-one-task-out study reports Recall@8, with a largest decrease of 0.009. End-to-end accuracy changes by at most 1.0 point across the three tasks tested (Appendix J, Tables 11–12).

Q3. Numeric operating points behind Figure 3(a).

The revision does not tabulate every point behind the original figure. The new sweep provides one matched-quality comparison on 100 LLaDA-Instruct GSM8K documents. Both methods score 74.0%. TraceQuery uses 0.094× dense FLOPs versus 0.187× for dLLM-Cache and takes 7.53 versus 10.88 seconds per sample. This operating point shows lower compute and latency at equal accuracy on that subset (Appendix G, Table 7).

Q4. Oracle quality, query budget, and block structure.

Across six tasks, oracle selection scores 48.3% on LLaDA-Instruct and 41.9% on LLaDA-Base, versus 49.5% and 43.1% for the learned router (Appendix H, Table 8). On the GSM8K subset, (h=4,8,16,32) yields 68%, 74%, 72%, and 66% accuracy at 5.19×, 5.12×, 4.95×, and 4.70× wall-clock speedup, respectively. The active block has 32 positions, so (h=64) is outside this sweep. Table 7 also sweeps the block refresh interval (Appendix G). In the revised full-set runs, (h=5) is used only for LLaDA-Base, and the other three backbones use (h=8). Both base models use one block, and both instruction-tuned models use 32-token blocks. These settings apply to the revised Table 4. The original Table 1 did not clearly identify its block configurations (Appendix D, Table 3).

Q5. Multi-token commits and zero transfer.

The router is trained per backbone on dense trajectories with one committed position per step. In the multi-token study, we reuse that router without retraining and compare accuracy against dense decoding at the same commit count. At four tokens per step, accuracy gaps range from −1.0 to −13.9 points across the three tasks, with 13.8–28.4× speedups relative to single-token dense decoding (Appendix N, Table 16). In the zero-transfer test, the LLaDA-Instruct router runs unchanged on LLaDA-1.5 and trails dense by 6.3 points on average, compared with 2.5 for d2 Cache and 5.0 for Fast-dLLM. We will clarify the paper’s claim to distinguish evaluation across model families with per-backbone training from this specific zero-transfer experiment (Appendix L, Table 14).

Thank you for the additional experiments. The revised results establish some gains over d²Cache, but I remain concerned about their magnitude.

In Table 4, the mean speedups over dense decoding are 3.06× versus 2.52× on LLaDA-Instruct and 2.56× versus 2.50× on LLaDA-Base, while TraceQuery’s mean accuracy is lower by 1.7 and 2.0 percentage points, respectively. On Dream-Instruct and Dream-Base, mean accuracy is within 0.1 points, with mean speedups of 2.80× versus 2.52× and 2.39× versus 2.29×. These results show improvements in some settings, but the remaining gains over the reimplemented d²Cache baseline are modest or accompanied by lower accuracy.

The original batching advantage has also been withdrawn. Although the revision adopts per-sample gathering for TraceQuery, Table 15 reports only its earlier union-based implementation. A matched-quality comparison of the revised implementations is still needed to establish a batching advantage.

I appreciate the improved experimental transparency, but the revised evidence does not establish a sufficiently substantial practical advantage over d²Cache to change my assessment. I maintain my weak rejection recommendation.

Best regards!

Dear Reviewer,

Thank you very much for your careful review and constructive comments. I agree that the current improvements are still not substantial enough, especially compared with d²Cache.

We will continue working on stronger improvements and more convincing comparisons, and hope to submit a much stronger version to a future venue.

Thank you again for your time, thoughtful feedback, and kind review.

Reviewer_TK2tOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes TraceQuery, a caching method for accelerating diffusion LLM inference. The main idea is to allocate recomputation based on the full-denoising trajectory, rather than representation error alone. A small MLP router is trained offline with a Plackett–Luce ranking loss to predict which positions will be unmasked soon. At inference time, it selects up to five positions to recompute, and the backbone chooses which token to commit from those candidates. The experiments cover the base and instruction-tuned versions of LLaDA and Dream.

Critical strengths
  1. The router design is simple. It uses inexpensive features and leaves the final token decision to the backbone. This is an appealing way to learn where to spend computation without adding a large inference module.

  2. The reduction in computation is substantial: approximately 14× fewer TFLOPs than full denoising in Table 1. The reported 2.1× throughput improvement at batch size 16 is also useful, although it comes with a loss in quality.

Critical weaknesses
  1. Quality degradation is substantial on several benchmarks. Calling it a “modest drop” understates the results. LLaDA-Base drops from 39.6 to 34.2 on MBPP and from 44.7 to 39.7 on BBH. Dream-Base drops from 37.8 to 30.5 on HumanEval. These losses make it difficult to assess the practical value of the reported acceleration. Figure 3 compares quality against FLOPs, but I would like to see wall-clock comparisons at matched quality. How much speedup remains when TraceQuery recovers most of the lost accuracy?

  2. Stating that all methods use the same hardware and precision is insufficient for reproduction. Please specify the hardware, precision, software stack, attention kernels and baseline implementations, Sparse inference is sensitive to these choices, so FLOP reductions alone do not establish runtime efficiency. In particular, d2Cache uses substantially fewer FLOPs than full denoising but runs at only 0.6× its throughput. This result needs an explanation.

  3. dKV-Cache-Greedy is a missing baseline. It also uses aggressive caching to trade quality for speed, making it directly relevant to this comparison. Although dKV is discussed in related work, neither variant appears in the experiments. A comparison with dKV-Cache-Greedy at matched quality and computation would help establish the contribution beyond existing caching strategies.

dKV-Cache: The Cache for Diffusion Language Models

  1. It is unclear how much learning the selector contributes. Section 5.4 includes router ablations, but does not answer the basic question of whether a learned selector is necessary. Attention-based ranking, distance-based selection, a fixed combination of these signals, and random selection would be useful baselines.

  2. LlaDA and Dream are too old.

Questions for authors
  1. What explains the 0.6× result for d2Cache? Please clarify how the setup differs from the original work, which reports acceleration, and whether the slowdown comes from batching, the chosen configuration, or implementation overhead.

  2. Table 1 reports batch size 16. What are the results at batch size 1? Since the paper argues that the advantage grows with batch size, the single-sample latency comparison is important.

  3. Under the same query budget and refresh schedule, how much does the learned router improve over unlearned selectors?

  4. If the query budget or refresh frequency is increased to recover near-baseline quality, what speedup remains?

1 comment

Thank you very much for the detailed review. We appreciate your thoughtful critiques and will take them into account as we revise the paper. We agree that “modest drop” understated several accuracy losses in the submission and that the paper needs clearer claims about the accuracy–latency trade-off, stronger baseline comparisons, and more implementation detail.

Since submission, we added an age-based rule that re-queries masked positions whose cached confidence is stale, and we introduced dense warm-up steps at the start of a block on three backbones. We also fixed several bugs in the shared backbone implementations and enabled CUDA graphs for each method’s per-step forward pass in the shared evaluation harness. We have submitted a new version of the PDF and added all the new experimental results to the appendix. Please download the latest version. Appendix D documents the hardware, BF16 precision, software stack, attention backend, and baseline configurations. Table 3 gives the revised TraceQuery settings. In the revised full-set evaluation at batch size 1, TraceQuery’s mean accuracy gaps to dense decoding are −3.6, −3.0, −1.1, and −1.8 percentage points on LLaDA-Instruct, LLaDA-Base, Dream-Instruct, and Dream-Base, respectively. Some task-level gaps remain substantial, including −7.8 points on LLaDA-Instruct GSM8K (Appendix E, Table 4). To quantify latency at matched quality, we added a wall-clock sweep in which TraceQuery matches dense accuracy on a 100-document GSM8K subset at 5.12× speedup (Appendix G, Table 7). We withdraw the original d2 Cache throughput result and our claim of a batching advantage over d2 Cache following the comparison with its official implementation (Appendix M, Table 15). We also added the authors’ dKV-Cache-Greedy implementation and its random-order dense control. Those runs report quality, compute, and speed at the tested settings. A matched-quality and matched-compute comparison with dKV-Cache-Greedy remains open (Appendix E, Table 5).

Q1. What explains the original 0.6× result for d2 Cache?

The original result came from our reimplementation at batch size 16, without CUDA graphs. It forwarded the union of query positions across samples, so each sample incurred extra computation when its query set differed from others in the batch. The official implementation gathers positions separately for each sample. In the updated benchmark, official d2 Cache takes 5.26 s per sample at batch size 16, a 7.6× speedup over its own batch-size-1 dense control. The rerun changes both query gathering and CUDA graph use, so it does not isolate their individual contributions (Appendix M, Table 15).

Q2. What are the results at batch size 1?

TraceQuery’s mean wall-clock speedups over dense decoding are 3.06×, 2.56×, 2.80×, and 2.39× on LLaDA-Instruct, LLaDA-Base, Dream-Instruct, and Dream-Base, respectively. It is faster than d2 Cache and dLLM-Cache on average on each backbone, and Fast-dLLM is faster than TraceQuery on all four (Appendix E, Table 4). The batch-size comparison is in Appendix M, Table 15.

Q3. How much does the learned router improve over unlearned selectors?

With the same query budget and refresh schedule, the learned router improves over random selection by 3.6 points on LLaDA-Instruct and 11.8 points on LLaDA-Base, with statistically significant gains on both. The study, which covers six tasks with 100 documents per task, finds no statistically significant gain over distance to the last commit, while achieved small accuracy gains over attention, and d2 Cache–style confidence × certainty density (Appendix H, Table 8).

Q4. What speedup remains when the query budget or refresh frequency is increased?

On the 100-document LLaDA-Instruct GSM8K subset, the deployed configuration with (h=8) matches dense accuracy at 74.0% and achieves 5.12× speedup. Increasing (h) to 16 or 32 yields 72.0% at 4.95× or 66.0% at 4.70×, respectively. Shortening the block refresh interval from 16 to 4 steps yields 71.0% at 5.09×. In this sweep, the deployed setting already matches dense quality, and the tested increases in query budget or refresh frequency do not improve it (Appendix G, Table 7).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.