acceptodds
Under review as a conference paper at ICLR 2027

LatticeSpec: From Draft Chains to Verification Lattices for Efficient Diffusion Language Model Decoding

Abstract

Masked diffusion language models can predict updates at many positions in parallel, but turning these predictions into faster decoding requires deciding which updates to accept and in what order. Speculative decoding addresses this challenge by proposing updates in advance and verifying them against a target decoder. Yet a draft can contain the right updates in the wrong order: acceptance along a fixed draft chain stops at an ordering mismatch, leaving useful candidates unaccepted. Allowing alternative orders can recover this progress, but their verification costs can erase the benefit. We introduce LatticeSpec, a training-free, plug-and-play extension that selectively relaxes local draft ordering constraints within a small verification budget. It retains the existing drafting procedure and organizes alternative orders into a lattice, sharing verification states whenever different paths commit the same updates. Across 5 tasks and 6 model configurations from the LLaDA, Dream, and SDAR families, LatticeSpec improves end-to-end throughput over the corresponding SSD baselines by 5.7% on average. Separate analyses show that local relaxation recovers updates rejected by chain verification and that state sharing reduces the cost of checking the same alternatives. These results show that recovering more value from existing drafts through limited order flexibility can yield practical decoding gains.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Speculative Decoding, Diffusion Language Models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_JpZoOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes LatticeSpec, a verification extension for speculative decoding of masked diffusion language models. It relaxes local ordering constraints in an existing draft and shares verification states reached through different update orders. A node budget limits the additional computation. The main configuration allows one adjacent pair to swap order and reports an average 5.7% throughput improvement over SSD across six model configurations and five tasks.

Critical strengths
  1. The controlled analyses help explain the gains. Table 2 measures how much progress order relaxation recovers from the same candidates, while Table 3 separately measures the benefit of sharing repeated states.

  2. The method reports consistent throughput improvements across all 30 model–task settings, including online selection and verification costs. The paper also acknowledges that larger structures and more elaborate selection rules do not always help.

Critical weaknesses
  1. The benefit over prior work remains unclear. SSD already studies mix-order verification, and Spiffy uses shared-state graphs with budgeted pruning, but neither alternative is directly evaluated. Comparisons with the SSD chain and an unmerged tree do not establish the advantage over these methods. A simple threshold with fixed-position selection also performs within 1% of the main calibrated method (Table 9).

  2. Output preservation needs clarification. Appendix D.1 reports different intermediate trajectories on 7/124 problems, without explaining the cause or whether final outputs differ. Agreement with the serial reference should be checked for the main configurations. Given the modest speedups, repeated timings and uncertainty estimates would also strengthen the results.

Questions for authors
  1. How does LatticeSpec compare with SSD mix-order and Spiffy under matched decoding settings and verification budgets?

  2. What causes the trajectory differences on 7/124 problems in Appendix D.1? Do final outputs differ, and how often do the main configurations match the serial reference exactly?

  3. Can you provide repeated, interleaved timings and uncertainty estimates for the main SSD comparisons, especially those with the smallest gains?

6 comments
AuthorsOpenReview ↗

Response to Reviewer JpZo (1/3)

Thank you for recognizing the controlled mechanism analyses and the inclusion of online costs. We appreciate your suggestion to extend the evaluation to existing mix-order and shared-state verifiers. Below we address those comparisons, the trajectory differences, and the reliability of the reported runtime gains.


Q1: SSD mix-order and Spiffy under comparable verification budgets

A1: Thank you for suggesting direct comparisons with SSD mix-order and Spiffy. The new results below evaluate LatticeSpec against these existing alternatives under matched mean parallel verification-state counts.

Comparison protocol. We add comparisons on GSM8K and HumanEval using LLaDA-8B-Instruct (block size 16), Dream-v0-Instruct-7B (block size 16), and SDAR-4B-Chat configurations (block sizes 4 and 8). Following the experimental settings in Table 1, all runs use FP16 and a maximum generation length of 256.

All three methods support alternative verification orders, but their default configurations evaluate different numbers of states in parallel. Evaluating more states in parallel can itself accelerate decoding by allowing more candidate updates to be accepted in each round, giving methods with larger verification budgets an advantage. To ensure a fair comparison, we keep each baseline's reference configuration fixed and adjust LatticeSpec so that its mean count differs from the baseline's by at most 5%, measured relative to the baseline:

ΔS=SLatticeSpec−SbaselineSbaseline×100\Delta_S = \frac{S_{\mathrm{LatticeSpec}}-S_{\mathrm{baseline}}}{S_{\mathrm{baseline}}} \times 100\\%, \qquad |\Delta_S| \leq 5\\%.

Here, SS is the mean number of states in verification forwards with batch size greater than one. Keeping SS similar makes the comparison fairer by preventing a method from gaining an advantage simply by verifying more candidate states in parallel.

R3-Table 1: Comparison with SSD mix-order at matched mean parallel verification-state counts

ModelTaskLatticeSpec SSSSD mix-order SSΔS\Delta_SLatticeSpec TPSSSD mix-order TPSTPS gain
LLaDA-8B, B16GSM8K7.07137.3665−4.01%67.46562.849+7.34%
LLaDA-8B, B16HumanEval7.06497.3936−4.45%60.46356.617+6.79%
Dream-7B, B16GSM8K7.31247.3603−0.65%81.79976.356+7.13%
Dream-7B, B16HumanEval7.30467.3727−0.92%77.88772.493+7.44%
SDAR-4B, B8GSM8K6.26976.1105+2.61%21.66520.131+7.62%
SDAR-4B, B8HumanEval6.34306.0996+3.99%21.47319.773+8.60%

Strengthening SSD mix-order with rejoining. The original mix-order mechanism in SSD, Section 5.2 and Figure 4(b) does not support rejoining: its additional out-of-order branches are leaves, with no continuation back to the main chain. To strengthen this baseline, we additionally implement rejoining, which lets a side branch accept the previously skipped update, return to the equivalent main-chain state, and reuse its predictions and remaining path within the same verification round. This extension adds connections without adding verification states. It brings SSD mix-order closer to LatticeSpec's reuse of shared states across alternative orders.

R3-Table 2: Comparison with SSD mix-order + rejoining

ModelTaskLatticeSpec SSMix-order + rejoining SSΔS\Delta_SLatticeSpec TPSMix-order + rejoining TPSTPS gain
LLaDA-8B, B16GSM8K7.07137.3955−4.38%67.46566.172+1.95%
LLaDA-8B, B16HumanEval7.06497.4190−4.77%60.46359.882+0.97%
Dream-7B, B16GSM8K7.31247.3767−0.87%81.79980.655+1.42%
Dream-7B, B16HumanEval7.30467.3764−0.97%77.88776.665+1.59%
SDAR-4B, B8GSM8K6.26976.0758+3.19%21.66520.638+4.98%
SDAR-4B, B8HumanEval6.34306.0760+4.39%21.47320.235+6.12%

Across these six comparisons, LatticeSpec improves TPS by 6.79%–8.60% over original SSD mix-order and by 0.97%–6.12% over its rejoining extension. On LLaDA and Dream, these improvements are achieved with fewer parallel verification states.

Each model uses one LatticeSpec configuration across both tasks, with both methods evaluated on the same hardware using the same inference implementation. We did not find an SDAR block-4 configuration allowing a fair comparison with SSD mix-order, as even the closest configuration we found had a mean parallel state count about 28% lower than the baseline, so we excluded it from these comparisons.

AuthorsOpenReview ↗

Response to Reviewer JpZo (2/3)

R3-Table 3: Comparison with a fixed Spiffy reference configuration at matched mean parallel verification-state counts

Following Spiffy’s main experimental configuration, we use an offline draft-graph budget of D=10D=10 and dynamically prune it to D∗=3D^*=3 during inference, with separate calibration for each model. LatticeSpec uses one model-level hyperparameter configuration shared by GSM8K and HumanEval.

ModelTaskLatticeSpec SSSpiffy SSΔS\Delta_SLatticeSpec TPSSpiffy TPSTPS gain
LLaDA-8B, B16GSM8K3.83083.9358−2.67%58.31256.129+3.89%
LLaDA-8B, B16HumanEval3.82823.9369−2.76%56.29954.438+3.42%
Dream-7B, B16GSM8K4.10383.9352+4.28%86.35980.064+7.86%
Dream-7B, B16HumanEval4.11113.9332+4.52%82.88177.113+7.48%
SDAR-4B, B4GSM8K3.15363.1341+0.62%18.25815.832+15.32%
SDAR-4B, B4HumanEval3.15833.1099+1.56%17.98915.769+14.08%
SDAR-4B, B8GSM8K3.87493.8492+0.67%23.46823.403+0.28%
SDAR-4B, B8HumanEval3.87163.8479+0.62%23.38023.218+0.70%

LatticeSpec improves observed TPS by 3.42%–3.89% on LLaDA, 7.48%–7.86% on Dream, and 14.08%–15.32% on SDAR block-4. On LLaDA, it also evaluates fewer parallel states (2.67% fewer on GSM8K and 2.76% fewer on HumanEval). SDAR block-8 is close to parity (+0.28% and +0.70%).

These results quantify LatticeSpec’s practical gains against existing mix-order and shared-state methods at comparable mean verification-state counts.


Q2: Output preservation and the 7/124 trajectory differences

A2: Thank you for asking us to clarify both the trajectory differences and final-output agreement. All seven cases in Appendix D.1 retain identical final output tokens and stopping behavior. The differences concern intermediate trajectories under finite-precision batched verification. Finite-precision batch sensitivity is an implementation consideration for methods that batch verification states, including but not limited to SSD, Spiffy, and LatticeSpec. Note that LatticeSpec focuses on efficient verification structures, while numerical precision is a separate implementation issue.

We also already account for output agreement in the main evaluation: all timing comparisons use a common exact-match subset, on which every compared method produces the same output tokens and stopping behavior as the one-token reference. The reported speedups therefore compare identical generated outputs and stopping behavior. To investigate the precision explanation, we compared output agreement at BF16, FP16, and FP32 in a separate precision comparison using LLaDA-8B-Instruct on 64 GSM8K problems. The table reports exact agreement with the one-token reference at the same precision across four repeated runs.

R3-Table 4: Exact output agreement with the one-token reference across model precisions

MethodBF16FP16FP32
SSD46/6460/6464/64
LatticeSpec46/6460/6464/64
Unmerged tree49/6460/6464/64

Agreement improves from BF16 to FP16 and reaches 64/64 for all three methods in FP32. These results demonstrate that the observed output discrepancies disappear in FP32 for both SSD and LatticeSpec on these 64 examples, confirming that the phenomenon is not specific to LatticeSpec.

For your question about how often the main configurations match the serial reference, the table below gives the number and proportion of evaluation examples on which all compared methods match exactly. These are the common subsets used for the reported timing comparisons.

R3-Table 5: Common exact-match subsets used in the main experiments

Model / configurationGSM8K (131)HumanEval (100)MBPP (500)MT-Bench (80)
LLaDA-8B-Instruct (B16)113 (86.3%)85 (85.0%)425 (85.0%)56 (70.0%)
Dream-7B (B16)109 (83.2%)84 (84.0%)424 (84.8%)47 (58.8%)
SDAR-4B-Chat (B4)128 (97.7%)97 (97.0%)496 (99.2%)61 (76.2%)
SDAR-4B-Chat-b8 (B8)125 (95.4%)93 (93.0%)487 (97.4%)62 (77.5%)
SDAR-8B-Chat (B4)130 (99.2%)96 (96.0%)494 (98.8%)74 (92.5%)
SDAR-8B-Chat-b8 (B8)131 (100.0%)97 (97.0%)488 (97.6%)65 (81.2%)

Each cell reports the common exact-match subset size and its percentage of the evaluation examples (totals in column headers). B denotes generation block length.

AuthorsOpenReview ↗

Response to Reviewer JpZo (3/3)

Q3: Repeated, interleaved timings and uncertainty

A3: Thank you for asking whether the smaller throughput gains persist across repeated measurements. We measured LatticeSpec three times on A100 GPUs in FP16 using the main evaluation settings and compared its throughput with the SSD baseline measurements.

R3-Table 6: Mean TPS gains over SSD across three measurements

ModelMeasurement 1Measurement 2Measurement 3SD (pp)
LLaDA-8B, B16+7.55%+7.81%+7.56%0.15
Dream-7B, B16+4.04%+4.33%+4.16%0.15
SDAR-4B, B4+6.13%+6.09%+6.85%0.43
SDAR-4B, B8+4.96%+4.95%+5.19%0.13
SDAR-8B, B4+6.28%+6.25%+5.98%0.16
SDAR-8B, B8+4.89%+4.91%+5.14%0.14

Each measurement reports the arithmetic mean of per-task TPS gains over SSD for the corresponding model configuration. SD is the sample standard deviation across the three reported gains, expressed in percentage points (pp), and describes their observed variability. Comparisons use their respective common exact-match subsets.

LatticeSpec achieves higher TPS than SSD in all 30 model–task settings in each measurement, with consistent gains across model configurations.


Q4: The benefit of calibration over a simpler alternative

A4: Thank you for this constructive suggestion to clarify what our calibrated configuration contributes beyond a simpler alternative. Addressing this question requires distinguishing the roles of construction and selection, which can be chosen independently in LatticeSpec. Construction determines which alternative orders are allowed, while selection chooses which candidate blocks to relax within the verification budget. These components are modular: calibrated rank intervals are one construction option, rather than a requirement of LatticeSpec. We can therefore retain simple threshold construction and test whether gain-based selection improves on fixed-position selection.

Gain-based selection follows a simple principle: prioritize candidate blocks by their estimated acceptance gain per additional verification state. Our additional experiments show that this simple selection rule can improve throughput even when paired with simple threshold construction. This separates the benefit of selection from the choice of calibrated rank intervals.

Table 9 compares different combinations of construction and selection and shows that a simple threshold with fixed-position selection is competitive in the evaluated settings. Building on Table 9, the additional experiments examine whether the same simple construction can benefit from gain-based selection across different decoding settings. We keep threshold construction fixed and vary only the selection rule, while exploring different generation windows.

The window comparison below uses LLaDA and Dream on GSM8K and HumanEval with generation windows of 16 and 32. Both selection rules use threshold 0.1, draft length 4, at most one selected pair, and the same verification budget.

R3-Table 7: Fixed-position versus gain-based selection under the same threshold construction

ModelTaskWindowFixed-position TPSGain-based TPSTPS gain
LLaDA-8BGSM8K1661.95062.728+1.26%
LLaDA-8BGSM8K3258.51759.140+1.07%
LLaDA-8BHumanEval1658.61459.168+0.95%
LLaDA-8BHumanEval3255.73056.467+1.32%
Dream-7BGSM8K1686.29386.637+0.40%
Dream-7BGSM8K3278.15479.353+1.53%
Dream-7BHumanEval1682.62983.008+0.46%
Dream-7BHumanEval3273.92074.158+0.32%

With the same threshold construction, gain-based selection increases accepted tokens per verification round and reduces model forward calls in all eight settings, with observed TPS gains of 0.32%–1.53%. Confidence intervals estimated by resampling evaluation examples support positive gains in five of the eight settings.

These controlled comparisons show that gain-based selection can improve throughput over fixed-position selection under the same threshold construction. The observed gains highlight the effectiveness of combining simple threshold construction with gain-based selection. This supports LatticeSpec’s modular design: a simple threshold construction can be combined with gain-based selection while retaining the same shared-state verification framework.

Reviewer_kDjZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

LatticeSpec is a training-free extension to speculative verification for masked diffusion language models. It retains existing draft candidates while relaxing selected local ordering constraints. Alternative prefixes that commit the same candidate subset share a verification state, represented through an order-ideal lattice. An additional-node budget controls verification cost, and exact-match traversal preserves deterministic target decoding under the stated execution assumptions. Across five tasks and six model configurations, the method reports an average throughput improvement of 5.7% over SSD chain verification.

Critical strengths
  1. A clear formulation of order flexibility and verification cost. Representing committed subsets as order ideals gives a principled way to merge equivalent states and quantify the extra work introduced by relaxing draft order. The distinction between candidate availability and order-related rejection is useful for understanding diffusion speculation.

  2. Controlled experiments support the proposed mechanism. The shared-root analysis separates increased acceptance from reduced verification cost. Calibrated selection recovers 34.1% of the chain-to-candidate-bound gap, while state sharing reduces verification time relative to an unmerged tree representing the same alternatives. End-to-end improvements are reported across all 30 configuration–task pairs, including online construction and selection costs.

Critical weaknesses
  1. The practical contribution is insufficiently distinguished from closely related verification methods. The main comparison uses SSD’s chain configuration, although the paper explicitly identifies SSD’s mix-order verification as addressing out-of-order updates. Spiffy and PSD also support multiple paths and shared states, but receive no experimental comparison. This matters because the measured improvement over the chain is relatively modest. Moreover, Table 9 shows that a simple threshold with a fixed pair position performs within 1% of the calibrated configuration, leaving the benefit of the more elaborate calibration machinery unestablished.

  2. Output preservation is established conditionally but insufficiently audited in the implementation. Appendix A.3 requires merged states to contain equivalent decoder information and batched predictions to agree pointwise with serial evaluation. The paper does not report a comprehensive output-agreement audit against the one-token reference across the main experiments. Appendix D.1 additionally reports different intermediate trajectories on 7 of 124 problems, without clearly distinguishing differences in verification-round boundaries from differences in committed target updates. This does not invalidate the abstract proof, but leaves its applicability to the measured implementation uncertain.

Questions for authors
  1. How does LatticeSpec compare with SSD’s mix-order verification and a shared-graph baseline such as Spiffy under matched drafting, cache policies, and comparable verification budgets?

  2. Are final token sequences identical to the one-token reference across all main evaluation settings? For the 7/124 cases in Appendix D.1, do only round boundaries differ, or do committed position–token updates differ? Please clarify how cache state and numerical differences affect equivalence.

  3. Can repeated, interleaved timing measurements and uncertainty estimates be provided for the main SSD comparisons? Given improvements as small as 2.9%, distinguishing method gains from runtime variability could materially affect the assessment.

5 comments
AuthorsOpenReview ↗

Response to Reviewer kDjZ (1/3)

Thank you for recognizing the order-ideal formulation and the controlled separation of order recovery from state-sharing cost. We address the remaining questions through closest-baseline comparisons, an output-agreement evaluation, and repeated runtime measurements. We also clarify the role of calibrated selection.


Q1: SSD mix-order and Spiffy under comparable verification budgets

A1: Thank you for highlighting the need to distinguish LatticeSpec’s practical contribution from existing verification methods. We have added direct comparisons with SSD mix-order and Spiffy, with particular attention to comparable verification budgets.

Comparison protocol. We add comparisons on GSM8K and HumanEval using LLaDA-8B-Instruct (block size 16), Dream-v0-Instruct-7B (block size 16), and SDAR-4B-Chat configurations (block sizes 4 and 8). Following the experimental settings in Table 1, all runs use FP16 and a maximum generation length of 256.

All three methods support alternative verification orders, but their default configurations evaluate different numbers of states in parallel. Evaluating more states in parallel can itself accelerate decoding by allowing more candidate updates to be accepted in each round, giving methods with larger verification budgets an advantage. To ensure a fair comparison, we keep each baseline's reference configuration fixed and adjust LatticeSpec so that its mean count differs from the baseline's by at most 5%, measured relative to the baseline:

ΔS=SLatticeSpec−SbaselineSbaseline×100\Delta_S = \frac{S_{\mathrm{LatticeSpec}}-S_{\mathrm{baseline}}}{S_{\mathrm{baseline}}} \times 100\\%, \qquad |\Delta_S| \leq 5\\%.

Here, SS is the mean number of states in verification forwards with batch size greater than one. Keeping SS similar makes the comparison fairer by preventing a method from gaining an advantage simply by verifying more candidate states in parallel.

R2-Table 1: Comparison with SSD mix-order at matched mean parallel verification-state counts

ModelTaskLatticeSpec SSSSD mix-order SSΔS\Delta_SLatticeSpec TPSSSD mix-order TPSTPS gain
LLaDA-8B, B16GSM8K7.07137.3665−4.01%67.46562.849+7.34%
LLaDA-8B, B16HumanEval7.06497.3936−4.45%60.46356.617+6.79%
Dream-7B, B16GSM8K7.31247.3603−0.65%81.79976.356+7.13%
Dream-7B, B16HumanEval7.30467.3727−0.92%77.88772.493+7.44%
SDAR-4B, B8GSM8K6.26976.1105+2.61%21.66520.131+7.62%
SDAR-4B, B8HumanEval6.34306.0996+3.99%21.47319.773+8.60%

Strengthening SSD mix-order with rejoining. The original mix-order mechanism in SSD, Section 5.2 and Figure 4(b) does not support rejoining: its additional out-of-order branches are leaves, with no continuation back to the main chain. To strengthen this baseline, we additionally implement rejoining, which lets a side branch accept the previously skipped update, return to the equivalent main-chain state, and reuse its predictions and remaining path within the same verification round. This extension adds connections without adding verification states. It brings SSD mix-order closer to LatticeSpec's reuse of shared states across alternative orders.

R2-Table 2: Comparison with SSD mix-order + rejoining

ModelTaskLatticeSpec SSMix-order + rejoining SSΔS\Delta_SLatticeSpec TPSMix-order + rejoining TPSTPS gain
LLaDA-8B, B16GSM8K7.07137.3955−4.38%67.46566.172+1.95%
LLaDA-8B, B16HumanEval7.06497.4190−4.77%60.46359.882+0.97%
Dream-7B, B16GSM8K7.31247.3767−0.87%81.79980.655+1.42%
Dream-7B, B16HumanEval7.30467.3764−0.97%77.88776.665+1.59%
SDAR-4B, B8GSM8K6.26976.0758+3.19%21.66520.638+4.98%
SDAR-4B, B8HumanEval6.34306.0760+4.39%21.47320.235+6.12%

Across these six comparisons, LatticeSpec improves TPS by 6.79%–8.60% over original SSD mix-order and by 0.97%–6.12% over its rejoining extension. On LLaDA and Dream, these improvements are achieved with fewer parallel verification states.

Each model uses one LatticeSpec configuration across both tasks, with both methods evaluated on the same hardware using the same inference implementation. We did not find an SDAR block-4 configuration allowing a fair comparison with SSD mix-order, as even the closest configuration we found had a mean parallel state count about 28% lower than the baseline, so we excluded it from these comparisons.

AuthorsOpenReview ↗

Response to Reviewer kDjZ (2/3)

R2-Table 3: Comparison with a fixed Spiffy reference configuration at matched mean parallel verification-state counts

Following Spiffy’s main experimental configuration, we use an offline draft-graph budget of D=10D=10 and dynamically prune it to D∗=3D^*=3 during inference, with separate calibration for each model. LatticeSpec uses one model-level hyperparameter configuration shared by GSM8K and HumanEval.

ModelTaskLatticeSpec SSSpiffy SSΔS\Delta_SLatticeSpec TPSSpiffy TPSTPS gain
LLaDA-8B, B16GSM8K3.83083.9358−2.67%58.31256.129+3.89%
LLaDA-8B, B16HumanEval3.82823.9369−2.76%56.29954.438+3.42%
Dream-7B, B16GSM8K4.10383.9352+4.28%86.35980.064+7.86%
Dream-7B, B16HumanEval4.11113.9332+4.52%82.88177.113+7.48%
SDAR-4B, B4GSM8K3.15363.1341+0.62%18.25815.832+15.32%
SDAR-4B, B4HumanEval3.15833.1099+1.56%17.98915.769+14.08%
SDAR-4B, B8GSM8K3.87493.8492+0.67%23.46823.403+0.28%
SDAR-4B, B8HumanEval3.87163.8479+0.62%23.38023.218+0.70%

LatticeSpec improves observed TPS by 3.42%–3.89% on LLaDA, 7.48%–7.86% on Dream, and 14.08%–15.32% on SDAR block-4. On LLaDA, it also evaluates fewer parallel states (2.67% fewer on GSM8K and 2.76% fewer on HumanEval). SDAR block-8 is close to parity (+0.28% and +0.70%).

These results quantify LatticeSpec’s practical gains against existing mix-order and shared-state methods at comparable mean verification-state counts.


Q2: Output preservation and the 7/124 trajectory differences

A2: Thank you for asking how the output-equivalence assumptions relate to the measured implementation. Our evaluation protocol already accounts for output agreement: all timing comparisons in our main experiments use a common exact-match subset, on which every compared method produces exactly the same output tokens and stopping behavior as the corresponding one-token reference. This ensures that the reported speedups compare identical generated outputs and stopping behavior, avoiding timing differences caused by divergent outputs.

The seven cases in Appendix D.1 concern intermediate trajectory differences under finite-precision batched verification. All seven retain identical final output tokens and stopping behavior. Finite-precision batch sensitivity is an implementation consideration for methods that batch verification states, including but not limited to SSD, Spiffy, and LatticeSpec. Note that LatticeSpec focuses on efficient verification structures, while numerical precision is a separate implementation issue.

To investigate the precision explanation, we compared output agreement at BF16, FP16, and FP32 in a separate precision comparison using LLaDA-8B-Instruct on 64 GSM8K problems. The table reports exact agreement with the one-token reference at the same precision across four repeated runs.

R2-Table 4: Exact output agreement with the one-token reference across model precisions

MethodBF16FP16FP32
SSD46/6460/6464/64
LatticeSpec46/6460/6464/64
Unmerged tree49/6460/6464/64

Agreement improves from BF16 to FP16 and reaches 64/64 for all three methods in FP32. These results demonstrate that the observed output discrepancies disappear in FP32 for both SSD and LatticeSpec on these 64 examples, confirming that the phenomenon is not specific to LatticeSpec.

The table below summarizes output agreement through the common exact-match subsets used in the main experiments. The reported speedups apply to these subsets.

R2-Table 5: Common exact-match subsets used in the main experiments

Model / configurationGSM8K (131)HumanEval (100)MBPP (500)MT-Bench (80)
LLaDA-8B-Instruct (B16)113 (86.3%)85 (85.0%)425 (85.0%)56 (70.0%)
Dream-7B (B16)109 (83.2%)84 (84.0%)424 (84.8%)47 (58.8%)
SDAR-4B-Chat (B4)128 (97.7%)97 (97.0%)496 (99.2%)61 (76.2%)
SDAR-4B-Chat-b8 (B8)125 (95.4%)93 (93.0%)487 (97.4%)62 (77.5%)
SDAR-8B-Chat (B4)130 (99.2%)96 (96.0%)494 (98.8%)74 (92.5%)
SDAR-8B-Chat-b8 (B8)131 (100.0%)97 (97.0%)488 (97.6%)65 (81.2%)

Each cell reports the common exact-match subset size and its percentage of the evaluation examples (totals in column headers). B denotes generation block length.

AuthorsOpenReview ↗

Response to Reviewer kDjZ (3/3)

Q3: Repeated, interleaved timings and uncertainty

A3: Thank you for emphasizing the need to distinguish method gains from runtime variability. We measured LatticeSpec three times on A100 GPUs in FP16 using the main evaluation settings and compared its throughput with the SSD baseline measurements.

R2-Table 6: Mean TPS gains over SSD across three measurements

ModelMeasurement 1Measurement 2Measurement 3SD (pp)
LLaDA-8B, B16+7.55%+7.81%+7.56%0.15
Dream-7B, B16+4.04%+4.33%+4.16%0.15
SDAR-4B, B4+6.13%+6.09%+6.85%0.43
SDAR-4B, B8+4.96%+4.95%+5.19%0.13
SDAR-8B, B4+6.28%+6.25%+5.98%0.16
SDAR-8B, B8+4.89%+4.91%+5.14%0.14

Each measurement reports the arithmetic mean of per-task TPS gains over SSD for the corresponding model configuration. SD is the sample standard deviation across the three reported gains, expressed in percentage points (pp), and describes their observed variability. Comparisons use their respective common exact-match subsets.

LatticeSpec achieves higher TPS than SSD in all 30 model–task settings in each measurement. The model-level mean gains have observed standard deviations of 0.13–0.43 percentage points.


Q4: The benefit of calibration over a simpler alternative

A4: Thank you for this constructive suggestion to clarify what our calibrated configuration contributes beyond a simpler alternative. Addressing this question requires distinguishing the roles of construction and selection, which can be chosen independently in LatticeSpec. Construction determines which alternative orders are allowed, while selection chooses which candidate blocks to relax within the verification budget. These components are modular: calibrated rank intervals are one construction option, rather than a requirement of LatticeSpec. We can therefore retain simple threshold construction and test whether gain-based selection improves on fixed-position selection.

Gain-based selection follows a simple principle: prioritize candidate blocks by their estimated acceptance gain per additional verification state. Our additional experiments show that this simple selection rule can improve throughput even when paired with simple threshold construction. This separates the benefit of selection from the choice of calibrated rank intervals.

Table 9 compares different combinations of construction and selection and shows that a simple threshold with fixed-position selection is competitive in the evaluated settings. The additional experiments therefore test whether the selection component offers a benefit without requiring calibrated rank intervals for construction. We keep threshold construction fixed and vary only the selection rule, while exploring different generation windows.

The window comparison below uses LLaDA and Dream on GSM8K and HumanEval with generation windows of 16 and 32. Both selection rules use threshold 0.1, draft length 4, at most one selected pair, and the same verification budget.

R2-Table 7: Fixed-position versus gain-based selection under the same threshold construction

ModelTaskWindowFixed-position TPSGain-based TPSTPS gain
LLaDA-8BGSM8K1661.95062.728+1.26%
LLaDA-8BGSM8K3258.51759.140+1.07%
LLaDA-8BHumanEval1658.61459.168+0.95%
LLaDA-8BHumanEval3255.73056.467+1.32%
Dream-7BGSM8K1686.29386.637+0.40%
Dream-7BGSM8K3278.15479.353+1.53%
Dream-7BHumanEval1682.62983.008+0.46%
Dream-7BHumanEval3273.92074.158+0.32%

With the same threshold construction, gain-based selection increases accepted tokens per verification round and reduces model forward calls in all eight settings, with observed TPS gains of 0.32%–1.53%. Confidence intervals estimated by resampling evaluation examples support positive gains in five of the eight settings.

These controlled comparisons show that gain-based selection can improve throughput over fixed-position selection under the same threshold construction. The observed gains highlight the effectiveness of combining simple threshold construction with gain-based selection. This supports LatticeSpec’s modular design: a simple threshold construction can be combined with gain-based selection while retaining the same shared-state verification framework.

Reviewer_xz48OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

LatticeSpec accelerates deterministic decoding in masked diffusion language models by relaxing local draft-order constraints and sharing verification states across orderings that reach the same committed subset. It selects relaxations under a small state budget and evaluates end-to-end throughput on LLaDA, Dream, and SDAR configurations.

Critical strengths

The paper identifies a practical source of wasted draft progress and gives a clear mechanism for sharing verification states. The evaluation reports end-to-end timing across several models and tasks, including online construction and selection overhead.

Critical weaknesses

The core idea of allowing out-of-order draft verification is already present in SSD’s mix-order verification. The paper acknowledges this, but its main throughput comparison is against SSD’s greedy chain rather than the existing mix-order variant. Consequently, the reported 5.7% average improvement over the chain does not establish the benefit of the proposed state sharing and selection relative to a relevant prior method. The paper’s contribution appears incremental, and without this direct comparison its practical significance is difficult to assess.

Questions for authors

Why is SSD’s mix-order verification not included as a direct end-to-end baseline? Please compare it with LatticeSpec under matched models, hardware, drafting and cache policies, and a comparable verification-state or compute budget, reporting throughput, latency, acceptance and verification cost. If exact matching is not possible, please explain the differences and their expected effect. This comparison would materially affect my assessment.

2 comments
AuthorsOpenReview ↗

Response to Reviewer xz48 (Part 1/2)

Thank you for recognizing the practical source of wasted draft progress and the end-to-end accounting of online overhead. Your suggestion motivates an expanded evaluation against SSD mix-order under matched mean parallel verification-state counts, as detailed below.


Q1: Direct comparison with SSD mix-order

A1: We sincerely appreciate this insightful and constructive suggestion. Your emphasis on a direct comparison with SSD mix-order identifies an important opportunity to make the empirical contribution of LatticeSpec clearer and more compelling. We have added direct comparisons with SSD mix-order and an enhanced rejoining variant, followed by an additional comparison with Spiffy.

Comparison protocol. We add comparisons on GSM8K and HumanEval using LLaDA-8B-Instruct (block size 16), Dream-v0-Instruct-7B (block size 16), and SDAR-4B-Chat (block size 8). Following the experimental settings in Table 1, all runs use FP16 and a maximum generation length of 256.

All three methods support alternative verification orders, but their default configurations evaluate different numbers of states in parallel. Evaluating more states in parallel can itself accelerate decoding by allowing more candidate updates to be accepted in each round, giving methods with larger verification budgets an advantage. To ensure a fair comparison, we keep each baseline's reference configuration fixed and adjust LatticeSpec so that its mean count differs from the baseline's by at most 5%, measured relative to the baseline:

ΔS=SLatticeSpec−SbaselineSbaseline×100\Delta_S = \frac{S_{\mathrm{LatticeSpec}}-S_{\mathrm{baseline}}}{S_{\mathrm{baseline}}} \times 100\\%, \qquad |\Delta_S| \leq 5\\%.

Here, SS is the mean number of states in verification forwards with batch size greater than one. Keeping SS similar makes the comparison fairer by preventing a method from gaining an advantage simply by verifying more candidate states in parallel.

R1-Table 1: Comparison with SSD mix-order at matched mean parallel verification-state counts

ModelTaskLatticeSpec SSSSD mix-order SSΔS\Delta_SLatticeSpec TPSSSD mix-order TPSTPS gain
LLaDA-8B, B16GSM8K7.07137.3665−4.01%67.46562.849+7.34%
LLaDA-8B, B16HumanEval7.06497.3936−4.45%60.46356.617+6.79%
Dream-7B, B16GSM8K7.31247.3603−0.65%81.79976.356+7.13%
Dream-7B, B16HumanEval7.30467.3727−0.92%77.88772.493+7.44%
SDAR-4B, B8GSM8K6.26976.1105+2.61%21.66520.131+7.62%
SDAR-4B, B8HumanEval6.34306.0996+3.99%21.47319.773+8.60%

Strengthening SSD mix-order with rejoining. The original mix-order mechanism in SSD, Section 5.2 and Figure 4(b) does not support rejoining: its additional out-of-order branches are leaves, with no continuation back to the main chain. To strengthen this baseline, we additionally implement rejoining, which lets a side branch accept the previously skipped update, return to the equivalent main-chain state, and reuse its predictions and remaining path within the same verification round. This extension adds connections without adding verification states. It brings SSD mix-order closer to LatticeSpec's reuse of shared states across alternative orders.

R1-Table 2: Comparison with SSD mix-order + rejoining

ModelTaskLatticeSpec SSMix-order + rejoining SSΔS\Delta_SLatticeSpec TPSMix-order + rejoining TPSTPS gain
LLaDA-8B, B16GSM8K7.07137.3955−4.38%67.46566.172+1.95%
LLaDA-8B, B16HumanEval7.06497.4190−4.77%60.46359.882+0.97%
Dream-7B, B16GSM8K7.31247.3767−0.87%81.79980.655+1.42%
Dream-7B, B16HumanEval7.30467.3764−0.97%77.88776.665+1.59%
SDAR-4B, B8GSM8K6.26976.0758+3.19%21.66520.638+4.98%
SDAR-4B, B8HumanEval6.34306.0760+4.39%21.47320.235+6.12%

Across these six comparisons, LatticeSpec improves TPS by 6.79%–8.60% over original SSD mix-order and by 0.97%–6.12% over its rejoining extension. On LLaDA and Dream, these improvements are achieved with fewer parallel verification states.

Each model uses one LatticeSpec configuration across both tasks, with both methods evaluated on the same hardware using the same inference implementation. We did not find an SDAR block-4 configuration allowing a fair comparison with SSD mix-order, as even the closest configuration we found had a mean parallel state count about 28% lower than the baseline, so we excluded it from these comparisons.

AuthorsOpenReview ↗

Response to Reviewer xz48 (Part 2/2)

Additional comparison with Spiffy. To extend this evaluation to a shared-state verification method, we also compare with Spiffy using the same state-count matching protocol. This comparison additionally includes SDAR block-4, for which a matched configuration was found against Spiffy.

R1-Table 3: Comparison with a fixed Spiffy reference configuration at matched mean parallel verification-state counts

Following Spiffy’s main experimental configuration, we use an offline draft-graph budget of D=10D=10 and dynamically prune it to D∗=3D^*=3 during inference, with separate calibration for each model. LatticeSpec uses one model-level hyperparameter configuration shared by GSM8K and HumanEval. All completed rows satisfy the 5% state-count tolerance.

ModelTaskLatticeSpec SSSpiffy SSΔS\Delta_SLatticeSpec TPSSpiffy TPSTPS gain
LLaDA-8B, B16GSM8K3.83083.9358−2.67%58.31256.129+3.89%
LLaDA-8B, B16HumanEval3.82823.9369−2.76%56.29954.438+3.42%
Dream-7B, B16GSM8K4.10383.9352+4.28%86.35980.064+7.86%
Dream-7B, B16HumanEval4.11113.9332+4.52%82.88177.113+7.48%
SDAR-4B, B4GSM8K3.15363.1341+0.62%18.25815.832+15.32%
SDAR-4B, B4HumanEval3.15833.1099+1.56%17.98915.769+14.08%
SDAR-4B, B8GSM8K3.87493.8492+0.67%23.46823.403+0.28%
SDAR-4B, B8HumanEval3.87163.8479+0.62%23.38023.218+0.70%

LatticeSpec improves observed TPS by 3.42%–3.89% on LLaDA, 7.48%–7.86% on Dream, and 14.08%–15.32% on SDAR block-4. On LLaDA, it also evaluates fewer parallel states (2.67% fewer on GSM8K and 2.76% fewer on HumanEval). SDAR block-8 is close to parity (+0.28% and +0.70%).

The SSD mix-order comparisons directly address your suggestion, and the Spiffy results provide additional evidence of LatticeSpec’s throughput advantage across several model configurations. Thank you again for encouraging a stronger evaluation.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Contributions and Manuscript Revisions

We sincerely thank the reviewers and the Area Chair for their time and thoughtful consideration. We appreciate the reviewers' recognition of the following strengths:

  • Clear motivation and formulation. The paper identifies wasted draft progress caused by ordering constraints and provides a principled formulation for sharing equivalent verification states (xz48, kDjZ).
  • Controlled evidence for the mechanism. The analyses separately establish the progress recovered through order relaxation and the verification cost saved through state sharing (kDjZ, JpZo).
  • End-to-end evaluation. The experiments cover multiple model families and tasks and include online construction and selection costs (all reviewers), with consistent throughput improvements across all 30 model–task settings (kDjZ, JpZo).

Following the reviewers' suggestions, we have expanded the experiments and revised the manuscript. The revisions are highlighted in green. The main updates are:

  1. Comparisons with stronger baselines (all reviewers). The Abstract and Introduction now highlight these broader baseline comparisons. Section 4.3 and Appendix D.1 now compare LatticeSpec with SSD mix-order, an enhanced rejoining variant, and Spiffy on GSM8K and HumanEval under a fair comparison protocol. These comparisons establish practical gains over alternative-order and shared-state baselines, including 6.79–8.60% higher throughput than SSD mix-order across the six evaluated model–task settings.

  2. Output agreement and numerical precision (kDjZ, JpZo). Appendices B.4–B.5 clarify the common exact-match timing protocol, report the evaluation subset sizes, and add a precision diagnostic that reaches full output agreement for all three evaluated methods on 64 examples in FP32. Appendix D.3 also clarifies that all seven previously discussed trajectory differences preserve identical final output tokens and stopping behavior.

  3. Repeated runtime measurements (kDjZ, JpZo). Appendix D.2 reports three measurements for both the main evaluation and the comparisons against rejoining and Spiffy. Gains remain positive in all 30 main-evaluation settings and all seven stronger-baseline comparisons in each measurement, supporting the stability of the throughput improvements.

  4. The role of construction and selection (kDjZ, JpZo). Sections 3.4 and 4.6 clarify the modular separation of order construction, block selection, and shared-state verification. New controlled experiments in Appendix D.4 show that gain-based selection improves measured throughput with simple threshold construction across two models, two tasks, and two generation windows.

Our individual responses provide the corresponding results and explanations. We thank the reviewers again for suggestions that have strengthened the empirical evidence and clarified the presentation of LatticeSpec.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.