acceptodds
Under review as a conference paper at ICLR 2027

SJD-VP: Speculative Jacobi Decoding with Verification Prediction for Autoregressive Image Generation

Abstract

Speculative Jacobi Decoding (SJD) has emerged as a promising method for accelerating autoregressive image generation. Despite its potential, existing SJD often suffers from the low acceptance rate issue of speculative tokens due to token selection ambiguity. Recent works attempt to mitigate this issue primarily from the perspective of revising token verification, but fail to fully exploit the dynamics of Jacobi decoding. In this paper, we conduct an in-depth analysis and find that tokens whose probabilities exhibit an increasing trend in the Jacobi dynamic are more likely to match the verification-accepted and correct token. Based on this, we propose a novel Speculative Jacobi Decoding with Verification Prediction (SJD-VP). The key idea is to leverage the change in token probabilities across Jacobi dynamics to guide the token drafting process, favoring tokens whose probabilities increase. This effectively predicts which tokens are likely to pass subsequent verification, boosting the acceptance rate. In particular, our SJD-VP is plug-and-play and can be seamlessly integrated into existing SJD variants. Extensive experiments on standard benchmarks demonstrate that our SJD-VP method consistently accelerates autoregressive decoding while maintaining image generation quality.

Primary area:
generative models
Keywords:
Autoregressive Image Generation, Speculative Jacobi Decoding, Verification Prediction, Inference Acceleration, Training-Free Decoding.

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_PERsOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the low acceptance rate of Speculative Jacobi Decoding (SJD; ICLR '25), a training-free method that accelerates autoregressive (AR) image generation by drafting and verifying multiple tokens in parallel. Prior SJD variants such as LANTERN and GSD raise acceptance by relaxing the verification criterion, but the drafting step still samples tokens without considering whether they will pass verification. The authors observe that, across Jacobi iterations, tokens whose probabilities increase consecutively are much more likely to be accepted, and also more likely to match the token produced by standard AR decoding. Their goal is to use this signal at the drafting step to predict which tokens will pass verification.

The proposed method, SJD-VP, changes only the drafting step and consists of two parts. First, it keeps each token's probabilities over the last few Jacobi iterations, computes an exponentially weighted reference, and scores each token by its log-probability gain over this reference. A mask keeps the score only for tokens that have grown for N consecutive iterations. Second, it treats this score as a prior and the current drafting probability as a likelihood, and reweights the top-k candidates to form the distribution from which draft tokens are sampled. Verification follows the original SJD rule, so the method can be attached to other SJD variants. The paper also gives a first-order analysis showing that the total variation distance between the draft and verification distributions decreases when the weighted directional accuracy Q_w exceeds 0.5.

Experiments use Lumina-mGPT on Parti-prompts and MS-COCO 2017, with additional results on LlamaGen, Emu3, and Janus-Pro. On Parti-prompts, SJD-VP reduces the NFE of SJD from 1035 to 721 and raises the latency speedup over AR decoding from 2.20× to 2.94×. On MS-COCO, FID stays at 30.77, compared with 30.78 for SJD. When combined with five other SJD variants, including LANTERN, GSD, and SJD-SV, it reduces NFE in every case. These results show that the drafting-side change consistently improves acceptance, although the latency gain is much smaller than the NFE gain on some models.

Critical strengths
  1. A simple and interesting observation about the dynamics of Jacobi decoding. Recent SJD variants improve acceptance by changing how tokens are verified. This paper instead looks at how each token's probability evolves across Jacobi iterations, which is information that SJD already computes but discards. It finds that most accepted tokens show consecutive probability growth before acceptance, 91.4% on average across Lumina-mGPT, LlamaGen, and Emu3. Tokens that keep growing are also likely to match the AR reference token, reaching 96.35% after three steps of growth. This moves attention from the verification step to the drafting step, and may motivate further work that exploits Jacobi trajectories.

  2. Consistent acceleration as a plug-in, with little change in generation quality. The method only changes how draft tokens are sampled and requires no training or additional model. On Lumina-mGPT, it reduces the NFE of SJD by about 30% on both benchmarks. It also reduces NFE when combined with each of five other SJD variants and on three other AR models.

Critical weaknesses
  1. The memory and computational overhead of the method is not properly characterized.

    • SJD-VP stores the probabilities of every vocabulary entry at every window position over the last L iterations. Its memory cost therefore grows with the vocabulary size, the window size, the history length, and the batch size. Recent AR image models use large vocabularies, so this overhead is non-negligible in large-batch settings. The paper reports no memory measurement.
    • The only overhead reported is a per-step time of 1.026 ms (Table 9). Yet the latencies in Table 5 imply a larger and model-dependent cost. The time per NFE rises from 34.9 to 42.3 ms on LlamaGen and from 53.7 to 67.5 ms on Emu3.
    • This overhead erodes the speedup. On LlamaGen, a 20% reduction in NFE yields only a 3% reduction in latency.
    • The overhead also grows with the window size. In Table 6, the time per NFE of SJD-VP rises from 37.5 ms at W = 16 to 46.6 ms at W = 64, while SJD stays at 34.8 ms. Since the strongest results use W = 64, the cost matters most exactly where the method looks best.
  2. The core observation and its theoretical justification are weakly grounded.

    • The method rests on the empirical finding in Fig. 2 that tokens with rising probabilities are likely to be accepted, but the paper does not explain why. Part of the effect may follow from the SJD acceptance rule itself, min(1, q/p), which always accepts a token whose probability has risen. Without the growth rate among rejected tokens, it is hard to judge how informative the signal is.
    • The conclusion of Sec. 3.4 is trivial. It states that the distance to the verification distribution decreases if Q_w exceeds 0.5, that is, if the predictor is better than a random guess. This restates the first-order condition and says nothing about how much acceptance improves.
    • The first-order Taylor expansion in Eq. 8 assumes a small shift m. However, the reweighting in Eq. 4 multiplies the probability of each growing token by p_t/p̄_t, which is not a small perturbation. If the shift were small enough for the expansion to hold, the correction would be too small to change acceptance. Appendix A.5 itself concedes that the analysis gives no guarantee for large shifts.
    • The analysis also does not describe the method as implemented. Draft tokens are sampled from p'_t but verified with the original p_t (Appendix A.3.1), so the acceptance rate is not governed by the distance between p'_t and q, as Appendix A.5 acknowledges. This mismatch also means that SJD-VP no longer preserves the target distribution, which the main text does not state.
Questions for authors
  • Please see the weaknesses. Could the authors report the peak memory and the time per NFE of SJD and SJD-VP for each model and window size, including batched settings?
  • Is the link between rising probability and acceptance specific to SJD, or is it also observed in general speculative decoding frameworks that use independent draft models?
  • Could the authors report results for a variant that verifies draft tokens with p'_t, which would preserve the target distribution?
13 comments
AuthorsOpenReview ↗

For W1.1. Memory Overhead

We thank the reviewer for highlighting the importance of memory overhead. SJD-VP maintains probability histories across the last LL Jacobi iterations for token positions within the decoding window. Therefore, the additional memory consumption depends on the vocabulary size, window size, history length, and batch size.

To quantify this overhead, we measure the peak GPU memory of SJD and SJD-VP during actual image generation under identical inference settings, with a batch size of 1.

Table 1: Peak GPU Memory and Time per NFE across Different Models and Window Sizes.

ModelWindowSJD Memory (GB)SJD-VP Memory (GB)Additional Memory (GB)SJD Time/NFE(ms)SJD-VP Time/NFE(ms)
Lumina-mGPT1618.6918.830.1434.837.5
Lumina-mGPT3218.7218.990.2734.940.3
Lumina-mGPT6418.8019.640.8435.046.6
LlamaGen162.332.350.0234.942.3
LlamaGen322.352.400.0535.144.6
LlamaGen642.392.470.0835.247.8
Emu31620.1920.490.3053.767.5
Emu33220.2620.700.4454.272.8
Emu36420.3921.551.1654.877.1

As shown in Table 1, memory overhead increases with the window size, reaching 0.84 GB (approximately 4.5% of SJD peak memory) for Lumina-mGPT at W=64W=64. These results indicate that SJD-VP introduces relatively modest additional memory consumption under the evaluated settings. We will include these measurements in the revised manuscript.

AuthorsOpenReview ↗

For W1.2. Runtime Overhead Discrepancy

We apologize for not clearly explaining the measurement settings in Table 9, which may have caused confusion. Specifically, Table 9 reports the runtime of the VP module measured independently, without executing the AR backbone. In contrast, Table 5 reports end-to-end inference performance, where the actual overhead of VP also depends on model-specific computational workloads and memory operations. Therefore, the isolated measurement of 1.026 ms in Table 9 does not directly reflect the additional time per NFE during actual model inference.

To clarify this discrepancy, we further profile the individual VP components during actual inference on three different models, as shown in Table 2.

Table 2: Runtime Overhead of SJD-VP during Actual Model Inference. (W=16)

ModelHistory Tracking (ms/NFE)Prior Estimation (ms/NFE)Bayesian Fusion (ms/NFE)Total VP (ms/NFE)
Lumina-mGPT0.10.81.72.6
LlamaGen0.21.25.67.0
Emu30.21.211.212.6

As shown in Table 2, Bayesian fusion accounts for most of the additional computation, while history tracking and prior estimation introduce relatively small overhead. The measured total overheads of 2.6, 7.0, and 12.6 ms/NFE closely align with the increases observed in Table 5 (approximately 2.6, 7.4, and 13.8 ms/NFE), explaining the previously noted discrepancy. We will clarify the two measurement settings and include this profiling analysis in the revised manuscript.

AuthorsOpenReview ↗

W1.3. NFE Reduction and Wall-Clock Speedup

We would like to clarify the discrepancy between NFE reduction and wall-clock speedup on LlamaGen. The relatively modest wall-clock improvement on LlamaGen is mainly due to the additional computation introduced by VP. Although SJD-VP reduces NFE by approximately 20%, the increased time per NFE partially offsets this benefit, resulting in a 3% latency reduction.

As shown in Table 2 of W1.2, the measured VP overhead is 7.0 ms/NFE, closely matching the 7.4 ms/NFE increase observed in Table 5 of the original manuscript. This explains the difference between NFE reduction and actual latency improvement. We will clarify this in the revised manuscript.

Reviewer_K1qjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SJD-VP, which reweights the drafting distribution in speculative Jacobi decoding using recent token-probability trends. Tokens with consecutively increasing probabilities receive greater drafting weight, aiming to improve verification acceptance and reduce decoding latency. The method is integrated into several SJD variants and evaluated across multiple autoregressive image models.

Critical strengths
  • SJD-VP is simple and easy to integrate into existing SJD variants, requiring no retraining or auxiliary model.
  • Despite its simplicity, it consistently reduces NFE and latency across multiple backbones and decoding methods while largely preserving FID and CLIP scores.
Critical weaknesses
  • The method is simple and practically useful, but the main contribution largely amounts to smoothing recent token probabilities and upweighting candidates with a consistent increasing trend. As a result, the conceptual advance beyond a reasonable drafting heuristic remains somewhat limited.
  • The evaluation mainly uses FID and CLIP-Score, which may not fully capture object count, attributes, and spatial relationships. Additional results on compositional benchmarks such as DPG-Bench or GenEval would provide a more complete assessment of generation quality.
  • SJD-VP changes the proposal distribution while retaining the original SJD verification correction. As acknowledged in Appendix A.3.1, the standard distribution-preservation guarantee therefore no longer applies. A more detailed analysis of the resulting sampling bias, or a verifier consistent with the modified proposal, would strengthen the method.
Questions for authors
  • Since SJD-VP samples from (pt′)(p'_t) but retains verification based on (pt)(p_t), could the authors clarify the resulting sampling bias and whether verification can be reformulated using (pt′)(p'_t) to recover distribution preservation?

  • How much runtime overhead comes from probability-history tracking and Bayesian fusion, particularly in settings where the wall-clock improvement is small?

  • If available, additional quality metrics or compositional benchmarks, such as DPG-Bench or GenEval, would provide a more complete evaluation beyond FID and CLIP-Score.

7 comments
AuthorsOpenReview ↗

For W1. Conceptual Contribution

We would like to clarify that smoothing recent token probabilities and upweighting candidates with consistently increasing probabilities are only implementation details of our approach. Actually, the novelty of our approach lies in three aspects:

(1) We identify an overlooked misalignment between drafting and verification in existing SJD variants: the drafting step does not explicitly predict whether candidate tokens will satisfy the subsequent verification criterion. This limits its ability to generate verification-accepted tokens and constrains the achievable decoding speedup.

(2) We identify the neglect of the verification prior as a key factor underlying this limitation. Our analysis in Sec. 3.1 reveals that tokens exhibiting consecutive probability growth across Jacobi iterations are more likely to pass verification and match the correct tokens. This finding provides an empirical basis for estimating the verification prior from probability histories and using it to guide token drafting.

(3) Building on this finding, we formulate token drafting within a Bayesian framework. Specifically, we treat the verification prior estimated from Jacobi probability growth patterns as the Bayesian prior and the current drafting probability as the conditional likelihood. We then derive a posterior distribution and use it as the final drafting distribution. This design combines current token probabilities with verification predictions inferred from probability histories, making anticipated verification outcomes an explicit consideration during candidate selection. It improves draft acceptance while favoring tokens consistent with reference AR generation, enabling faster decoding and improved generation quality.

We will revise the final version to articulate these three contributions more clearly and distinguish them from the operations used to implement them.

AuthorsOpenReview ↗

For W2. Additional Generation Quality Evaluation

We thank the reviewer for this valuable suggestion. To provide a more comprehensive evaluation beyond FID and CLIP-Score, we additionally evaluate SJD-VP on GenEval and DPG-Bench, which assess compositional generation quality, including object attributes, counts, and spatial relationships.

Table 1: Detailed Generation Quality Evaluation on GenEval and DPG-Bench.

(a) GenEval Evaluation.

MethodSingle Object ↑Two Objects ↑Counting ↑Colors ↑Position ↑Color Attribution ↑Overall ↑
AR (Lumina-mGPT)0.980.650.270.810.130.270.52
SJD0.970.660.300.800.120.230.51
SJD-VP0.980.660.310.800.110.220.51

(b) DPG-Bench Evaluation.

MethodGlobal ↑Entity ↑Attribute ↑Relation ↑Other ↑Overall ↑
AR (Lumina-mGPT)83.4282.5781.2387.9561.0076.82
SJD83.4082.2481.0787.2960.8176.46
SJD-VP83.4182.3281.0187.4160.9076.52

As shown in Table 1, SJD-VP maintains generation quality comparable to the original AR model and standard SJD on both benchmarks. We will incorporate these additional evaluations into the revised manuscript.

AuthorsOpenReview ↗

For W3. Sampling Bias and Verification Consistency

We thank the reviewer for this suggestion. We would like to clarify our design rationale. In SJD-VP, we retain the original SJD verification procedure with the aim of maintaining generation quality and facilitating integration into existing SJD implementations.

First, following the reviewer's suggestion, we provide a detailed analysis of the resulting sampling bias by comparing the output distributions of SJD and SJD-VP. In standard SJD, draft tokens are sampled from ptp_t, and verification preserves the target distribution qq. In contrast, SJD-VP samples from pt′p'_t while retaining the original verification rules. Its resulting single-step output distribution hth_t is given by

ht(x)=pt′(x)a(x)+(1−∑ypt′(y)a(y))r(x),h_t(x)=p'_t(x)a(x)+\left(1-\sum_y p'_t(y)a(y)\right)r(x),

where a(x)=min⁡(1,q(x)/pt(x))a(x)=\min(1,q(x)/p_t(x)) is the acceptance probability, and r(x)=[q(x)−pt(x)]+/∑y[q(y)−pt(y)]+r(x)=[q(x)-p_t(x)]_+/\sum_y[q(y)-p_t(y)]_+ is the residual distribution used upon rejection.

Since standard SJD preserves qq, we measure DTV(ht,q)D_{\mathrm{TV}}(h_t,q) to quantify the actual single-step sampling deviation relative to standard SJD. We also report DTV(pt′,pt)D_{\mathrm{TV}}(p'_t,p_t) to measure the change introduced to the drafting distribution. Both quantities are averaged over all verification steps for 500 randomly sampled prompts from PartiPrompts.

Table 2: Token-Level Sampling Distribution Fidelity across Different Models.

ModelDTV(ht,q)D_{\mathrm{TV}}(h_t,q) ↓DTV(pt′,pt)D_{\mathrm{TV}}(p'_t,p_t) ↓
Lumina-mGPT0.1030.325
LlamaGen0.1310.421
Emu30.1140.385

As shown in Table 2, although VP introduces changes to the drafting distribution (0.325–0.421), the resulting single-step sampling deviation after verification is substantially smaller (0.103–0.131).

Furthermore, following the reviewer's suggestion, we evaluate a verifier consistent with the modified proposal pt′p'_t. Specifically, we replace ptp_t with pt′p'_t in both the acceptance probability and rejection residual distribution, while keeping the target distribution qq unchanged. This preserves the target distribution under standard speculative sampling.

Table 3: Performance Comparison under Different Verification Strategies.

MethodNFE ↓Latency ↓CLIP-Score ↑
SJD1035.3036.07s32.09
SJD-VP720.6927.00s32.15
SJD-VP (Verification with pt′p'_t)718.4226.91s32.10

As shown in Table 3, verification with pt′p'_t achieves substantial acceleration over standard SJD while maintaining performance nearly identical to the original SJD-VP. These results indicate that adopting a distribution-preserving verification strategy does not noticeably affect the acceleration performance. We will incorporate these additional analyses and experiments into the revised manuscript.

Reviewer_1uueOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SJD-VP, a training-free, plug-and-play method for accelerating autoregressive image generation. It reweights draft probabilities to favor tokens whose probabilities consistently increase across Jacobi iterations. Experiments show reduced latency across several SJD variants with broadly comparable FID and CLIP scores. However, the method does not guarantee preservation of the original sampling distribution.

Critical strengths

The proposed algorithm further improves the efficiency of existing SJD methods, with a clear improvement in decoding speed.

Critical weaknesses
  1. Insufficient evidence of quality preservation. The proposed method uses a lossy speculative sampling strategy rather than one that preserves the target distribution. The authors therefore need to demonstrate that the resulting quality degradation is acceptable. However, the paper reports only FID and CLIP scores, both of which have substantial limitations. A measurable and theoretically justified assessment of the deviation from the target distribution is needed.

  2. Insufficient justification for the trade-off between quality and speed. A key motivation for speculative sampling is to accelerate generation while preserving the target distribution. Relaxing this guarantee raises the question of why one should prefer the proposed method over non-autoregressive alternatives. If lossy speculative sampling is the intended approach, the paper should compare its speedup with that of non-autoregressive methods under matched levels of quality degradation.

Questions for authors
  1. Why not directly improve the drafter’s proposal distribution? For example, if the target model’s probability pp for a particular token consistently increases across iterations, could the drafter increase its proposal probability for that token accordingly? Such an approach could potentially improve acceptance rates while retaining the distribution-preserving guarantee of speculative sampling through the standard verification procedure.

  2. Could the authors provide a more comprehensive evaluation of generation quality? In my experience, FID and CLIP scores alone are insufficient: images with reduced detail can sometimes achieve better FID scores. The proposed lossy sampling strategy may also affect the variance of generated outputs. Additional metrics and analyses are needed to establish whether these changes are acceptable.

  3. Can the authors establish a controllable upper bound on the total variation (TV) distance between the proposed sampling distribution and the original target distribution, and quantify the actual deviation? I would be willing to raise my score if the paper provided such evidence. As it stands, the method appears to introduce a loss of distributional fidelity whose magnitude is not adequately characterized or controlled.

6 comments
AuthorsOpenReview ↗

For W1. Sampling Distribution Fidelity

We thank the reviewer for these helpful comments. We would first like to clarify our design rationale. In SJD-VP, we retain the original SJD verification procedure rather than replacing ptp_t with the reweighted distribution pt′p'_t. This design aims to maintain generation quality by preserving the established verification framework, while allowing SJD-VP to be readily integrated into existing SJD implementations.

To address the reviewer's concerns regarding generation quality, we conduct additional evaluations on GenEval and DPG-Bench, comparing the original autoregressive model, SJD, and SJD-VP using the same Lumina-mGPT backbone.

Table 1: Detailed Generation Quality Evaluation on GenEval and DPG-Bench.

(a) GenEval Evaluation.

MethodSingle Object ↑Two Objects ↑Counting ↑Colors ↑Position ↑Color Attribution ↑Overall ↑
AR (Lumina-mGPT)0.980.650.270.810.130.270.52
SJD0.970.660.300.800.120.230.51
SJD-VP0.980.660.310.800.110.220.51

(b) DPG-Bench Evaluation.

MethodGlobal ↑Entity ↑Attribute ↑Relation ↑Other ↑Overall ↑
AR (Lumina-mGPT)83.4282.5781.2387.9561.0076.82
SJD83.4082.2481.0787.2960.8176.46
SJD-VP83.4182.3281.0187.4160.9076.52

As shown in Table 1, SJD-VP maintains generation quality comparable to standard SJD on both benchmarks, achieving the same overall GenEval score (0.51) and a slightly higher DPG-Bench score (76.52 vs. 76.46).

Following the reviewer's suggestion, we further quantify the sampling deviation introduced by Verification Prediction. In standard SJD, draft tokens are sampled from ptp_t, and the corresponding verification procedure ensures that the final output follows the target distribution qq. In contrast, SJD-VP samples draft tokens from the reweighted distribution pt′p'_t while retaining the original verification rules. Therefore, its final single-step output distribution, denoted by hth_t, is calculated as

ht(x)=pt′(x)a(x)+(1−∑ypt′(y)a(y))r(x),h_t(x)=p'_t(x)a(x)+\left(1-\sum_y p'_t(y)a(y)\right)r(x),

where a(x)=min⁡(1,q(x)/pt(x))a(x)=\min(1,q(x)/p_t(x)) is the original SJD acceptance probability, and r(x)=[q(x)−pt(x)]+/∑y[q(y)−pt(y)]+r(x)=[q(x)-p_t(x)]_+/\sum_y[q(y)-p_t(y)]_+ is the residual distribution used upon rejection. The two terms represent the probabilities of outputting token xx through direct acceptance and rejection sampling, respectively.

Since standard SJD produces qq after verification, DTV(ht,q)D_{\mathrm{TV}}(h_t,q) directly measures the single-step sampling deviation introduced by SJD-VP relative to standard SJD. For comparison, we also report DTV(pt′,pt)D_{\mathrm{TV}}(p'_t,p_t), which measures how much Verification Prediction modifies the original drafting distribution. We calculate both quantities across all verification steps for 500 randomly sampled prompts from PartiPrompts and report their average values in Table 2.

Table 2: Token-Level Sampling Distribution Fidelity across Different Models.

ModelDTV(ht,q)D_{\mathrm{TV}}(h_t,q) ↓DTV(pt′,pt)D_{\mathrm{TV}}(p'_t,p_t) ↓
Lumina-mGPT0.1030.325
LlamaGen0.1310.421
Emu30.1140.385

As shown in Table 2, although VP introduces considerable changes to the drafting distribution (0.325–0.421), the resulting single-step sampling deviation after verification is substantially smaller (0.103–0.131). This indicates that the original SJD verification procedure mitigates the impact of probability reweighting on the final sampling distribution.

Furthermore, the additional GenEval and DPG-Bench results in Table 1 show that SJD-VP maintains generation quality comparable to standard SJD. Together, these results quantify the sampling deviation introduced by VP while showing comparable generation quality despite the measured sampling deviation. We will incorporate these additional evaluations and experimental details into the revised manuscript.

AuthorsOpenReview ↗

For W2. Trade-off Between Quality and Speed

We would like to clarify that lossy speculative sampling does not necessarily lead to degraded image quality. In autoregressive image generation, multiple plausible token choices can produce comparable visual outcomes. Previous methods, such as COOL-SD [1] and CSD [2], also employ lossy speculative sampling to improve decoding efficiency while maintaining comparable generation quality. Our additional GenEval and DPG-Bench results in W1 further support this observation.

To address the reviewer's concern about preserving the target distribution, we further investigate whether the acceleration of SJD-VP requires lossy speculative sampling. Specifically, we evaluate a variant that replaces ptp_t with the reweighted distribution pt′p'_t in both the acceptance probability and rejection residual distribution, while keeping the target distribution qq unchanged. This preserves the target distribution under standard speculative sampling.

Table 3: Performance Comparison under Different Verification Strategies.

MethodNFE ↓Latency ↓CLIP-Score ↑
SJD1035.3036.07s32.09
SJD-VP720.6927.00s32.15
SJD-VP (Verification with pt′p'_t)718.4226.91s32.10

As shown in Table 3, verification with pt′p'_t achieves 718.42 NFE and 26.91s latency, compared with 1035.30 NFE and 36.07s for standard SJD, while maintaining comparable generation quality. Notably, its performance is nearly identical to the original SJD-VP (720.69 NFE and 27.00s latency). These results indicate that adopting a distribution-preserving verification strategy does not noticeably affect the acceleration performance.

We also appreciate the reviewer's suggestion regarding quality-matched comparisons with non-autoregressive methods. Such comparisons would provide a broader perspective on the trade-off between generation quality and speed. Our current work focuses on accelerating existing pretrained AR models without retraining, and we will consider this comparison in future work.

References

[1] Li et al., Annealed Relaxation of Speculative Decoding for Faster Autoregressive Image Generation. AAAI, 2026.

[2] Wang et al., CSD: Content-aware Speculative Decoding for Efficient Image Generation. ICML, 2026.

AuthorsOpenReview ↗

For Q1. Improving the Drafting Distribution

We thank the reviewer for this constructive suggestion. As discussed in W1, the reason we did not directly use the reweighted distribution pt′p'_t for verification was to retain the established SJD verification framework, with the aim of maintaining generation quality and compatibility with existing implementations.

Following the reviewer's suggestion, we additionally evaluate a variant that uses pt′p'_t for verification. As shown in Table 3 of W2, this variant preserves the target distribution while maintaining substantial acceleration. We will include this experiment in the main text of the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.