acceptodds
Under review as a conference paper at ICLR 2027

The Shape of Speculation: Efficient Draft Topology for Speculative Jacobi Decoding in Autoregressive Image Generation

Abstract

Speculative Jacobi Decoding (SJD) has recently emerged as an efficient way to accelerate autoregressive image generation by refining multiple tokens in parallel. Recent multi-candidate extensions further improve SJD by introducing alternative continuations, yet their efficiency depends critically on how auxiliary candidates are organized. We argue that existing multi-candidate SJD strategies treat additional speculation as uniformly beneficial, overlooking that the contribution of candidate expansions varies substantially along the Jacobi trajectory. To address this issue, we propose FlexJacobi, a training-free framework that rethinks draft topology construction from the perspective of speculative candidate efficiency. Instead of treating all candidate expansions equally, FlexJacobi devises Verification-aware Topology Shaping (VTS), which derives candidate priorities from current draft distributions and distributes a fixed auxiliary-node budget across the Jacobi trajectory to favor more promising continuation slots. To further improve the effectiveness of these continuations, Context-guided Hybrid Drafting (CHD) is further proposed, which exploits predictive-context consistency with previously generated visual tokens to select continuation entry tokens, while reusing cached draft distributions for the continuation tails. Comprehensive experiments show that FlexJacobi consistently improves decoding efficiency while maintaining competitive generation quality, matching or surpassing SJD-PAC with only of its auxiliary nodes.

Primary area:
generative models
Keywords:
Autoregressive Image Generation, Speculative Jacobi Decoding, Inference Acceleration

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_msBaOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes FlexJacobi, a training-free framework for improving the efficiency of multi-candidate Speculative Jacobi Decoding (SJD) in autoregressive image generation. The key idea is to improve speculative candidate utilization by dynamically organizing auxiliary continuations along the Jacobi trajectory. Specifically, the proposed Verification-aware Topology Shaping (VTS) allocates a fixed auxiliary-node budget based on draft-distribution statistics, while Context-guided Hybrid Drafting (CHD) leverages historical predictive-context similarity to select continuation entry tokens and reuses cached draft distributions for subsequent tokens. The resulting candidates are jointly verified without additional drafting forward passes. Experiments on Lumina-mGPT and Janus-Pro demonstrate consistent improvements in decoding efficiency while maintaining competitive generation quality.

Critical strengths

1.The paper identifies a meaningful limitation of existing multi-candidate SJD methods: allocating more speculative candidates does not necessarily translate into longer accepted sequences. The proposed approach of redistributing a limited candidate budget along the Jacobi trajectory is intuitive and addresses an important efficiency problem.

2.FlexJacobi improves draft construction without requiring additional training or a separate draft model. Both VTS and CHD reuse information already available during decoding, making the framework lightweight and practical. The proposed design also provides a reasonable way to exploit the characteristics of visual tokens.

3.The experiments cover two autoregressive image generation models, multiple benchmarks, and a range of competitive baselines. The results demonstrate consistent improvements in decoding latency and NFE while maintaining competitive image quality. The auxiliary-budget ablations and compatibility experiments with relaxed verification methods further support the practical effectiveness and applicability of the framework.

4.The paper is well written and logically organized. The motivation is clearly presented, the proposed components are explained in an accessible manner, and the figures help illustrate the overall methodology. The experimental section is also well structured and easy to follow.

Critical weaknesses
  1. The paper argues that SJD-PAC's shallow local candidate tree provides redundant recovery opportunities as alternatives accumulate within the same region, while redistributing auxiliary continuations along the Jacobi trajectory can improve accepted length. However, the current experiments do not directly demonstrate how additional candidates in SJD-PAC lead to redundant recovery opportunities or how their contribution to accepted length gradually saturates. This lack of empirical evidence raises concerns about the validity of the paper's central motivation.

  2. The authors use the product of the maximum probabilities in the draft distributions as a draft-based proxy for the likelihood that the prefix will be accepted. I interpret this choice as implicitly assuming that higher prefix support corresponds to higher prefix acceptance rates. However, the actual acceptance probability depends on both the draft and target distributions, and higher draft confidence does not necessarily imply a higher acceptance probability. Therefore, the authors' current discussion does not sufficiently justify this implicit assumption underlying the proposed scoring criterion.

My main concerns are the insufficient empirical support for the central motivation and the lack of justification for the proposed scoring criterion in VTS. While I appreciate the proposed idea and the promising experimental results, the current evidence does not sufficiently support the underlying assumptions of the method.

Questions for authors

1.Could the authors provide more concrete empirical evidence supporting the claim that additional candidates in SJD-PAC's shallow local candidate tree provide redundant recovery opportunities and that their contribution to accepted length gradually saturates?

2.Could the authors further explain why the product of the maximum probabilities in the draft distributions is a reasonable draft-based proxy for prefix acceptance? In particular, is there empirical evidence showing that higher prefix support corresponds to higher prefix acceptance rates during verification?

If these concerns are satisfactorily addressed, I would be inclined to support acceptance of this paper.

7 comments

W1 and Q1: Could the authors provide more concrete empirical evidence supporting the claim that additional candidates in SJD-PAC's shallow local candidate tree provide redundant recovery opportunities and that their contribution to accepted length gradually saturates?

A1. Thank you for your valuable question. We measured the contribution of each additional auxiliary node to the mean accepted length in SJD-PAC. The results show a clear decline as the auxiliary budget increases. For example, on Lumina-mGPT / PartiPrompts, the contribution per node decreases from 0.02846 (R:0→4R:0\to4) to 0.00613 (R:10→18R:10\to18). This directly demonstrates the diminishing contribution of additional candidates to accepted length. Details are shown below:

Mean Accepted Length Gain per Auxiliary Node in SJD-PAC. We fix the main path and tree depth D=3D=3 of SJD-PAC, while increasing the branching factor from K=2K=2 to K=4K=4, corresponding to auxiliary-node budgets of R=4,10,18R={4,10,18}. We measure the mean accepted length MAT(R)MAT(R) as the average number of draft tokens accepted per verification round with RR auxiliary nodes. To measure the contribution of additional auxiliary nodes, we calculate the gain in mean accepted length per added node:

g(R1!→!R2)=MAT(R2)−MAT(R1)R2−R1.g(R_1\\!\to\\!R_2)=\frac{MAT(R_2)-MAT(R_1)}{R_2-R_1}.

SJD-PAC: Model / BenchmarkΔ4(0→4)\Delta4 (0\to4)Δ6(4→10)\Delta6(4\to10)Δ8(10→18)\Delta8(10\to18)
Lumina-mGPT / Parti-Prompts0.028460.010920.00613
Janus-Pro-7B / Parti-Prompts0.026310.011870.00560
Janus-Pro-7B / MS-COCO 20170.024600.010370.00499
Lumina-mGPT / MS-COCO 20170.026630.010950.00693

Results analysis. As the shallow candidate tree expands, the marginal gain per auxiliary node decreases consistently. Across all four settings, the gain drops substantially from the first expansion (R:0→4R:0\to4) to the last (R:10→18R:10\to18). These results show that additional candidates contribute progressively less to accepted length, consistent with increasingly redundant recovery opportunities in SJD-PAC's shallow local tree.


W2 and Q2: (1) Could the authors further explain why the product of the maximum probabilities in the draft distributions is a reasonable draft-based proxy for prefix acceptance? (2) In particular, is there empirical evidence showing that higher prefix support corresponds to higher prefix acceptance rates during verification?

A2. Thank you for your valuable question. In order to demonstrate the advantage of VTS, we additionally compared VTS with two alternative Strategies including Local-prob, Entropy-prob, where Local-prob, Entropy modify the product of maximum draft probabilities in the VTS scoring criterion. Moreover, we also analyzed the relationship between VTS priorities and actual side branch acceptance rates. The results show that higher VTS priorities are associated with higher side-branch acceptance rates, providing empirical support for the proposed scoring criterion.

(1) Comparison of Allocation Strategies. We compare VTS against two alternative strategies under identical experimental settings. The results are presented below.

ModelMetricVTS (ours)Local-probEntropy
Janus-Pro-7BLatency (s) ↓4.5355.8275.789
NFE ↓164.85208.31206.70
Lumina-mGPTLatency (s) ↓17.01820.29320.336
NFE ↓458.91530.11529.75

Specifically, the two alternatives are:

  • Local-prob: Uses only the local draft probability mikm_i^k, removing the prefix-support term from VTS.
  • Entropy: Uses H(qi)mikH(q_i)m_i^k, favoring alternatives at positions with higher draft uncertainty.

VTS achieves the lowest latency and NFE among all strategies, demonstrating the effectiveness of VTS.

(2) Higher VTS is associated with higher acceptance rates. We examine the relationship between VTS score and the actual acceptance rates of side-branch entries. We group entries into five VTS-priority quintiles and calculate the acceptance rate within each quintile. The results are presented below.

Side-branch acceptance rate (%)Q1: lowest 20%Q2: 20–40%Q3: 40–60%Q4: 60–80%Q5: highest 20%
Lumina-mGPT26.628.129.431.433.1
Janus-Pro-7B20.223.124.425.526.7

Results analysis. As VTS priority increases, the observed side-branch acceptance rate also tends to rise on both models, showing relatively positive association. This suggests that VTS provides a useful empirical signal for identifying promising auxiliary continuation entries.

Together, the controlled allocation comparisons and the observed priority–acceptance relationship support the effectiveness of the proposed VTS scoring strategy.

AuthorsOpenReview ↗

Response to Reviewer msBa

Dear Reviewer msBa,

Thank you again for your thoughtful and constructive comments on our submission. As the author–reviewer discussion period is approaching its deadline, we would greatly appreciate it if you could let us know whether our responses have resolved your concerns. If you have any further questions or suggestions, we would be happy to discuss them.

Best regards,

The Authors of Paper

AuthorsOpenReview ↗

Follow-up to Reviewer msBa

Dear Reviewer msBa,

Thank you for reviewing our work so carefully and for your thoughtful suggestions. Below, we summarize our efforts to address the concerns you raised.

  • Measured the marginal benefit of widening SJD-PAC’s shallow tree. At fixed tree depth, the gain in accepted tokens per added node fell sharply across both models and both benchmarks. This finding supports our decision to distribute branches along the main path.
  • Evaluated the VTS scoring rule. VTS achieved lower latency and required fewer model evaluations than Local-prob and Entropy on both models. A separate analysis showed that side-branch entries with higher VTS scores were accepted more often.

We would value your thoughts on whether these analyses address your concerns and would be glad to discuss any questions that remain. We fully respect your current assessment and recognize the difficulty of weighing new evidence during the ICLR discussion. If you feel that our response has addressed most of your concerns and are comfortable doing so, we would be very grateful if you would consider a modest increase in your score in light of the revisions. Thank you again for the care you have given our work.

Best regards,
The Authors

Reviewer_zvGSOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FlexJacobi, a training-free method for accelerating autoregressive image generation through improved draft construction in multi-candidate Speculative Jacobi Decoding. Verification-aware Topology Shaping (VTS) allocates a fixed auxiliary-node budget to short continuations along the main Jacobi trajectory, using priorities computed from ranked draft probabilities. Context-guided Hybrid Drafting (CHD) selects continuation entry tokens through hidden-state-based retrieval from committed image tokens and samples continuation tails from cached draft distributions. Experiments on Lumina-mGPT-7B and Janus-Pro-7B, using PartiPrompts and MS-COCO 2017, demonstrate lower decoding latency than the evaluated baselines. Additional experiments investigate auxiliary-node efficiency, retrieval, and compatibility with relaxed verification.

Critical strengths
  1. The paper addresses a practically meaningful bottleneck and reports consistent wall-clock improvements. FlexJacobi achieves the lowest decoding latency among the evaluated methods across both models and benchmarks. For example, on Lumina-mGPT with PartiPrompts, latency decreases from 21.21 seconds for SJD-PAC to 16.99 seconds; on Janus-Pro, it decreases from 4.94 to 4.53 seconds. Reporting actual latency alongside NFE is valuable because additional speculative candidates can reduce sequential evaluations while increasing the cost of each evaluation.

  2. The auxiliary-budget experiments support the usefulness of the proposed draft structure. With a fixed 46-token main path and retrieval disabled, FlexJacobi with four auxiliary nodes achieves a 4.59× NFE speedup on Lumina-mGPT, compared with 4.54× for SJD-PAC with eighteen auxiliary nodes. This provides concrete evidence that candidate placement matters beyond candidate count. The retrieval ablation and compatibility experiments further suggest that draft construction can complement other acceleration mechanisms.

Critical weaknesses
  1. The verification procedure is insufficiently specified for the proposed candidate construction. Section 2.1 describes standard acceptance using a proposal distribution q, whereas CHD selects entry tokens by deterministic retrieval ranking, excludes duplicates and the main-path token, and uses sampling without replacement for fallback candidates. Section 3.3 refers to a multi-candidate verification procedure without specifying candidate testing order, the proposal probabilities used for these candidates, residual updates after rejection, or the treatment of retrieval-selected entries. Appendix D only specifies draft construction. Consequently, it is unclear whether standalone FlexJacobi preserves the target sampling distribution or introduces an approximation. This distinction materially affects the interpretation of its speed–quality trade-off. Table 1 also reports a Lumina-mGPT COCO FID of 30.77, compared with 29.32 for AR and 29.57 for SJD-PAC; without uncertainty estimates, this difference cannot be confidently dismissed as sampling variation.

  2. The experiments do not sufficiently isolate the contribution of the central VTS scoring rule. Figure 5 supports distributing short continuations along the main path, but does not establish that the specific priority in Equation (1) is responsible for the improvement. Comparisons against uniform placement, random placement, and alternative draft-confidence heuristics under identical budgets, continuation lengths, entry construction, and verification would be informative. In particular, the prefix product uses maximum draft probabilities rather than realized main-path token probabilities, while the ranked alternative probabilities need not correspond to the entries subsequently selected by retrieval. The paper acknowledges that the score is a proxy, but does not show how well it predicts actual continuation utility. This leaves the evidence stronger for the general topology design than for the claimed verification-aware allocation mechanism.

Questions for authors
  1. Verification clarification: Please specify the complete multi-candidate acceptance and residual-resampling procedure, including candidate order and the handling of retrieval-selected entries and sampling without replacement. Does standalone FlexJacobi preserve the target distribution? If so, what argument establishes this for CHD?

  2. Contribution validation: Under the same total budget, main-path length, continuation length, entry construction, and verification procedure, how does VTS compare with uniform/random allocation and simpler confidence-based allocation? Can you report whether its priorities correlate with measured branch utilization or additional accepted tokens?

  3. Quality and reproducibility clarification: How many generated images and independent seeds were used for each quality evaluation? Can you provide uncertainty estimates, particularly for the Lumina-mGPT COCO FID difference? Please also specify the retrieval neighborhood, temperature τ\tau, and whether reported latency includes retrieval, topology construction, and verification overhead.

5 comments
AuthorsOpenReview ↗

Official Comment [1/3]

W1 and Q1: (1) Verification clarification: Please specify the complete multi-candidate acceptance and residual-resampling procedure, including candidate order and the handling of retrieval-selected entries and sampling without replacement. (2) Does standalone FlexJacobi preserve the target distribution? If so, what argument establishes this for CHD?

A1. Thank you for requesting this clarification.

(1) Verification clarification. The overall verification procedure follows the verification procedure used in SJD-PAC. The detailed steps are as follows:

Candidate order and proposal probabilities. At a position with verified prefix hh, let p(x)=pθ(x∣h)p(x)=p_\theta(x\mid h) be the target sampling distribution and q(x)q(x) the recorded distribution used to generate the draft candidates. For siblings c1,…,cKc_1,\ldots,c_K sampled sequentially without replacement, verification follows their sampling order. With C<k=c1,…,ck−1\mathcal C_{ < k}=\\{c_1,\ldots,c_{k-1}\\}, the proposal for candidate ckc_k is

q(k)(x)={q(x)1−∑y∈C<kq(y),x∉C<k,0,x∈C<k.q^{(k)}(x)= \begin{cases} \displaystyle\frac{q(x)}{1-\sum_{y\in\mathcal C_{ < k}}q(y)}, & x\notin\mathcal C_{ < k},\\\\[6pt] 0, & x\in\mathcal C_{ < k}. \end{cases}

Thus, verification accounts for the removal of previously sampled candidates and renormalization of the remaining proposal mass; it does not repeatedly use the unadjusted qq. This is the without-replacement correction specified in SJD-PAC, Appendix A.3, Eq. (7).

Acceptance and residual resampling. Initialize r1=pr_1=p. For each candidate, draw uk∼Uniform(0,1)u_k\sim\mathrm{Uniform}(0,1) and accept ckc_k if

uk≤min⁡!(1,rk(ck)q(k)(ck)).u_k\leq\min\\!\left(1,\frac{r_k(c_k)}{q^{(k)}(c_k)}\right).

Acceptance ends the sibling search at this position, after which verification proceeds along the accepted continuation. Following rejection, update

rk+1(x)=[rk(x)−q(k)(x)]+∑v∈V[rk(v)−q(k)(v)]+,[a]+=max⁡(a,0),r_{k+1}(x)= \frac{[r_k(x)-q^{(k)}(x)]_+} {\sum_{v\in\mathcal V}[r_k(v)-q^{(k)}(v)]_+}, \qquad [a]_+=\max(a,0),

and test the next candidate using rk+1r_{k+1} and q(k+1)q^{(k+1)}. If all candidates are rejected, sample the output token from rK+1r_{K+1}. This is the sequential correction for sampling without replacement.

Distribution preservation. Let Zk=∑v[rk(v)−q(k)(v)]+Z_k=\sum_v[r_k(v)-q^{(k)}(v)]_+ for the rejection probability at the current test. The accepted and the residual contributions sum to

min⁡!(rk(x),q(k)(x))⏟accepted contribution+Zkrk+1(x)⏟residual contribution=min⁡!(rk(x),q(k)(x))+[rk(x)−q(k)(x)]+=rk(x).\underbrace{\min\\!\big(r_k(x),q^{(k)}(x)\big)}_{\text{accepted contribution}} + \underbrace{Z_k r_{k+1}(x)}_{\text{residual contribution}} = \min\\!\big(r_k(x),q^{(k)}(x)\big)+[r_k(x)-q^{(k)}(x)]_+ =r_k(x).

Recursing through the candidates recovers r1=pr_1=p, as established by SJD-PAC’s Appendix A.3, Theorem 2. The guarantee covers the complete output, including residual samples.

(2) Clarification of CHD. For CHD-selected entries, we use the token's probability under the draft distribution as the draft-side probability during verification. For a continuation-entry token xentryx_{\mathrm{entry}} at position ii, the verifier uses qi(xentry)q_i(x_{\mathrm{entry}}), where qiq_i is the draft distribution predicted at that position in the current iteration, so CHD is a lossy method. Nevertheless, our extensive experiments show that FlexJacobi with CHD achieves generation quality comparable to lossless methods. We also emphasize that CHD is not essential. Appendix C.3 reports results for CHD without retrieval. In this configuration, FlexJacobi replaces retrieval with direct sampling from the draft distribution (Q-sampling), making FlexJacobi lossless. The ablation studies in Section 4.2.2 and Appendix C.2 (Figs. 5 and 10) also use Q-sampling for auxiliary branch construction. These results further demonstrate the effectiveness of FlexJacobi.

Results in the ablation study in Appendix C.3:

ConfigurationLatency (s/image) ↓NFE ↓CLIP Score ↑
CHD with retrieval16.99458.9426.89
CHD with Q-sampling17.71487.9526.83

Results in Fig. 5 of Section 4.2.2 and Fig. 10 of Appendix C.2, where "ours" refers to FlexJacobi with Q-sampling:

ModelMetricOurs, 4Ours, 8Ours, 12Ours, 16SJD-PAC, 18
Lumina-mGPTNFE ↓513.05495.92492.83486.90518.96
CLIP Score ↑26.8426.8126.7726.9126.82
Janus-Pro-7BNFE ↓209.16200.02197.35194.17209.06
CLIP Score ↑26.9527.0026.9826.9926.93

We will expand Section 3.3 and the pseudocode to specify candidate ordering, proposal conditioning, residual updates, and the guarantee associated with each configuration.


AuthorsOpenReview ↗

Official Comment [2/3]


W2 and Q2: Contribution validation: Under the same total budget, main-path length, continuation length, entry construction, and verification procedure, (1) how does VTS compare with uniform/random allocation and simpler confidence-based allocation? (2) Can you report whether its priorities correlate with measured branch utilization or additional accepted tokens?

A2. Thank you for your valuable question. We compared VTS with four alternative allocation strategies under identical experimental settings, including Uniform, Random, Local-prob and Entropy. VTS achieved the best performance in both latency and NFE, demonstrating the effectiveness of VTS. Moreover, we also analyzed the relationship between VTS priorities and actual side branch acceptance rates. The results show that higher VTS priorities are associated with higher side-branch acceptance rates, providing empirical support for the proposed scoring criterion.

(1) Comparison of Allocation Strategies. We compare VTS against four alternative strategies under identical experimental settings. The results are presented below.

ModelMetricVTS (ours)UniformRandomLocal-probEntropy
Janus-Pro-7BLatency (s) ↓4.5355.5225.8205.8275.789
NFE ↓164.85212.55207.43208.31206.70
Lumina-mGPTLatency (s) ↓17.01819.51520.14720.29320.336
NFE ↓458.91533.76526.87530.11529.75

Specifically, the four alternatives are:

  • Uniform: Distributes auxiliary continuations evenly along the main path.
  • Random: Randomly assigns continuations to valid attachment positions.
  • Local-prob: Uses only the local draft probability mikm_i^k, removing the prefix-support term from VTS.
  • Entropy: Uses H(qi)mikH(q_i)m_i^k, favoring alternatives at positions with higher draft uncertainty.

Results analysis. VTS achieves the lowest latency and NFE among all five strategies. Uniform is the fastest alternative despite requiring more NFEs, as its fixed topology avoids the allocation overhead of dynamic strategies. Compared with Uniform, VTS further reduces latency and NFE on both models.

(2) Higher VTS is associated with higher acceptance rates. We examine the relationship between VTS score and the actual acceptance rates of side-branch entries. We group entries into five VTS-priority quintiles and calculate the acceptance rate within each quintile. The results are presented below.

Side-branch acceptance rate (%)Q1: lowest 20%Q2: 20–40%Q3: 40–60%Q4: 60–80%Q5: highest 20%
Lumina-mGPT26.628.129.431.433.1
Janus-Pro-7B20.223.124.425.526.7

Results analysis. As VTS priority increases, the observed side-branch acceptance rate also tends to rise on both models, showing relatively positive association. This suggests that VTS provides a useful empirical signal for identifying promising auxiliary continuation entries.

Together, the controlled allocation comparisons and the observed priority–acceptance relationship support the effectiveness of the proposed VTS scoring strategy.


AuthorsOpenReview ↗

Official Comment [3/3]

W1 and Q3: Quality and reproducibility clarification: (1) How many generated images and independent seeds were used for each quality evaluation? (2) Can you provide uncertainty estimates, particularly for the Lumina-mGPT COCO FID difference? (3) Please also specify the retrieval neighborhood, temperature (τ\tau), and (4) whether reported latency includes retrieval, topology construction, and verification overhead.

A3. (1) Evaluation protocol and random seeds. Our main evaluation generates one image per prompt, covering 5,000 images on MS-COCO 2017 and 1,632 images on PartiPrompts for each method. We assign a distinct random seed to each prompt to cover diverse sampling conditions, while using the same prompt–seed pairs across methods to ensure consistent comparisons.

(2) Multi-seed evaluation. We repeated the evaluation with two additional seed configurations to examine the consistency of the results. We also introduced two metrics (VQAScore and TIFA) to evaluate fine-grained image quality. The table below reports the performance of Lumina-mGPT on MS-COCO 2017, with the mean and standard deviation across three runs. Bold and underline‾\underline{\text{underline}} indicate the best and second-best mean within the lossy methods, respectively.

Lumina-mGPT / MS-COCO 2017

MethodLatency (s) ↓NFE ↓FID ↓CLIP Score↑VQAScore ↑TIFA ↑
Lossless
AR68.536 ± 0.0922357.000 ± 0.00029.471 ± 0.15226.337 ± 0.0040.8104 ± 0.00070.8343 ± 0.0012
SJD-PAC22.202 ± 0.009542.474 ± 0.38329.721 ± 0.15226.325 ± 0.0120.8117 ± 0.00110.8318 ± 0.0007
Lossy
CSD28.459 ± 0.062888.894 ± 0.47531.237±0.053‾\underline{31.237 ± 0.053}26.243±0.022‾\underline{26.243 ± 0.022}0.8103 ± 0.00230.8326 ± 0.0003
GSD23.050±0.043‾\underline{23.050 ± 0.043}678.165 ± 0.57232.525 ± 0.26526.230 ± 0.0290.8114 ± 0.00100.8315 ± 0.0003
PathSpec27.569 ± 0.115539.177±0.186‾\underline{539.177 ± 0.186}39.846 ± 0.29125.897 ± 0.0120.8061 ± 0.00080.8246 ± 0.0020
FlexJacobi (ours)17.779 ± 0.036480.212 ± 0.04230.525 ± 0.31826.252 ± 0.0280.8110±0.0008‾\underline{0.8110 ± 0.0008}0.8318±0.0011‾\underline{0.8318 ± 0.0011}

Specifically, we adopt the following two evaluation metrics:

  • VQAScore[1] evaluates overall text–image alignment, assessing whether the generated image faithfully reflects the scene and content described in the user's prompt.
  • TIFA[2] evaluates fine-grained text–image faithfulness by checking whether the generated image correctly depicts the objects, attributes, counts, and spatial relationships specified in the prompt.

Results analysis. Across three runs, FlexJacobi achieves an average FID of 30.525 ± 0.318, with the lowest mean FID and highest mean CLIP Score among lossy decoding methods. Its VQAScore and TIFA are also among the strongest for lossy methods, and all additional quality metrics remain close to those of the lossless SJD-PAC baseline. Overall, FlexJacobi achieves a favorable speed–quality trade-off while maintaining strong fine-grained generation quality.

(3) Retrieval configuration. We set the retrieval temperature to τ=0.05\tau=0.05 for both models to control cosine-similarity weighting. For Lumina-mGPT, we retrieve from committed image tokens in the preceding two rows and the already committed tokens in the current row. For Janus-Pro, which uses a lower-resolution 24×2424\times24 token grid, we retrieve from all previously committed image tokens.

(4) Latency measurement. The reported latency covers the complete autoregressive token-decoding procedure, including retrieval, dynamic topology construction, target-model computation, verification, and decoding-state updates.

[1] Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating Text-to-Visual Generation with Image-to-Text Generation. In ECCV, pages 366–384, 2024.

[2] Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A. Smith. TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering. In ICCV, pages 20406–20417, 2023


Reviewer_qSqvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes flexJacobi, which improves multi-candidate speculative Jacobi decoding by allocating a fixed auxiliary-node budget to promising positions along the draft trajectory. It combines verification-aware topology shaping with context-guided token retrieval to construct efficient candidate continuations without additional training or drafting forward passes.

Critical strengths
  • FlexJacobi uses a fixed speculative budget more efficiently by distributing auxiliary continuations along the Jacobi trajectory rather than concentrating them near early positions.
  • The method is training-free, introduces no additional drafting forward passes, and consistently reduces latency across two autoregressive image models while maintaining comparable generation quality.
Critical weaknesses
  • CHD selects continuation-entry tokens through hidden-state retrieval rather than sampling them from the stated draft distribution (qi)(q_i). The paper does not specify the corresponding proposal probability or explain how verification corrects for this change, so preservation of the target autoregressive distribution is unclear.
  • VTS relies on a heuristic priority computed only from draft probabilities, although actual acceptance depends on the target distribution. Its advantage over more general dynamic draft-tree allocation is not sufficiently established, and evaluation is limited to two models with FID and CLIP-Score.
Questions for authors
  • Since CHD retrieves continuation-entry tokens rather than sampling them from (qi)(q_i), could the authors clarify the proposal probability used during verification and whether the target AR distribution remains preserved?

  • Could the authors clarify the “22% auxiliary nodes” claim, given that the main experiments use the same total 64-node draft budget as SJD-PAC?

  • If available, additional compositional quality metrics beyond FID and CLIP-Score would help assess whether the modified drafting procedure preserves fine-grained generation quality.

5 comments
AuthorsOpenReview ↗

Official Comment [1/2]

W1 and Q1:Since CHD retrieves continuation-entry tokens rather than sampling them from qiq_i, could the authors clarify the proposal probability used during verification and whether the target AR distribution remains preserved?

A1. Thank you for your valuable question. For a continuation-entry token xentryx_{\mathrm{entry}} at position ii, the verifier uses qi(xentry)q_i(x_{\mathrm{entry}}), where qiq_i is the draft distribution predicted at that position in the current iteration. Since retrieval selects the entry token without sampling it from qiq_i, we acknowledge that CHD is a lossy decoding method and does not guarantee preservation of the target AR distribution. Nevertheless, our extensive experiments show that FlexJacobi with CHD achieves generation quality comparable to lossless methods. We also emphasize that CHD is not essential. Appendix C.3 reports results for CHD without retrieval. In this configuration, FlexJacobi replaces retrieval with direct sampling from the draft distribution (Q-sampling), making FlexJacobi lossless. The ablation studies in Section 4.2.2 and Appendix C.2 (Figs. 5 and 10) also use Q-sampling for auxiliary branch construction. These results further demonstrate the effectiveness of FlexJacobi. The results are presented below:

Results in the ablation study in Appendix C.3:

ConfigurationLatency (s/image) ↓NFE ↓CLIP Score ↑
CHD with retrieval16.99458.9426.89
CHD with Q-sampling17.71487.9526.83

Results in Fig. 5 of Section 4.2.2 and Fig. 10 of Appendix C.2, where "ours" refers to FlexJacobi with Q-sampling:

ModelMetricOurs, 4Ours, 8Ours, 12Ours, 16SJD-PAC, 18
Lumina-mGPTNFE ↓513.05495.92492.83486.90518.96
CLIP Score ↑26.8426.8126.7726.9126.82
Janus-Pro-7BNFE ↓209.16200.02197.35194.17209.06
CLIP Score ↑26.9527.0026.9826.9926.93

W2: VTS relies on a heuristic priority computed only from draft probabilities, although actual acceptance depends on the target distribution. Its advantage over more general dynamic draft-tree allocation is not sufficiently established.

A2. Thank you for your valuable question.

(1) Comparison of Allocation Strategies. In order to demonstrate the advantage of VTS, we additionally compared VTS with three general dynamic draft-tree allocation strategies under identical experimental settings, including Random, Local-prob and Entropy. VTS achieved the best performance in both latency and NFE, demonstrating the effectiveness of VTS.

The results are presented below.

ModelMetricVTS (ours)RandomLocal-probEntropy
Janus-Pro-7BLatency (s) ↓4.5355.8205.8275.789
NFE ↓164.85207.43208.31206.70
Lumina-mGPTLatency (s) ↓17.01820.14720.29320.336
NFE ↓458.91526.87530.11529.75

Specifically, the three strategies are:

  • Random: Randomly assigns continuations to valid attachment positions.
  • Local-prob: Uses only the local draft probability mikm_i^k, removing the prefix-support term from VTS.
  • Entropy: Uses H(qi)mikH(q_i)m_i^k, favoring alternatives at positions with higher draft uncertainty.

VTS achieves the lowest latency and NFE among all strategies, demonstrating the effectiveness of VTS.

(2) Higher VTS is associated with higher acceptance rates. We examine the relationship between VTS score and the actual acceptance rates of side-branch entries. We group entries into five VTS-priority quintiles and calculate the acceptance rate within each quintile. The results are presented below.

Side-branch acceptance rate (%)Q1: lowest 20%Q2: 20–40%Q3: 40–60%Q4: 60–80%Q5: highest 20%
Lumina-mGPT26.628.129.431.433.1
Janus-Pro-7B20.223.124.425.526.7

Results analysis. As VTS priority increases, the observed side-branch acceptance rate also tends to rise on both models, showing relatively positive association. This suggests that VTS provides a useful empirical signal for identifying promising auxiliary continuation entries.

Together, the controlled allocation comparisons and the observed priority–acceptance relationship support the effectiveness of the proposed VTS scoring strategy.

AuthorsOpenReview ↗

Official Comment [2/2]

Q2: Could the authors clarify the “22% auxiliary nodes” claim, given that the main experiments use the same total 64-node draft budget as SJD-PAC?

A3. Thank you for your question. The 22% figure refers to the number of auxiliary nodes in the quantitative study shown in Fig 5 and Fig 10 in the paper, not the total draft budget in the main experiments. With the main path fixed, FlexJacobi achieves comparable or lower NFE using only 4 auxiliary nodes, matching or surpassing SJD-PAC with 18 auxiliary nodes (4/18 ≈ 22.2%). The total node counts are 50 (FlexJacobi) and 64 (SJD-PAC), respectively. These results demonstrate the higher auxiliary node efficiency of our method. We will clarify this distinction in the revised manuscript.


W2 and Q3: If available, additional compositional quality metrics beyond FID and CLIP-Score would help assess whether the modified drafting procedure preserves fine-grained generation quality.

A4. Following your suggestion, we have added VQAScore and TIFA to evaluate fine-grained image quality on MS-COCO 2017 and PartiPrompts using Lumina-mGPT. Notably, FlexJacobi achieves the highest VQAScore and TIFA scores, demonstrating its ability to maintain fine-grained generation quality. The results are presented below, bold and underline‾\underline{\text{underline}} indicate the best and second-best result.

Lumina-mGPT / Parti-Prompts

MethodLatency (s/image) ↓NFE ↓VQAScore ↑TIFA ↑
Lossless
SJD-PAC21.21518.960.7470.809
Lossy
CSD27.22851.250.7500.805
GSD22.22‾\underline{22.22}654.580.750‾\underline{0.750}0.804‾\underline{0.804}
PathSpec26.88524.88‾\underline{524.88}0.7470.801
FlexJacobi (ours)16.99458.940.7540.812

Lumina-mGPT / MS-COCO 2017

MethodLatency (s/image) ↓NFE ↓VQAScore ↑TIFA ↑
Lossless
SJD-PAC22.20542.250.8110.832
Lossy
CSD28.45888.850.8080.832
GSD23.08‾\underline{23.08}678.590.8110.831‾\underline{0.831}
PathSpec27.70539.35‾\underline{539.35}0.8070.824
FlexJacobi (ours)17.80480.180.810‾\underline{0.810}0.831‾\underline{0.831}

Specifically, we adopt the following two evaluation metrics:

  • VQAScore[1] evaluates overall text–image alignment, assessing whether the generated image faithfully reflects the scene and content described in the user's prompt.
  • TIFA[2] evaluates fine-grained text–image faithfulness by checking whether the generated image correctly depicts the objects, attributes, counts, and spatial relationships specified in the prompt.

The tables report results for Lumina-mGPT on MS-COCO 2017 and Parti-Prompts. We compare SJD-PAC as the lossless baseline with CSD, GSD, PathSpec, and retrieval-enabled FlexJacobi as lossy methods.

Results analysis. FlexJacobi achieves the lowest decoding latency on both benchmarks while maintaining competitive image quality. On PartiPrompts, it also obtains the highest VQAScore and TIFA scores, demonstrating strong text–image alignment and fine-grained generation quality. Overall, these results demonstrate that FlexJacobi achieves a favorable trade-off between decoding efficiency and fine-grained generation quality.

[1] Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating Text-to-Visual Generation with Image-to-Text Generation. In ECCV, pages 366–384, 2024.

[2] Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A. Smith. TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering. In ICCV, pages 20406–20417, 2023


AuthorsOpenReview ↗

Response to Reviewer qSqv

Dear Reviewer qSqv,

Thank you again for your thoughtful and constructive comments on our submission. As the author–reviewer discussion period is approaching its deadline, we would greatly appreciate it if you could let us know whether our responses have resolved your concerns. If you have any further questions or suggestions, we would be happy to discuss them.

Best regards,

The Authors of Paper

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary Comment (to all reviewers)

Dear All,

We sincerely thank all reviewers for their thoughtful feedback. Within the limited rebuttal period, we have made our best effort to address every concern with new experiments, additional image quality evaluations, and detailed clarifications.

Our main efforts are summarized as follows:

  • Clarified FlexJacobi’s sampling guarantees, verification procedure, and auxiliary-node budget (qSqv W1/Q1/Q2, zvGS W1/Q1)\color{blue}{\text{(qSqv W1/Q1/Q2, zvGS W1/Q1)}}. The sampling guarantee depends on how FlexJacobi constructs branch entries. The reported node savings come from a separate budget ablation.

    Sampling guarantees (qSqv W1/Q1, zvGS W1/Q1)\color{blue}{\text{(qSqv W1/Q1, zvGS W1/Q1)}}. With retrieval enabled, FlexJacobi selects branch entries by similarity and verifies them using their draft probabilities, making decoding lossy. With retrieval disabled, FlexJacobi samples entries from the draft and preserves the target distribution. On each model, lossless FlexJacobi achieved the best decoding speedup among the configurations evaluated in the auxiliary-node ablation.

    Scope of the auxiliary-node claim (qSqv Q2)\color{blue}{\text{(qSqv Q2)}}. The reported 22% is the ratio of FlexJacobi’s auxiliary-node count to SJD-PAC’s in the fixed-main-path ablation. At this smaller auxiliary budget, FlexJacobi required a comparable number of model evaluations. Both methods used the same total node budget in the main experiments.

    Multi-candidate verification (zvGS W1/Q1)\color{blue}{\text{(zvGS W1/Q1)}}. FlexJacobi uses the same multi-candidate verification procedure as SJD-PAC.

  • Evaluated the effectiveness of VTS (qSqv W2, zvGS W2/Q2, msBa W2/Q2)\color{blue}{\text{(qSqv W2, zvGS W2/Q2, msBa W2/Q2)}}. We compared alternative branch allocation rules and measured how often the verifier accepted side-branch entries at different VTS scores.

    Controlled allocation comparison (qSqv W2, zvGS W2/Q2)\color{blue}{\text{(qSqv W2, zvGS W2/Q2)}}. We compared VTS with four alternative allocation rules (Uniform, Random, Local-prob, and Entropy) under identical experimental settings. VTS achieved the lowest latency and NFE on both models.

    Higher VTS is associated with higher acceptance rates (zvGS W2/Q2, msBa W2/Q2)\color{blue}{\text{(zvGS W2/Q2, msBa W2/Q2)}}. We grouped side-branch entry tokens by VTS score and measured the fraction the verifier accepted in each group. Score and acceptance rate were positively associated on both models.

  • Assessed image quality in greater detail (qSqv W2/Q3, zvGS W1/Q3)\color{blue}{\text{(qSqv W2/Q3, zvGS W1/Q3)}}. We examined variation across random seeds and added fine-grained image quality metrics.

    Variation across seeds (zvGS W1/Q3)\color{blue}{\text{(zvGS W1/Q3)}}. We repeated the Lumina-mGPT evaluation on MS-COCO and reported the mean and standard deviation across seeds. FlexJacobi had a higher mean FID and lower mean decoding latency than SJD-PAC.

    Fine-grained image quality (qSqv W2/Q3)\color{blue}{\text{(qSqv W2/Q3)}}. We evaluated Lumina-mGPT on MS-COCO and PartiPrompts using VQAScore and TIFA. On each benchmark, FlexJacobi achieved the best or second-best score on each metric among the lossy methods, with scores comparable to those of the lossless SJD-PAC baseline.

  • Provided direct empirical support for the motivation behind FlexJacobi’s draft topology (msBa W1/Q1)\color{blue}{\text{(msBa W1/Q1)}}. We widened SJD-PAC’s shallow tree at fixed depth and measured the gain in accepted tokens per added node. On both models and both benchmarks, the marginal gain per node declined sharply as the tree widened.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.