- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes FlexJacobi, a training-free framework for improving the efficiency of multi-candidate Speculative Jacobi Decoding (SJD) in autoregressive image generation. The key idea is to improve speculative candidate utilization by dynamically organizing auxiliary continuations along the Jacobi trajectory. Specifically, the proposed Verification-aware Topology Shaping (VTS) allocates a fixed auxiliary-node budget based on draft-distribution statistics, while Context-guided Hybrid Drafting (CHD) leverages historical predictive-context similarity to select continuation entry tokens and reuses cached draft distributions for subsequent tokens. The resulting candidates are jointly verified without additional drafting forward passes. Experiments on Lumina-mGPT and Janus-Pro demonstrate consistent improvements in decoding efficiency while maintaining competitive generation quality.
Critical strengths
1.The paper identifies a meaningful limitation of existing multi-candidate SJD methods: allocating more speculative candidates does not necessarily translate into longer accepted sequences. The proposed approach of redistributing a limited candidate budget along the Jacobi trajectory is intuitive and addresses an important efficiency problem.
2.FlexJacobi improves draft construction without requiring additional training or a separate draft model. Both VTS and CHD reuse information already available during decoding, making the framework lightweight and practical. The proposed design also provides a reasonable way to exploit the characteristics of visual tokens.
3.The experiments cover two autoregressive image generation models, multiple benchmarks, and a range of competitive baselines. The results demonstrate consistent improvements in decoding latency and NFE while maintaining competitive image quality. The auxiliary-budget ablations and compatibility experiments with relaxed verification methods further support the practical effectiveness and applicability of the framework.
4.The paper is well written and logically organized. The motivation is clearly presented, the proposed components are explained in an accessible manner, and the figures help illustrate the overall methodology. The experimental section is also well structured and easy to follow.
Critical weaknesses
-
The paper argues that SJD-PAC's shallow local candidate tree provides redundant recovery opportunities as alternatives accumulate within the same region, while redistributing auxiliary continuations along the Jacobi trajectory can improve accepted length. However, the current experiments do not directly demonstrate how additional candidates in SJD-PAC lead to redundant recovery opportunities or how their contribution to accepted length gradually saturates. This lack of empirical evidence raises concerns about the validity of the paper's central motivation.
-
The authors use the product of the maximum probabilities in the draft distributions as a draft-based proxy for the likelihood that the prefix will be accepted. I interpret this choice as implicitly assuming that higher prefix support corresponds to higher prefix acceptance rates. However, the actual acceptance probability depends on both the draft and target distributions, and higher draft confidence does not necessarily imply a higher acceptance probability. Therefore, the authors' current discussion does not sufficiently justify this implicit assumption underlying the proposed scoring criterion.
My main concerns are the insufficient empirical support for the central motivation and the lack of justification for the proposed scoring criterion in VTS. While I appreciate the proposed idea and the promising experimental results, the current evidence does not sufficiently support the underlying assumptions of the method.
Questions for authors
1.Could the authors provide more concrete empirical evidence supporting the claim that additional candidates in SJD-PAC's shallow local candidate tree provide redundant recovery opportunities and that their contribution to accepted length gradually saturates?
2.Could the authors further explain why the product of the maximum probabilities in the draft distributions is a reasonable draft-based proxy for prefix acceptance? In particular, is there empirical evidence showing that higher prefix support corresponds to higher prefix acceptance rates during verification?
If these concerns are satisfactorily addressed, I would be inclined to support acceptance of this paper.
7 comments
Response to Reviewer msBa
Dear Reviewer msBa,
Thank you again for your thoughtful and constructive comments on our submission. As the author–reviewer discussion period is approaching its deadline, we would greatly appreciate it if you could let us know whether our responses have resolved your concerns. If you have any further questions or suggestions, we would be happy to discuss them.
Best regards,
The Authors of Paper
Follow-up to Reviewer msBa
Dear Reviewer msBa,
Thank you for reviewing our work so carefully and for your thoughtful suggestions. Below, we summarize our efforts to address the concerns you raised.
- Measured the marginal benefit of widening SJD-PAC’s shallow tree. At fixed tree depth, the gain in accepted tokens per added node fell sharply across both models and both benchmarks. This finding supports our decision to distribute branches along the main path.
- Evaluated the VTS scoring rule. VTS achieved lower latency and required fewer model evaluations than Local-prob and Entropy on both models. A separate analysis showed that side-branch entries with higher VTS scores were accepted more often.
We would value your thoughts on whether these analyses address your concerns and would be glad to discuss any questions that remain. We fully respect your current assessment and recognize the difficulty of weighing new evidence during the ICLR discussion. If you feel that our response has addressed most of your concerns and are comfortable doing so, we would be very grateful if you would consider a modest increase in your score in light of the revisions. Thank you again for the care you have given our work.
Best regards,
The Authors
A1. Thank you for your valuable question. We measured the contribution of each additional auxiliary node to the mean accepted length in SJD-PAC. The results show a clear decline as the auxiliary budget increases. For example, on Lumina-mGPT / PartiPrompts, the contribution per node decreases from 0.02846 (R:0→4) to 0.00613 (R:10→18). This directly demonstrates the diminishing contribution of additional candidates to accepted length. Details are shown below:
Mean Accepted Length Gain per Auxiliary Node in SJD-PAC. We fix the main path and tree depth D=3 of SJD-PAC, while increasing the branching factor from K=2 to K=4, corresponding to auxiliary-node budgets of R=4,10,18. We measure the mean accepted length MAT(R) as the average number of draft tokens accepted per verification round with R auxiliary nodes. To measure the contribution of additional auxiliary nodes, we calculate the gain in mean accepted length per added node:
g(R1!→!R2)=R2−R1MAT(R2)−MAT(R1).
Results analysis. As the shallow candidate tree expands, the marginal gain per auxiliary node decreases consistently. Across all four settings, the gain drops substantially from the first expansion (R:0→4) to the last (R:10→18). These results show that additional candidates contribute progressively less to accepted length, consistent with increasingly redundant recovery opportunities in SJD-PAC's shallow local tree.
A2. Thank you for your valuable question. In order to demonstrate the advantage of VTS, we additionally compared VTS with two alternative Strategies including Local-prob, Entropy-prob, where Local-prob, Entropy modify the product of maximum draft probabilities in the VTS scoring criterion. Moreover, we also analyzed the relationship between VTS priorities and actual side branch acceptance rates. The results show that higher VTS priorities are associated with higher side-branch acceptance rates, providing empirical support for the proposed scoring criterion.
(1) Comparison of Allocation Strategies. We compare VTS against two alternative strategies under identical experimental settings. The results are presented below.
Specifically, the two alternatives are:
VTS achieves the lowest latency and NFE among all strategies, demonstrating the effectiveness of VTS.
(2) Higher VTS is associated with higher acceptance rates. We examine the relationship between VTS score and the actual acceptance rates of side-branch entries. We group entries into five VTS-priority quintiles and calculate the acceptance rate within each quintile. The results are presented below.
Results analysis. As VTS priority increases, the observed side-branch acceptance rate also tends to rise on both models, showing relatively positive association. This suggests that VTS provides a useful empirical signal for identifying promising auxiliary continuation entries.
Together, the controlled allocation comparisons and the observed priority–acceptance relationship support the effectiveness of the proposed VTS scoring strategy.