- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Spatially Speculative Decoding (SSD), an inference acceleration framework for autoregressive image generation that exploits the two-dimensional spatial correlations of visual tokens. SSD introduces lightweight horizontal and vertical multi-token prediction (MTP) heads to predict future hidden states while keeping the pretrained backbone frozen. By leveraging vertical prediction, SSD enables row-level parallel drafting, followed by iterative verification and correction to maintain generation quality. Experiments on Janus-Pro-7B, Lumina-mGPT-7B, and Emu3-8B demonstrate wall-clock speedups of 5.70×, 7.35×, and 11.03×, respectively, with relatively small degradation in generation quality.
Critical strengths
1.The motivation is well supported by the analysis of spatial correlations in visual tokens. The demonstrated feasibility of vertical prediction naturally leads to the proposed row-level parallel decoding approach.
-
The experiments cover multiple backbone models, competitive baselines, and ablation studies. The reported results demonstrate substantial decoding speedups with relatively minor degradation in generation quality.
-
The paper is well organized and clearly written, with a coherent progression from motivation and analysis to methodology and experimental validation.
Critical weaknesses
-
Table 1 shows unexplained discrepancies between the step and wall-clock speedups of SJD and GSD across different backbones, particularly on Lumina-mGPT and Emu3-8B. For example, on Emu3-8B, SJD achieves only 1.08× wall-clock speedup despite a 2.24× step reduction, whereas GSD achieves 2.90× speedup with a 3.46× step reduction. Given that GSD largely follows the same Jacobi decoding workflow as SJD while introducing additional sorting and grouping overhead, these differences in computational efficiency are difficult to explain. Even these relatively peripheral baselines show unexplained inconsistencies. This raises my concerns about the reliability of the more critical SSD speedup results.
-
The reported training costs do not account for the computational expense of generating training data and extracting backbone hidden states.
My primary concern is Weakness 1, particularly the unexplained discrepancies between step reduction and wall-clock speedups across different baselines, which raise questions about the reliability of the reported SSD speedup results. The incomplete accounting of training costs in Weakness 2 is a secondary concern. I encourage the authors to focus their rebuttal on explaining the discrepancies identified in Weakness 1.
If this concern can be satisfactorily addressed, I would be inclined to support acceptance of this paper.
Questions for authors
-
Could the authors explain the discrepancies between SJD and GSD across all evaluated backbones, particularly on Emu3-8B, at the decoding-workflow and computational-cost levels? An explanation based solely on improved acceptance rates is insufficient to account for the reported differences between step and wall-clock speedups.
-
Could the authors report the total computational cost, including autoregressive data generation, hidden-state extraction, and MTP head training?
6 comments
Part 2/2
Question 1: Clarification of the discrepancies between SJD and GSD
We thank the reviewer for pointing this out. After inspecting GSD's code [2], we find that GSD's implementation batches the conditional and unconditional CFG branches, whereas SJD [1] and AR use serial CFG on Lumina-mGPT and Emu3. Batching runs both branches in one backbone call rather than two serial calls, reducing latency without changing step counts. SSD uses the same serial CFG strategy as AR in Table 1, ensuring a fair comparison.
We therefore evaluate SJD, GSD, and SSD under both serial and batched CFG strategies on the same hardware using 100 DPG-Bench prompts. See the results in the tables below; speedups are relative to the same serial AR baseline.
Batched CFG.
| Backbone | Method | Step speedup | Latency (s) | Latency speedup |
|---|---|---|---|---|
| Lumina-mGPT-7B | SJD | 32.90 | ||
| Lumina-mGPT-7B | GSD | 24.24 | ||
| Lumina-mGPT-7B | SSD | 7.53 | ||
| Emu3-8B | SJD | 171.75 | ||
| Emu3-8B | GSD | 120.91 | ||
| Emu3-8B | SSD | 19.10 |
Serial CFG.
| Backbone | Method | Step speedup | Latency (s) | Latency speedup |
|---|---|---|---|---|
| Lumina-mGPT-7B | SJD | 59.78 | ||
| Lumina-mGPT-7B | GSD | 42.86 | ||
| Lumina-mGPT-7B | SSD | 13.15 | ||
| Emu3-8B | SJD | 260.44 | ||
| Emu3-8B | GSD | 182.31 | ||
| Emu3-8B | SSD | 25.10 |
Under matched CFG strategies, SJD and GSD exhibit consistent overall latency trends: batching reduces latency for both methods while leaving step counts essentially unchanged. SSD outperforms both methods under matched experimental settings. We will make these CFG settings explicit in the revision.
Question 2: Additional reporting of the total computational cost
We thank the reviewer for this suggestion. We report the total computational cost in the table below.
-
Autoregressive data generation: Data generation is a separate, one-time cost. The costs are 29.7 GPU-hours for Janus-Pro, 27.78 for Lumina-mGPT, and 40.97 for Emu3.
-
Hidden-state extraction and MTP head training: Hidden-state extraction is already included in MTP head training. Hidden states are computed by the frozen backbone within each training step. The reported training costs of 3.02, 5.55, and 3.31 GPU-hours for Janus-Pro, Lumina-mGPT, and Emu3 therefore include both hidden-state extraction and head optimization.
| Backbone | AR data generation (GPU-h) | Head training, including hidden-state extraction (GPU-h) | Total (GPU-h) |
|---|---|---|---|
| Janus-Pro-7B | 29.7 | 3.02 | 32.7 |
| Lumina-mGPT-7B | 27.78 | 5.55 | 33.33 |
| Emu3-8B | 40.97 | 3.31 | 44.28 |
References
[1] Teng et al. Accelerating Auto-regressive Text-to-Image Generation with Training-free Speculative Jacobi Decoding. ICLR, 2025.
[2] So et al. Grouped Speculative Decoding for Autoregressive Image Generation. ICCV, 2025.
We welcome your feedback.
We thank the reviewer for their time and consideration. We hope our responses have addressed their questions and concerns, and would appreciate any further feedback. We would be happy to clarify any remaining points.
Part 1/2
We thank the reviewer for recognizing SSD's strong empirical motivation, substantial speedups, and clear and coherent presentation.
Weakness 1: Clarification of implementation details: discrepancies in speedups between SJD and GSD
Our reported SSD speedups are reliable and based on a fair comparison with AR. We thank the reviewer for pointing out the discrepancy between SJD [1] and GSD [2]. After inspecting GSD's code, we find that GSD's implementation batches the conditional and unconditional CFG branches, whereas SJD and AR use serial CFG on Lumina-mGPT and Emu3. Batching runs both branches in one backbone call rather than two serial calls, reducing latency without changing step counts. SSD uses the same serial CFG strategy as AR in Table 1, ensuring a fair comparison.
We therefore evaluate SJD, GSD, and SSD under both serial and batched CFG strategies on the same hardware using 100 DPG-Bench prompts. See the results in the tables below; speedups are relative to the same serial AR baseline.
Batched CFG.
Serial CFG.
Under matched CFG strategies, SJD and GSD exhibit consistent overall latency trends: batching reduces latency for both methods while leaving step counts essentially unchanged. SSD outperforms both methods under matched experimental settings. We will make these CFG settings explicit in the revision.
Weakness 2: Additional reporting of computational costs
We thank the reviewer for bringing this up. We report the computational costs of generating training data and extracting backbone hidden states below. We will include these costs in the revision.
To be continued.