- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SpecSlot, a pixel-space image generation architecture that uses frequency information to guide tokenization and attention. It divides each spatial patch into multiple frequency-based slots, allowing only the lowest-frequency slot to participate in global attention while the others interact locally. This reduces global attention costs while preserving the ability to generate fine details. The model also combines spectral and spatial outputs for image reconstruction.
The authors evaluate SpecSlot on ImageNet-256 and ImageNet-512 against JiT baselines, comparing generation quality, FLOPs, and throughput. They also conduct ablations on token allocation, attention patterns, frequency roles, and output representations. The results show comparable quality at ImageNet-256 and a better quality–compute trade-off at ImageNet-512, where SpecSlot improves FID by 2.84 over a compute-matched JiT baseline with slightly fewer FLOPs.
Critical strengths
1.The paper uses frequency information to guide token communication, allowing only low-frequency slots to participate in global attention. I think this is a simple and interesting way to improve efficiency, and the idea of separating output detail from global communication has potential beyond this specific architecture.
- The results are particularly encouraging at higher resolution, where SpecSlot achieves a 2.84 FID improvement over the compute-matched JiT baseline with slightly fewer FLOPs. The larger improvement at ImageNet-512 compared to ImageNet-256 is also a positive signal for its potential to scale to higher resolutions.
Critical weaknesses
- While the stronger results at ImageNet-512 are encouraging, the experiments are still limited to relatively small models (~140–160M parameters) and class-conditional image generation. It remains unclear whether the advantage will hold for larger models, higher resolutions, or more complex generation tasks. Since efficiency is a main motivation of the paper, I think broader scaling experiments would strengthen the contribution.
Questions for authors
Can you show that the quality–compute advantage of SpecSlot can hold for larger models (e.g., ~1B parameters)?
3 comments
Thank you for your response and clarification. I will maintain my current score.
Dear Reviewer vyef,
We appreciate your encouraging assessment and the opportunity to clarify our work. Your suggestions on broader task coverage and larger models have helped guide our revisions and future experiments. We are grateful for your thoughtful feedback throughout the discussion.
Best regards, The Authors
Response to Reviewer vyef
We sincerely thank you for the encouraging review and for recognizing the potential of separating output detail from global communication. We address your concern about broader task coverage and larger models below.
W1: Extend evaluation beyond class-conditional generation.
To address your concern that the experiments are limited to class-conditional ImageNet, we added text-to-image experiments in Section 4.3, p. 8, Table 2. All three models use the same training data, text encoder, and adapter design; the protocol is given in Appendix B, p. 17, Table 5. SpecSlot retains its spectral output roles and carrier/local routing: carriers read the text context during global blocks, while local blocks update the image slots within each anchor. Table a reports the results.
Table a. Text-to-image generation at 256×256.
SpecSlot achieves the highest DPG-Bench score among these three models, exceeding JiT-B/8 by 1.06 percentage points with 40.6% fewer forward FLOPs. On GenEval, its score lies between the two JiT baselines. These results extend the measured quality–compute trade-off to text-conditioned generation using the same allocation and routing design. Text-to-image examples are shown in Figure 5, p. 8, and Appendix Figure 11, pp. 27–28.
We also strengthened the ImageNet-512 comparison following the JiT-B/16 CFG sweep. With CFG 5.0 for JiT, SpecSlot-K4 retains a 0.88 FID advantage over JiT-CM at slightly lower forward cost (Section 4.2, p. 7, Table 1), updating the original 2.84 margin.
Q1: Does the quality–compute advantage hold at approximately 1B parameters?
Thank you for suggesting this larger-scale comparison. Completing the additional training and evaluation at approximately 1B parameters requires more time than the rebuttal period allows. We will include an approximately 1B-parameter ImageNet-256 comparison in the camera-ready version, comparing SpecSlot with JiT under a shared training and evaluation protocol. We will report FID-50K, forward FLOPs, and measured sampling throughput.
The same carrier rule can be retained when increasing backbone width or depth, keeping the global sequence length independent of the number of output slots. The larger-model comparison will test whether this design continues to provide a favorable quality–compute trade-off at that scale.
Thank you again for the encouraging feedback and for helping us broaden the evaluation. We would be glad to clarify any remaining points during the discussion.