- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates limited sample diversity in autoregressive text-to-image generation and proposes a training-free cluster-level truncation strategy that samples over groups of visually similar tokens rather than individual tokens. The method is simple, broadly applicable, and improves the diversity–quality trade-off across several AR image generators. However, the overall novelty and technical depth are relatively limited, as the approach mainly combines visual-token clustering with existing truncation-based sampling, and stronger evidence is needed to verify that the gains reflect meaningful high-level semantic diversity.
Critical strengths
-
This paper Identifies an interesting mismatch between token-level randomness and semantic-level diversity in autoregressive image generation.
-
The authors propose a simple, training-free cluster-based sampling strategy that can be easily integrated into existing AR models. They also provide reasonably broad experiments across multiple models, showing consistent improvements in the diversity–quality trade-off.
Critical weaknesses
-
The core idea mainly extends existing truncation strategies from token-level to cluster-level sampling. Similar visual-token clustering has already been explored in prior work such as ClusterMark [1], although for a different purpose, making the methodological novelty somewhat limited. The proposed method is essentially a training-free decoding heuristic based on offline K-means clustering, without substantial modifications to the model architecture, training objective, or representation learning.
-
The experiments mainly rely on existing diversity metrics, while it remains unclear whether the method truly improves high-level diversity in composition, object relations, and semantic modes.
[1] D. Lukovnikov, A. Müller, E. Quiring, and A. Fischer, “ClusterMark: Towards Robust Watermarking for Autoregressive Image Generators with Visual Token Clustering,” CVPR 2026.
Questions for authors
-
How does the proposed cluster truncation fundamentally differ from prior visual-token clustering methods such as ClusterMark, beyond applying clustering to a different downstream objective?
-
How sensitive is the method to the choice of clustering algorithm, number of clusters, and feature space used for clustering? Would alternative clustering strategies lead to significantly different results?
-
Can the authors provide stronger evidence that the diversity improvement comes from high-level semantic variations, such as composition, object relations, or pose, rather than mainly low-level appearance or texture changes?
-
Given that the method is training-free and technically lightweight, what is the main methodological contribution beyond combining K-means clustering with existing truncation-based sampling?
-
Does cluster-level sampling introduce any additional inference overhead, especially for large visual vocabularies, and how does this overhead compare with standard top-k/top-p sampling?
3 comments
Response 2/2
Q2. How sensitive is the method to the choice of clustering algorithm, number of clusters, and feature space used for clustering?
We included a study on the effect of the number of clusters on both image quality and diversity in Appendix E.3. While diversity gains tend to increase with a larger number of clusters, using fewer clusters provides greater robustness to quality degradation across temperatures.
To study the effect of the clustering algorithm, we use a small set of 120 prompts from GenEval and compare the performance of two clustering algorithms: K-Means (reported in the paper) and balanced K-Means (using the same number of clusters, CFG, temperature, and k as defined for each base model in Table 3, Appendix D.2). On Janus Pro the two variants are indistinguishable. On Anole, balanced K-means raises CLIP alignment slightly and costs a small, significant amount of diversity.
| Model | Setting | Variant | CLIP | DINOv2 | Vendi |
|---|---|---|---|---|---|
| Janus Pro | CFG 5, T 4.0, k 1.5, n 500 | K-means | 0.319 | 0.332 | 2.500 |
| Balanced K-means | 0.318 | 0.336 | 2.519 | ||
| Δ | −0.001 [−0.003, +0.002] | +0.004 [−0.011, +0.018] | +0.019 [−0.041, +0.079] | ||
| Anole | CFG 3, T 2.5, k 1.5, n 1000 | K-means | 0.298 | 0.568 | 3.577 |
| Balanced K-means | 0.310 | 0.546 | 3.461 | ||
| Δ | +0.012 [+0.008, +0.015] | −0.022 [−0.040, −0.004] | −0.115 [−0.192, −0.039] |
GenEval tuning subset (120 prompts, 5 images each), same prompts for both variants. Δ = balanced − K-means, 95% paired bootstrap CI; bold = CI excludes 0.
Q5. Does cluster-level sampling introduce any additional inference overhead, especially for large visual vocabularies, and how does this overhead compare with standard top-k/top-p sampling?
We measure per-image generation time on an A100-80GB GPU using 20 PartiPrompts with five images generated per batch, including the generation loop and VQ decoding. Measurements are CUDA-synchronized, with one warm-up prompt excluded. Percentages in parentheses indicate runtime increases relative to Default.
| Model | Default (s/image) | Dyn. Temp. (s/image) | p-less cluster (s/image) |
|---|---|---|---|
| Janus Pro | 2.08 | 2.17 (+4.3%) | 2.17 (+4.4%) |
| Emu3 | 74.87 | 75.19 (+0.4%) | 75.71 (+1.1%) |
| LlamaGen | 1.07 | 1.10 (+2.8%) | 1.10 (+2.3%) |
| Anole | 2.99 | 2.98 (−0.3%) | 3.03 (+1.2%) |
Our method incurs minimal runtime overhead, increasing per-image generation time by only 1.1–4.4% relative to Default across all four models. For Janus Pro and LlamaGen, its runtime is nearly identical to Dynamic Temperature, suggesting that cluster-based truncation introduces negligible additional overhead.
Summary. We believe that your feedback and suggested additional experiments have improved our paper. We hope that you can re-evaluate our paper in light of these new results and response.
Thanks for the rebuttal. After reading the rebuttal, I am still concerned that the method shares many similarities with cluster-based method, especially in the comparison with the cluster-based by experiments and analysis experiments are important. Besides, the diversity measurement is important and needs to be analyzed from many perspectives. Therefore, I would like to keep the rating unchanged.
Response 1/2
We thank the reviewer for appreciating our analysis of the mismatch between token-level randomness and semantic diversity in autoregressive (AR) image generation. Beyond the proposed method itself, we believe these insights offer valuable guidance for future research, particularly given the growing interest in unified autoregressive image–text generation. We would like to address your feedback and questions below.
We thank the reviewer for pointing us to ClusterMark. We will revise the manuscript to cite this work in Related Works and clarify our methodological distinction as below.
While both methods leverage codebook-based token clustering, they address different problems through fundamentally different decoding operations. ClusterMark uses clusters to assign visually similar tokens to the same watermark partition, improving watermark robustness under image perturbations. In contrast, our method uses clusters as the unit of probability aggregation and entropy-adaptive truncation to address the mismatch between token-level randomness and image-level diversity. We further provide a theoretical justification for why clustering is needed for this diversity-tradeoff problem in Appendix B.
We would also like to highlight the timely and potentially broad impact of a training-free decoding algorithm such as p-less cluster. With growing interest in developing new autoregressive models for image generation, there is an increasing need for decoding strategies that can improve generation quality and diversity without requiring model retraining. As a plug-and-play method that makes minimal assumptions about model architecture and training procedures, p-less cluster can be readily integrated into different AR image generators. Its effectiveness across four distinct AR models demonstrates its potential for broad applicability, while its low computational overhead makes it practical for deployment.
We agree that our paper does not explicitly evaluate high-level diversity in composition, object relationships, and semantic modes, and we will acknowledge this limitation in the revised manuscript. To ensure a rigorous and well-grounded evaluation, we adopt the diversity metrics established in [1].
[1] Parmar, Gaurav, et al. "Scaling group inference for diverse and high-quality generation." International Conference on Learning Representations. Vol. 2026. 2026.