- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies the reconstruction–generation dilemma in visual tokenization models, arguing that reconstruction quality does not necessarily determine generative learnability. The authors analyze the effects of latent geometry, training data, model capacity, and optimization budget. Based on a series of theoretical analyses (including Gaussian models, minimax analysis, and random matrix theory), they propose Capacity-Matched Spectral Tokenization (CMST), which adjusts the latent spectrum according to downstream computational resources. Experiments demonstrate improvements in generation quality under fixed training budgets.
Critical strengths
The reconstruction-preserving transformation experiment is interesting. It shows that identical reconstruction quality can lead to substantially different downstream generation performance.
CMST achieves meaningful empirical improvements over existing baselines.
Critical weaknesses
The main motivation and observations, particularly the reconstruction–generation trade-off, are already well established in prior work, such as [1], [2], and [3].
The paper presents an extensive number of theoretical results and proofs (the paper is 53 pages long!) to address what is fundamentally an empirical problem. However, most of these results are derived under Gaussian or linear assumptions, whereas the actual experiments involve nonlinear diffusion Transformers. Moreover, given the capabilities of modern generative AI systems (e.g., GPT-6 Astra, maybe with Ultra thinking level in codex or ChatGPT Pro), which can now help derive large numbers of lemmas and proofs relatively quickly (see, e.g., [4]), I believe it is increasingly important to ensure that theoretical complexity is clearly justified. In particular, theoretical results should provide concrete, precise, and ideally testable predictions that go beyond existing empirical observations, rather than merely offering post hoc explanations.
In this paper, I am not convinced that such an extensive theoretical framework is necessary, nor is it clear to me how closely the theoretical results are connected to the proposed method and experimental findings. I would therefore like the authors to clarify this point, as elaborated in the questions below.
[1] When Worse is Better: Navigating the Compression-Generation Trade-off in Visual Tokenization. NeurIPS 2025.
[2] Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models. CVPR 2025.
[3] GigaTok: Scaling Visual Tokenizers to 3 Billion Parameters for Autoregressive Image Generation. ICCV 2025.
Questions for authors
- The paper develops an extensive theoretical framework spanning over 50 pages. While the authors explicitly connect several theoretical results to the CMST objective, I am not convinced that the specific mathematical constructions are necessary to explain the observed phenomena or design the proposed method. Could the authors clarify what insights or design decisions fundamentally depend on these results, rather than on existing intuitions about representation complexity, spectral conditioning, and resource constraints?
- While the paper quantitatively validates several theoretical results in Gaussian or local surrogate settings, it remains unclear whether the framework has quantitative predictive power for actual nonlinear diffusion models. After calibrating the surrogate parameters, can the proposed generative cost accurately predict or rank the optimal latent dimensions, spectral configurations, or downstream generation performance under different resource budgets? Such evidence would help distinguish a genuinely predictive theory from a theoretical motivation for an empirically effective regularizer.