- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
The paper proposes VQ-DSGAN to address codebook underutilization and dimensional collapse in learned vector-quantized tokenizers. It shares an encoder–decoder backbone between generator and discriminator modes through FiLM-style modulation, combining image reconstruction energies with latent recovery energies under a code-centered prior. Experiments on ImageNet 256×256 report improved reconstruction and class-conditional generation, supported by ablations of the discriminator design, target prior, and latent noise.
Critical strengths
1.The shared bidirectional discriminator offers a coherent way to combine image fidelity and latent recoverability without redesigning the quantizer. The FiLM implementation adds few parameters and introduces no additional discriminator computation at inference.
2.The component ablations are informative. In Table 6, removing noise preserves 98% utilization but reduces effective dimension from 128 to 32 and worsens rFID from 1.26 to 1.54. This supports the practical value of addressing code usage and latent geometry separately.
Critical weaknesses
1.The theory does not establish the dimensional-collapse property measured experimentally. Section 4.1 measures the PCA spectrum of codebook vectors, whereas Theorem 9 concerns the local Jacobian of the discriminator-mode decoder–encoder map. Distinct codewords can all lie on a line while this map is the identity, satisfying the local recovery condition despite rank-one codebook covariance. Moreover, invertibility of individual FiLM transformations does not establish equivalent geometry across the complete generator and discriminator networks. These gaps weaken the paper’s central mechanistic explanation, although they do not invalidate the reported improvements.
2.The generation comparison does not isolate the tokenizer’s contribution. Appendix D.2 also strengthens the autoregressive generator through adaptive conditioning and other refinements, but Table 2 provides no comparison holding the generator, training budget, and sampling settings fixed. This matters given the relatively small FID gains over IBQ. Reproduction is further hindered by unspecified training durations, the exact 604M architecture, and selected noise/EMA hyperparameters.
Questions for authors
1.What additional argument connects Theorem 9’s local Jacobian result to codebook PCA effective dimension, and establishes transfer between generator and discriminator modes? 2.Do the generation gains persist when VQ-DSGAN and IBQ tokenizers use the same autoregressive architecture, training budget, and sampling protocol? 3.Please specify the training durations, exact 604M configuration, sampling settings, EMA decay, and chosen noise coefficient.