- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The authors analyse the receptive field (set of pixels) influencing each token, and replace components of the encoder by more spatially-constrained components, both reducing the receptive fields and improving encoder efficiency. They show that a local-biased encoder improves both rFID and gFID.
Critical strengths
The authors provide extensive experiments to study the impact of their design choices on rFID and gFID, providing clear justification for their design choices.
Critical weaknesses
- The main concern about encoders for latent diffusion is the quality of the latent space for latent generative models, and as such, the gFID obtained with such models. The experiments in this direction are very weak: the paper only uses under-trained DiT models. Comparisons using fully-trained DiT, and possibly a more modern Flow-Matching baseline (SiT) would greatly strengthen the results. Additionally, evaluation without CFG doesn’t correlate with the actual quality of the model, and results should be reported with an optimal-CFG evaluation sweep. Results from Table 7 seem to indicate that, under per-setting optimal CFG, the SD-VAE baseline beats the proposed method.
- The concern about the encoder efficiency seems mostly irrelevant in the studied setting (latent diffusion), as the decoder is used once a dataset for training, and never at sampling time. With this consideration, the second part of the method (LatentMaid) is unjustified and superfluous as it is consistently outperformed by the base ABCRP.
Questions for authors
Could the author provide proper evaluation of the resulting encoder(s) with complete training, modern latent model (SiT), and more importantly, with per-encoder optimal CFG?