- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes LP-GAN (Latent Prior GAN), which enhacnes the vae decoder by post-training of it using adversarial and paired reconstruction loss. The main motivation is that two-stage latent generative models does not consider the gap between the distribution of generated latents and ground-truth latents (encoded by vae). To reduce this gap, they suggest to finetune the vae decoder with extra adversarial training for reducing the divergence between the rgb images rendered from generated latents and groud-truth one. Also, for maintaining the reconstruction quality (correspondence to its input), they additionally introduce the reconstruction objective for real latent. Experimental results show that LP-GAN successfully reduces FID compared to iMF-XL/2 with lower computational cost compared to generative models (e.g., GAN) training from scratch.
Critical strengths
-
The motivation and approach of LP-GAN make sense for me. The problem they suggest is well-motivated and important research direction, as I believe. Similarly, the solution they propose is intuitive and expect to work well.
-
Quantitative results show that the proposed method successfully enhances the metrics such as FID compared to the pure latent generative models which does not use any post-training or finetuning methods.
Critical weaknesses
-
The most critical weakness is that their motivation is not that new. In other words, the importance of correction of generated latent and ground-truth latent is already considered in prior works. For example, every teacher forcing relevant techniques (especially for VAR [2, 3]) suggests to reduce the distributional gap between the generated and actual tokens. Thus, as I understand, the motivation and approach of LP-GAN is not that new and its contribution would be a proper application of finetuning of decoder in one-step generative model.
-
The paper shows the quantitaive comparison mainly focused on FID. However, as the generative backbone (such as diffusion transformers) are frozen, the metrics beyond distributional matching should be reported, as this part would be already modeled in backbone. For example, what property of samples from the enhanced decoder actually enhances? It is not intuitive for me and qualitative examples in appendix also does not imply actual difference.
-
Training of LP-GAN uses pretrained vision encoder-based discriminator (e.g., dino). As these pretrained encoder have features space favorable to the FID, as they are both trained on ImageNet dataset and downstream task. I suspect that this may largely affect to FID enhancement rather it actually enhances the visual quality of distributional gap between the generated and actual data. I believe this discriminator design should be analyzed more specifically for proving the effectiveness of LP-GAN.
[1] Zheng, Boyang, et al. "Diffusion transformers with representation autoencoders." ICLR, 2026.
[2] Rajagopalan, Sudarshan, Kartik Narayan, and Vishal M. Patel. "Restorevar: Visual autoregressive generation for all-in-one image restoration." ICLR, 2026.
[3] Han, Jian, et al. “Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesis.” CVPR, 2025.
Questions for authors
First of all, please see the critical weaknesses.
- Can the post-trained decoder be applied to the other types of generative models? For example, if decoder trained with improved meanflow is used for original meanflow, does it enhances FID scores?