- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes a multi-step VAE decoder that uses rectified flow to decode latent to image. The decoder works as an encoder by reversing the flow direction and encoding images down to latents. The central motivation to do this is to support generation quality by allowing the VAE to potentially fix a one-step latent to image process that might have errors from imperfect generated latents. As a consequence of this pipeline, the encoding-decoding process requires less parameters because the decoder can act as an encoder.
Critical strengths
S1: The paper proposes the novel concept of decoding latents through a continuous path instead of a one-step latent to image jump.
Critical weaknesses
W1: The paper needs to be redesigned with significant effort on writing and presentation of the text. L267 has "generation quality" folded into the reconstruction quality section in several areas where it is not possible to pin down what is being discussed. For reconstruction quality, the authors seem to use global metrics like rFiD instead of per-sample metrics that are important to discuss. Per sample metrics are sidelined into the appendix, and even then do not compare the default RAC against its own SD-VAE teacher. There is no LPIPS anywhere either. As a paper focused on improving VAE quality, this seems to be a poor set of experiments to evaluate quality.
W2: The paper discusses 5 different components for loss computation but never ablates any of them. How did the authors come upon 4 different hyper-parameters for training? The ablation discusses some configurations but they are nowhere enough.
W3: The paper does not attempt to scale image generation or reconstruction beyond 256 px resolution. Higher resolutions might be slower or more computationally intensive at multi-step decoding. 256px resolution is already slower than the SD-VAE. Finally, quality needs to be consistent across resolutions and some form of scaling is required in the experiments.
W4: The paper claims that latents are being improved but trains in the same latent space with the same encoder teacher. How is the generation v/s reconstruction gap handled without changing the latent space?
Questions for authors
- 4 sign-language papers from Fang et al. are cited but appear to be off-topic. Please justify or remove.
- See weaknesses.