- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes Paired Flow Matching (PairedFM), a framework that jointly learns representations and observation-space flows without an explicit reconstruction objective. By conditioning the flow on learned source and target representations, the method aims to overcome the reconstruction bottleneck of conventional autoencoders.
Critical strengths
1.The paper presents an interesting perspective on jointly learning representations and generative models without relying on an explicit reconstruction objective.
2.The theoretical analysis provides useful insights into representation identifiability and generation guarantees under certain assumptions.
Critical weaknesses
1.Lack of validation on modern generative models. The experiments are primarily conducted on synthetic or relatively simple datasets. Despite the paper's motivation concerning the limitations of VAEs and latent generative models, there are no evaluations on modern image generation architectures such as DiT or SiT. Without experiments on realistic image generation benchmarks, it is difficult to assess the practical effectiveness and scalability of the proposed framework. This is a major concern.
2.Insufficient comparisons with representative VAEs. The paper does not compare against widely adopted visual tokenizers, such as SD-VAE or the FLUX series of VAEs. These comparisons are essential to establish whether the proposed method offers meaningful advantages over modern autoencoding approaches, particularly in terms of representation quality, reconstruction fidelity, and downstream generation performance.
3.Limited evaluation of generalizability. The paper lacks experiments across different image resolutions and more challenging generation tasks, such as text-to-image (T2I) generation. Evaluations on realistic, high-resolution datasets would substantially strengthen the empirical evidence and clarify the applicability of the proposed method.
Questions for authors
1.Lack of validation on modern generative models.
2.Insufficient comparisons with representative VAEs.
3.Limited evaluation of generalizability.
3 comments
Can you please respond to the authors even in the last minute?
Thank you
Thank you very much for your efforts; however, the authors have not provided any additional, compelling experiments to demonstrate effectiveness. I have decided to maintain my score.
Response to Reviewer YTQy
We appreciate your recognition of the reconstruction-free learning perspective and the insights provided by the theoretical analysis. We have substantially expanded the empirical evaluation in the revised manuscript, adding a spatial DiT velocity network, pretrained SD-VAE comparisons, natural-image experiments across three resolutions, and an exploratory text-conditioned generation study. These additions also address R1's scalability questions and R2's request for more realistic representation evaluation. The added material is marked in dark red in the revised manuscript.
We respond to your concerns and questions point by point below.
W1. Validation on modern generative architectures
We have added natural-image experiments on CIFAR-10, Tiny ImageNet, and FFHQ (Appendix A.4), together with a spatial DiT implementation of the PairedFM velocity network (Appendix A.5, Table~14). The DiT and U-Net velocity networks are each trained across three seeds using the same frozen encoder and training budget. Both support code-directed image recovery, with the U-Net achieving better fidelity at the evaluated budget. These additions demonstrate PairedFM on natural images and its compatibility with a transformer-based velocity architecture.
W2. Comparisons with representative VAEs
We now compare against pretrained SD-VAE, evaluating image recovery and frozen-code classification (Appendix A.4, Tables 12 and 13). In the one-step comparison, PairedFM achieves lower pixel MSE on all three image benchmarks. SD-VAE achieves lower LPIPS using the resized reference on CIFAR-10 and native resolution on Tiny ImageNet and FFHQ. PairedFM also attains comparable classification accuracies under the common linear-probe procedure.
These results support useful representations and competitive recovery, with a tradeoff between pixel and perceptual fidelity. The manuscript specifies the differences in code size, source access, and training history.
W3. Generalizability across resolutions and text-conditioned generation
The expanded experiments cover image resolutions of 32×32, 64×64, and 128×128. We also added text-conditioned generation on FFHQ (Appendix A.6, Table
15 and Figures15--16). A learned prior generates target codes from noise and captions, which condition the PairedFM flow from a source image. The target code is therefore sampled rather than computed from an observed target image.At guidance 2, PairedFM with one image step achieves higher CLIP alignment than the decoder of the same sampled codes (0.2607 versus 0.2480), together with greater caption-swap responsiveness. The figures illustrate responses to face and clothing descriptions. These results demonstrate compatibility with text conditioning in the evaluated FFHQ setting. Within the revision period, we expanded the evaluation to natural images at three resolutions and text-conditioned generation on FFHQ. These additions demonstrate PairedFM's applicability beyond the original controlled settings, including caption-conditioned generation in the face domain.
Q1. Validation on modern generative models
Please see our response to W1 above.
Q2. Comparisons with representative VAEs
Please see our response to W2 above.
Q3. Generalizability
Please see our response to W3 above.
We hope these responses address your concerns and welcome any follow-up questions.