- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes a two-stage generative framework. The key observation is that an image and its low-frequency representation naturally induce an optimal transport coupling, allowing the model to learn a coarse-to-fine transformation without explicitly solving the OT problem. To enable sampling from the proposed low-frequency prior at inference time, the authors additionally train a flow matching model to map Gaussian noise to the low-frequency image distribution. Since this intermediate distribution lies in a substantially lower-dimensional space, the authors argue that it can be modeled more efficiently using a lightweight network with fewer parameters and lower computational cost.
Critical strengths
The paper provides a theoretical analysis of the OT properties of low-frequency projection-based couplings, complemented by empirical investigations of their robustness to noise interpolation.
Critical weaknesses
-
My primary concern is the limited novelty of the proposed method. PixelFlow[1] has already introduced a multi-stage coarse-to-fine generation framework that progressively generates images from low to high resolution during noise schedule. In contrast, this paper adopts a simplified two-stage pipeline, where a lightweight FM model first generates a fully denoised low-frequency image, followed by a noise-perturbed refinement stage. Despite this difference, the core idea remains highly similar. Both methods exploit low-frequency image representations to construct correlated training pairs for coarse-to-fine flow matching. Given the substantial methodological overlap, I find it difficult to identify a sufficiently novel algorithmic contribution. More concerningly, PixelFlow is neither discussed nor included as a baseline, making it difficult to assess the actual contribution of this work.
-
Another concern is that the strong few-step generation performance may largely depend on the Reflow procedure applied to the first-stage generator, rather than the proposed OT-based coupling. Since the first stage is responsible for generating the global image structure, its ability to produce high-quality samples in a single step is crucial to the entire pipeline. However, the paper does not provide an ablation without Reflow, making it difficult to disentangle the contribution of Reflow from that of the proposed coupling strategy.
[1] PixelFlow: Pixel-Space Generative Models with Flow
Questions for authors
-
Although the paper reports results on 256×256 images, these experiments are conducted in a 32×32 VAE latent space rather than directly in pixel space. It would be interesting to discuss whether the proposed two-stage framework could effectively scale to high-resolution pixel-space generation, and what potential challenges might arise in such settings.
-
Figure 6(a) only reports FID at 1, 2, and 4 NFEs. Could the authors extend the comparison to larger numbers of sampling steps (e.g., 8, 16, and 32 NFEs)? It would be interesting to see whether the performance advantage persists or diminishes as the number of sampling steps increases.
-
Could the authors provide an ablation study comparing the proposed framework with and without Reflow in the first-stage generator? In particular, how does removing Reflow affect the final few-step generation quality? This would help clarify to what extent the reported improvements depend on Reflow rather than the proposed OT-based coupling.
3 comments
Thank you for your answers. My concern is not simply that the method’s advantage diminishes at larger NFE, but that the learned prior ultimately becomes a liability despite being introduced to simplify generation. Stage 2 receives coarse structure in advance and uses the same backbone and training-step budget as the baselines, while the full system additionally benefits from a separately trained stage-1 generator. Nevertheless, Table B.5 shows that at 32 NFE, it underperforms every compared baseline on both CIFAR-10 and FFHQ—including standard IFM, which must generate the entire sample from unstructured noise.
The oracle comparison makes this limitation particularly clear: keeping stage 2 unchanged and replacing only the generated low-frequency component with real low-frequency representations reduces FID from 7.54 to 5.01 on CIFAR-10 and from 21.89 to 13.99 on FFHQ. Thus, the second-stage model can achieve substantially better results, but the proposed prior generator prevents the full system from realizing that capability. More accurate integration does not resolve this bottleneck. This is not merely a secondary implementation issue. Generating a sufficiently accurate prior is an essential part of the proposed method, and attributing the performance gap to stage 1 does not address the weakness of the complete approach.
How do the authors propose to overcome this limitation—either by improving the learned prior or by training stage 2 to accommodate and correct its errors? Stage 2 is trained with starting states constructed from real low-frequency representations, whereas inference uses generated ones. The current results suggest that this discrepancy remains consequential even at larger sampling budgets. Without addressing this bottleneck, the method offers a low-NFE shortcut but falls short of the stronger promise of prior design. It supplies additional structure to simplify the task, yet at larger sampling budgets produces worse samples than a standard model that receives no such structural head start.
We thank the reviewer for the follow-up. We would like to clarify our choice of the stage 1 generator. We chose to change as little as possible in stage 1, and we use a basic, small and lightweight version of the stage 2 model. This was done to keep the pipeline as simple as possible. As the two stages are decoupled, the stage 1 prior can be improved orthogonally to the rest of the solution. As an example, the table below shows that using a pretrained generator as stage 1 gives a lower FID than IFM. We believe that our 2-part architecture, with the 2nd stage achieving empirical OT coupling, is our key contribution, and we hope that improving each of the two stages directs future work in a new, underexplored direction.
The table reports FID-50k on FFHQ at 16 and 32 NFE. As in the few-step results of the paper, our rows tune the inference noise level per step count:
| 16 NFE | 32 NFE | |
|---|---|---|
| IFM | 18.79 | 16.43 |
| IFM, trained 2.6% more steps | 19.07 | 16.40 |
| Ours | 19.87 | 18.62 |
| Ours, stage 1 replaced by a pretrained StyleGAN2 | 17.44 | 16.24 |
| Ours, stage 2 fine-tuned for 2.6% more steps on generated starting states | 19.14 | 17.89 |
Overcoming the limitation: We explored both ways the reviewer suggested, and both show directions to improve the full system. Improving the prior is the StyleGAN2 row discussed above. To train stage 2 to correct the prior's errors, we fine-tune it on starting states where half of the real low-frequency priors are partially noised and regenerated by , so they carry its errors while staying paired with their training images (last row). IFM trained for the same extra steps does not improve.
Following the reviewer's comment, we will add these rows to Tab. B.5 and this discussion to Sec. 6 in the final version. Finally, the reviewer's follow-up concerns only the higher-NFE question. We would be grateful to know whether our replies on PixelFlow (W1), the role of Reflow (W2, Q3) and high-resolution pixel space (Q1) resolved those concerns, or which issues remain, so that we can address them during the discussion period.
We thank the reviewer for the constructive feedback.
Relation to PixelFlow (W1): We agree that PixelFlow is closely related. Following the reviewer's comment, we now discuss it in Sec. 2.2 and compare against it directly in Sec. 5 and App. B.4. The difference is where the low-resolution image comes from. In PixelFlow it is a noisy intermediate state produced by the same network, so all four of its resolution stages must run at every sampling budget. In our method it is a clean sample from a separate lightweight prior, decoupled from the second stage. Moreover, PixelFlow does not study optimal transport or other data couplings for training, which are our focus.
To compare with PixelFlow directly, we run the released ImageNet-256 PixelFlow checkpoint in our evaluation pipeline: FID-10k with the same reference statistics, and time per image on one A5000 GPU, including decoding (Fig. B.5). Its network is much larger (677M parameters, in pixel space, against 163M for our two networks), so step counts are not comparable and we compare time per image. At comparable FID ours is about 15× faster: 9.73 in 56 ms per image, against 10.25 in 837 ms for PixelFlow. In this setting, PixelFlow reaches a lower FID (8.33) with four steps per stage, at 30 times our cost. The two methods are also complementary: replacing PixelFlow's first stage with our prior improves its FID at nearly the same cost (Fig. B.5).
Role of Reflow (W2, Q3): Reflow is used in our pipeline only for speed, to reduce the overhead of generating the prior by cutting the number of Gϕ steps. Without Reflow, Gϕ needs several steps to produce good samples. For example, 8 steps cost about 0.72 of one vθ step, and Reflow cuts this overhead to about 0.09 with a single step. Reflow is not part of the coupling between the prior and the data, and Gϕ is only a sampler of the low-frequency prior, so any other sampler could replace it. The requested ablation is in the new Tab. C.8 (App. C.3). It shows that removing Reflow and running Gϕ for more steps gives a comparable FID. Thus, the gain does not depend on Reflow, which only reduces the cost of the prior. We also note that there is no clear way to compare against other methods with Reflow added only to a first stage, since they do not split generation into a prior stage and a refinement stage. We would be glad to run any further comparison the reviewer finds useful during the discussion period.
Pixel space at high resolution (Q1): Following the reviewer's remark, the revised version discusses high-resolution pixel-space generation in App. B.5. We expect the identity coupling to carry over to high resolution pixel space, since natural images have a steeper spectrum in pixels than in VAE latents, so the low-frequency projection keeps more of each image (Tab. B.4). The main challenge is cost. In pixel space both stages grow with the resolution. The prior can be kept small either by a larger downsampling factor, which loses structure (App. C.2), or by splitting its generation into further stages, each with its own decoupled prior. We did not train at this scale and leave it to future work.
Higher NFE (Q2): Our method is designed for the few-step regime and does not perform as well at high NFE. Our advantage comes from straighter trajectories, which need fewer steps to integrate accurately. With more steps the integration error shrinks for every method, so the advantage diminishes as the number of steps grows, as we expect. Following the reviewer's request, we extend the comparison to 32 NFE in Tab. B.5. On CIFAR-10 we still lead every baseline through 16 NFE. On FFHQ the strongest baseline, AlignFlow, leads from 8 NFE. The table also includes the noised oracle of App. B.2, the same vθ started from exact low-frequency images of training samples. The oracle leads every baseline on FFHQ from 8 NFE, so the gap at higher NFE comes mainly from the prior generator Gϕ and not from the flow vθ. The oracle is a bound, since these priors come from training images. The revision also scopes the claims in Sec. 5 to the few-step regime.