acceptodds
Under review as a conference paper at ICLR 2027

ROMS-IMLE: A Minimalist Approach to Competitive Single-Step Generative Modelling

13 views

Abstract

Generative models have undergone many generations of evolution, from VAEs/GANs to diffusion/flow matching. Along the way, the underlying techniques have become more complicated and various beliefs about what drives strong empirical performance have taken hold. Due to the success of diffusion models and flow matching, one of the more common beliefs is the importance of transforming the noise distribution to the data distribution gradually through many small transformations. We ask whether this is truly necessary, and take a minimalist approach to designing a competitive generative model. We start with the bare-bones essentials, namely just a training objective and a model. We purposefully make both simple. For the training objective, we choose Implicit Maximum Likelihood Estimation (IMLE), and eschew more complicated alternatives such as variational inference, adversarial training and numerical integration. For the model, we eschew transformers and instead choose a moderately sized convolutional network. Then we judiciously added elements that are truly essential, which surprisingly do not include iterative denoising. The result is a single-step parameter-efficient generative model that produces high quality samples at fast speed: it achieves an FID of 2.56 on ImageNet 256 and simultaneously attains good precision and recall.

Primary area:
generative models
Keywords:
image generation, generative models, diffusion, flow matching, IMLE

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_566WOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes an IMLE training recipe combining supervision at intermediate decoder resolutions with a Geman–McClure robust loss for matched real/generated pairs. It uses ConvNeXt generators for pixel-space and latent-space generation. Experiments cover CIFAR-10, CelebA-HQ, and ImageNet; on ImageNet, round-trip rejection improves the reported FID from 4.16 to 2.56.

Critical strengths
  1. The Flowers ablation provides clear evidence of improvement within the tested setting. With the same backbone and training iterations, multi-stage supervision improves FID from 62.61 to 31.24, and adding the robust loss further improves it to 13.90, with three-seed statistics (Table 1). The attention-based variant also shows the same qualitative trend (Appendix F).

  2. Demonstrating useful image generation with a non-adversarial, single-pass IMLE generator is valuable. The evaluation spans pixel and latent spaces, and Table 4 explicitly reports ImageNet results both before and after rejection, allowing readers to distinguish the underlying generator from the filtered sampling procedure.

Critical weaknesses
  1. The contribution of the proposed training changes is insufficiently established. CHIMLE (Peng et al., NeurIPS 2022, Sections 2.2–2.3) already introduces intermediate output heads, downsampled targets, and supervision at multiple generator resolutions within IMLE. Its image-conditioned setting and hierarchical latent search differ from ROMS-IMLE, but this directly relevant precedent is not discussed, leaving the novelty of multi-stage supervision unclear. Empirically, the component ablations are confined to Flowers. The ImageNet experiment uses a different latent-space setup with EQ-VAE, without controlled evidence isolating the training changes there. Thus the substantial Flowers gains do not yet establish the contribution of these changes at the scale underlying the main performance claim. CHIMLE

  2. The method specification and resource comparison leave the practical result difficult to assess. The Geman–McClure formula/scale and its application within the pixel-space LPIPS/pixel-loss combination are unspecified. Algorithm 1 uses one distance for matching and regression, whereas Appendix D describes LPIPS/L2 matching and a different regression loss; the implemented objective needs to be stated explicitly. Main-benchmark training budgets and measured matching costs are also missing. For ImageNet, the headline FID additionally uses rejection, whose encoder/decoder/LPIPS costs are not captured by generator NFE. Finally, the cited iMeanFlow reports a 174M-parameter, 1-NFE model with FID 2.27, while Table 4 includes only its 610M model. This smaller operating point is relevant to the claimed parameter efficiency of the 310M ROMS model. These reported results use different training setups, so they motivate a clearer comparison rather than establish a matched-budget ranking. iMeanFlow, Table 2

Questions for authors
  1. What is the substantive distinction from CHIMLE's intermediate supervision? Can you provide a representative ImageNet ablation, holding the backbone, tokenizer, and training budget fixed, to establish the gains from multi-stage supervision and the robust loss before rejection?

  2. What are the exact matching and regression objectives, including the robust-loss scale and pixel-space combination? Please also specify the ImageNet conditioning/matching procedure, learning rate, batch size, and training duration.

  3. What are the measured training cost, including nearest-neighbour search, and sampling throughput before/after rejection? Please specify the rejection threshold and its selection, and contextualize the cost/quality trade-off against the smaller iMeanFlow models already available in the cited work.

7 comments
AuthorsOpenReview ↗

Rebuttal Part-1

We thank the reviewer for their feedback. We are happy that the reviewer found our paper useful, valuable and clear.

UPDATED RESULTS

Since the reviewer had doubts about the roundtrip rejection, we decided to remove it all together. It turns out our model faced the same misalignment issue identified in RSIMLE [f]: the latent codes matched during training differ in distribution from those sampled at test time. Finetuning our existing model with RSIMLE objective (ϵ\epsilon = 6.0 in EQVAE latent space) resolves this, so no inference-time filtering is needed. Our aim in this paper is to propose a minimal recipe, a simple objective and a convolutional network, that achieves competitive performance. Modern state-of-the-art models, by contrast, combine many sophisticated techniques, such as advanced architectures, complex feature spaces and additional losses. To show that our minimal recipe can benefit from such techniques as well, we added one of them, FD-loss [g] to our existing model, which brings our model close to state-of-the-art performance. Here are the updated results:

ModelFID
ROMS-IMLE (with RS-IMLE training)2.86
ROMS-IMLE + FD Loss0.83

We discuss these results and address the concerns of the reviewer below:

W1 stands for Weakness 1, Q1 stands for Question 1

  1. [W1,Q1] Comparision with CHIMLE

    We thank the reviewer for bringing this up and we updated the main paper to cite CHIMLE. As the reviewer pointed out, CHIMLE does have some overlap with our contributions. However, there are key differences which separates our work from CHIMLE:

    1. CHIMLE addresses image-to-image tasks, where the output is conditioned on an input image (e.g., a low-resolution or grayscale version of the target) that already fixes much of the target’s structure. This makes it an easier generative modelling problem: the model only needs to capture the variation consistent with the input. ROMS-IMLE generates from pure noise alone, which is a substantially less constrained problem, making it a much harder task.
    2. Another difference is the matching phase; CHIMLE selects a latent component per resolution. We match one latent code once, on the final output, and supervise all stages toward the same matched target.
  2. [W1, Q1] ImageNet ablations

    We trained our model from scratch for 50K iterations, using the same training budget and backbone. Here are the results, evaluated over 50K generated samples:

    ConfigurationFID ↓Precision ↑Recall ↑
    Vanilla IMLE149.920.120.15
    + Robust loss only165.380.120.06
    + Multi-stage + robust loss (ROMS-IMLE)130.360.170.18

    Even early in training (50K iterations), ROMS-IMLE already improves FID, precision and recall over vanilla IMLE. In general, it is established practice to conduct ablation studies and hyperparameter tuning on smaller datasets, and finally transferring the best configuration to a large scale. Ablating on a large dataset like ImageNet is very computationally expensive, and repeating it per component till convergence is beyond our academic budget. This is the reason why we choose to do ablation over smaller datasets, which is consistent with the rest of the field (Sphere Encoder 2[a], EDM[b], Consistency Models[c], Score SDE[d]).

  3. [W1] The ImageNet experiment uses a different latent-space setup than Flowers ablation. Claims do not transfer.

    We would like to clarify that the Flowers ablation is comparable to ImageNet experiment: both are trained in the same EQ-VAE latent space. They differ in dataset size, conditioning (unconditional vs. class-conditional) and model size (95.8M vs. 310M parameters). We have updated the paper to clarify this. As mentioned above, ablating on a smaller scale is standard practice in the field.

AuthorsOpenReview ↗

Rebuttal Part-2

  1. [W2] Comparison to other baselines like smaller iMeanFlow

    We thank the reviewer for pointing to iMeanFlow’s smaller models, which we have added to Table 4. It is important to not that different generative models have vastly different training procedures. In this instance, iMeanFlow builds on a transformer backbone, while our generator is a convolutional network. Other techniques like Drifting use other advancements like training in a MAE feature space. As the reviewer notes, a comprehensive parameter-matched comparison across methods with such different training strategies is difficult, and it is not the aim of this paper. Our goal in this paper is not a parameter-matched race for the best score, but to show that a minimal recipe is already competitive. To demonstrate that this minimal recipe is not a ceiling and our model can too benefit from recent advances, we added FD-loss (Yang et al., 2026), to the same 310M model: it reaches FID 0.83 at 1 NFE. Other techniques, such as stronger architectures and feature encoders, could potentially be added in the same way. Our focus, however, remains on showing that a minimal recipe alone is sufficient for competitive generation.

  2. [Q2, W2] Robust Loss scale and configuration

    We use the Geman–McClure loss:

    ρ(r)=r2r2+s2,s=0.5\rho(r) = \frac{r^2}{r^2 + s^2}, \qquad s = 0.5

    Latent space (ImageNet). At each stage ll, ρ\rho is applied elementwise to the residual between the stage output x~l\tilde{x}^l and the downsampled target Kl(x)K_l(x) in the EQ-VAE latent space, and averaged over elements:

    Ll=1∣Dl∣∑jρ(x~jl−Kl(x)j)\mathcal{L}^l = \frac{1}{|D_l|} \sum_{j} \rho\big(\tilde{x}^l_j - K_l(x)_j\big)

    Pixel space (CIFAR-10, CelebA-HQ). The loss combines a perceptual (LPIPS) term and a pixel term with weights 1.0 and 0.1 respectively (Appendix D):

    Ll=ρ(LPIPS(x~l,Kl(x)))+0.1⋅1∣Dl∣∑jρ(x~jl−Kl(x)j)\mathcal{L}^l = \rho\big(\mathrm{LPIPS}(\tilde{x}^l, K_l(x))\big) + 0.1 \cdot \frac{1}{|D_l|} \sum_{j} \rho\big(\tilde{x}^l_j - K_l(x)_j\big)

    The pixel term applies ρ\rho elementwise, as in the latent-space case. For the perceptual term, we apply ρ\rho to the LPIPS distance of each image pair rather than to individual feature differences. LPIPS combines feature differences using learned weights calibrated for an ℓ2\ell_2 distance, so applying ρ\rho to individual features would change what LPIPS measures.

  3. [Q3, W2] Method specifications for ImageNet experiments.

    We are happy to provide requested implementation details for ImageNet experiments. We have updated the Appendix with these additional details.

    1. Training duration: We initially trained for 2 weeks on 8 L40s GPU. The finetuning with RSIMLE was done for an additional 2 days. For the FD loss finetuning, we trained for an additional 1 day for 50K iterations.

    2. Learning rate: 2e-4 with cosine annealing

    3. Batch size: 1024

    4. Matching cost: For ImageNet, the nearest neighbour search happens within each class (condition). This is standard practice in conditional IMLE [e] literature. Nearest-neighbour search is highly optimized in modern libraries such as FAISS, which we use on GPU. In aggregate, nearest-neighbour search represents less than 1% of the total training time.

    5. Conditioning: Each class has a separate learned class embedding vector ece_c, which is added to the noise vector zz. The sum z+ecz + e_c is passed to the mapping network. We have updated Figure 8 to show this.

    6. Sampling throughput: The results were measured on a single NVIDIA L40S GPU with batch size 256, bf16 and torch.compile for the generator, timed after warm-up. Sampling is a single forward pass of the generator (1 NFE) followed by one VAE decode. The VAE decode runs in fp32, the same setting used for the reported FID.

      PipelineThroughput (img/s)
      Generator only (noise, class → latent)410
      Generator + VAE decode (noise, class → 256×256 image)82

      The VAE decoder accounts for ~80% of end-to-end time, while the generator takes ~2.4 ms per image. Latent diffusion models pay the same decode cost once per image, on top of 250 sampling steps (DiT, SiT). Since we no longer rely on roundtrip rejection, we omit the cost for it for the sake of clarity.

AuthorsOpenReview ↗

Rebuttal Part-3

  1. [W2] Algorithm 1 uses one symbol d for matching and regression, but Appendix D describes different ones.

    The matching phase depends only on the relative ranking of samples, not on the distance values themselves. As long as matching phase determines a sensible ranking between the generated samples, the IMLE procedure works fine. Hence, the matching distance need not be the same as the training loss; prior IMLE work also use different functions for the two phases(AdaIMLE [h], RS-IMLE [f]). We have updated Algorithm 1 to make this explicit: it now uses d′d' for nearest-neighbour matching and dd for the per-stage loss.

We have updated the relevant sections of the paper, with all changes highlighted in blue. Your input is valuable to us, and we would be happy to address any remaining concerns if you still have any. If you feel your concerns have been resolved, we would be grateful if you would kindly consider updating your score. Thank you again for taking the time and effort to review our paper.

[a] Yue, Kaiyu, et al. "Sphere Encoder 2." arXiv preprint arXiv:2610.02208 (2026).

[b] Karras, Tero, et al. "Elucidating the design space of diffusion-based generative models." Advances in neural information processing systems 35 (2022): 26565-26577.

[c] Song, Yang, et al. "Consistency models." arXiv preprint arXiv:2303.01469 (2023).

[d] Song, Yang, et al. "Score-based generative modeling through stochastic differential equations." arXiv preprint arXiv:2011.13456 (2020).

[e] Li, Ke, et al. "Multimodal image synthesis with conditional implicit maximum likelihood estimation." International Journal of Computer Vision 128.10 (2020): 2607-2628.

[f] Vashist, Chirag, Shichong Peng, and Ke Li. "Rejection sampling imle: Designing priors for better few-shot image synthesis." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.

[g] Yang, Jiawei, et al. "Representation Fr'echet Loss for Visual Generation." arXiv preprint arXiv:2604.28190 (2026).

[h] Aghabozorgi, Mehran, Shichong Peng, and Ke Li. "Adaptive IMLE for few-shot pretraining-free generative modelling." (2023).

Reviewer_scheOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ROMS-IMLE, combining multi-resolution supervision and a robust Geman–McClure loss to improve one-step generation. Experiments cover CIFAR-10, CelebA-HQ, and ImageNet 256. On ImageNet, the method achieves FID 4.16, improving to 2.56 with round-trip rejection sampling.

Critical strengths
  1. The method is simple and addresses weaknesses of IMLE: limited intermediate supervision and unreliable nearest-neighbor matches.

  2. The Flowers ablation shows substantial improvements that FID decreases from 62.61 to 31.24 with multi-stage supervision, and to 13.90 with the robust loss. Experiments span pixel and latent-space generation. The attention-based backbone experiment provides additional evidence that the training recipe transfers across architectures.

Critical weaknesses
  1. Ablations are mainly done on the small Flowers dataset. The paper shows main FID results on ImageNet, but it lacks controlled comparisons isolating multi-stage supervision and robust loss, despite using a different representation and model scale. The robust-loss-only configuration is also missing from the main ablation. The paper claims "This training-time property, rather than iterative sampling, is what we believe is the reason for high-quality generation in SI models." However, reinterpreting diffusion sampling as a composition of stages does not establish that intermediate supervision explains its success. The experiments show that the supervision helps IMLE, but do not discuss diffusion’s other properties. Further, the diffusion process is different from the multi-stage process used in the method. The explanation is not clear and convincing enough.

  2. The baselines compared in the paper are not sufficient. There are many relevant strong one-step methods such as Drifting and W-Flow that are not included in the comparison. The pretrained components seem to be different with those papers, but ROMS also incurs candidate-generation and nearest-neighbor-search costs during training. Its best FID also requires extra autoencoder and LPIPS evaluations, and is still much higher than those baselines. The quality and efficiency trade-off is unclear. Also, some important implementation details are not specified. For example, the robust-loss formula and scale, its combination with the pixel-space perceptual loss, the complete ImageNet training configuration, and rejection-threshold selection.

Questions for authors

Could the authors address the concerns raised in the Weaknesses section, particularly the missing ImageNet ablations, computation costs comparison and comparison with more baselines, and underspecified implementation details?

7 comments
AuthorsOpenReview ↗

Rebuttal Part-1

We thanks the reviewer for their feedback. We are glad that the reviewer found our method intuitive to understand and supported with extensive experiments.

UPDATED RESULTS

Since the reviewer had doubts about the roundtrip rejection, we decided to remove it all together. It turns out our model faced the same misalignment issue identified in RSIMLE [f]: the latent codes matched during training differ in distribution from those sampled at test time. Fine-tuning our existing model with RSIMLE objective (ϵ\epsilon = 6.0 in EQVAE latent space) resolves this, so no inference-time filtering is needed. Our aim in this paper is to propose a minimal recipe, a simple objective and a convolutional network, that achieves competitive performance. Modern state-of-the-art models, by contrast, combine many sophisticated techniques, such as advanced architectures, complex feature spaces and additional losses. To show that our minimal recipe can benefit from such techniques as well, we added one of them, FD-loss [g] to our existing model, which brings our model close to state-of-the-art performance. Here are the updated results:

ModelFID
ROMS-IMLE (with RS-IMLE training)2.86
ROMS-IMLE + FD Loss0.83

We discuss these results and address the concerns of the reviewer below:

W1 stands for Weakness 1, Q1 stands for Question 1

  1. [W1] Robust only loss is missing from the main ablation (Table 1)

    We are happy to provide the requested ablation. The below table follows the same setting as Table 1 of the main paper. Rows F–G add a robust loss without multi-stage supervision.

    ConfigurationFID ↓Precision ↑Recall ↑
    AVanilla IMLE62.61 ± 2.280.45 ± 0.030.46 ± 0.01
    B+ multi-stage supervision31.24 ± 1.080.79 ± 0.010.72 ± 0.02
    F(A) + Pseudo-Huber loss26.89 ± 0.910.80 ± 0.010.87 ± 0.00
    G(A) + Geman-McClure loss21.45 ± 0.450.86 ± 0.000.86 ± 0.02
    C(B) + Cauchy loss19.80 ± 0.580.88 ± 0.010.74 ± 0.01
    D(B) + Pseudo-Huber loss16.49 ± 0.090.91 ± 0.000.72 ± 0.02
    E(B) + Geman-McClure loss (ours)13.90 ± 0.080.93 ± 0.000.73 ± 0.02

    Our ablation show that each component (multi-stage supervision and robust loss) improves on vanilla IMLE on its own, and combining them (Row E, our method) gives the best FID.

  2. [W1] ImageNet ablations

    We trained our model from scratch for 50K iterations, using the same training budget and backbone. Here are the results, evaluated over 50K generated samples:

    ConfigurationFID ↓Precision ↑Recall ↑
    Vanilla IMLE149.920.120.15
    + Robust loss only165.380.120.06
    + Multi-stage + robust loss (ROMS-IMLE)130.360.170.18

    Even early in training (50K iterations), ROMS-IMLE already improves FID, precision and recall over vanilla IMLE. In general, it is an established practice to conduct ablation studies and hyperparameter tuning on smaller datasets, and transferring the best configuration to a large scale. Ablating on a large dataset like ImageNet is very computationally expensive, and repeating it per component till convergence is beyond our academic budget. This is the reason why we choose to do ablation over smaller datasets, which is consistent with the rest of the field (Sphere Encoder 2[a], EDM[b], Consistency Models[c], Score SDE[d]).

  3. [W1] The ImageNet experiment uses a different representation and model scale than Flowers ablation. Claims do not transfer.

    We would like to clarify that the Flowers ablation is comparable to ImageNet experiment: both are trained in the same EQ-VAE latent space. They differ in dataset size, conditioning (unconditional vs. class-conditional) and model size (95.8M vs. 310M parameters). We have updated the paper to clarify this. As mentioned above, ablating on a smaller scale is standard practice in the field.

  4. [W2] Best FID requires extra autoencoder and LPIPS evaluation.

    As reported above, ****our updated result no longer uses round-trip rejection: the model reaches FID 2.86 in a single forward pass, decoded once by the EQ-VAE as in every latent-space baseline (like DiT, SiT, iMeanFlow). The improvement comes from a training-time change to matching objective (as shown in RS-IMLE[f]), and not from any inference-time filtering.

AuthorsOpenReview ↗

Rebuttal Part-2

  1. [W2] Insufficient baselines (Drifting and W-Flow)

    We have added the baselines in Table-4. The reviewer is right to point out that training setups for different generative models are vastly different. For example, Drifting uses a transformer backbone and their latent based models uses a MAE for training. Our model, on the other hand, do not use any of these. A comparison across methods with such different training strategies is difficult, and it is not the aim of this paper. Our goal in this paper is not a parameter-matched race for the best score, but to show that a minimal recipe is already competitive enough. To demonstrate that this minimal recipe is not a ceiling and our model can too benefit from recent advances, we added FD-loss to the same 310M model: it reaches FID 0.83 at 1 NFE. Other techniques, such as stronger architectures and feature encoders, could potentially be added in the same way. Our focus, however, remains on showing that a minimal recipe alone is sufficient for competitive generation.

  2. Robust Loss scale and configuration

    We use the Geman–McClure loss

    ρ(r)=r2r2+s2,s=0.5\rho(r) = \frac{r^2}{r^2 + s^2}, \qquad s = 0.5

    Latent space (ImageNet). At each stage ll, ρ\rho is applied elementwise to the residual between the stage output x~l\tilde{x}^l and the downsampled target Kl(x)K_l(x) in the EQ-VAE latent space, and averaged over elements:

    Ll=1∣Dl∣∑jρ(x~jl−Kl(x)j)\mathcal{L}^l = \frac{1}{|D_l|} \sum_{j} \rho\big(\tilde{x}^l_j - K_l(x)_j\big)

    Pixel space (CIFAR-10, CelebA-HQ). The loss combines a perceptual (LPIPS) term and a pixel term with weights 1.0 and 0.1 respectively (Appendix D):

    Ll=ρ(LPIPS(x~l,Kl(x)))+0.1⋅1∣Dl∣∑jρ(x~jl−Kl(x)j)\mathcal{L}^l = \rho\big(\mathrm{LPIPS}(\tilde{x}^l, K_l(x))\big) + 0.1 \cdot \frac{1}{|D_l|} \sum_{j} \rho\big(\tilde{x}^l_j - K_l(x)_j\big)

    The pixel term applies ρ\rho elementwise, as in the latent-space case. For the perceptual term, we apply ρ\rho to the LPIPS distance of each image pair rather than to individual feature differences. LPIPS combines feature differences using learned weights calibrated for an ℓ2\ell_2 distance, so applying ρ\rho to individual features would change what LPIPS measures.

  3. [W2] Method specifications for ImageNet experiments.

    We are happy to provide requested implementation details for ImageNet experiments. We have updated the Appendix with these additional details.

    1. Training duration: We initially trained for 2 weeks on 8 L40s GPU. The finetuning with RSIMLE was done for an additional 2 days. For the FD loss finetuning, we trained for an additional 1 day for 50K iterations.

    2. Learning rate: 2e-4 with cosine annealing

    3. Batch size: 1024

    4. Matching cost: For ImageNet, the nearest neighbour search happens within each class (condition). This is standard practice in conditional IMLE [e] literature. Nearest-neighbour search is highly optimized in modern libraries such as FAISS, which we use on GPU. In aggregate, nearest-neighbour search represents less than 1% of the total training time.

    5. Conditioning: Each class has a separate learned class embedding vector ece_c, which is added to the noise vector zz. The sum z+ecz + e_c is passed to the mapping network. We have updated Figure 8 to show this.

    6. Sampling throughput: The results were measured on a single NVIDIA L40S GPU with batch size 256, bf16 and torch.compile for the generator, timed after warm-up. Sampling is a single forward pass of the generator (1 NFE) followed by one VAE decode.

      PipelineThroughput (img/s)
      Generator only (noise, class → latent)410
      Generator + VAE decode (noise, class → 256×256 image)82

      The VAE decoder accounts for ~80% of end-to-end time, while the generator takes ~2.4 ms per image. Latent diffusion models pay the same decode cost once per image, on top of 250 sampling steps (DiT, SiT). Since we no longer rely on roundtrip rejection, we omit the cost for it for the sake of clarity.

AuthorsOpenReview ↗

Rebuttal Part-3

  1. [W1] The diffusion process is different from the multi-stage process used in the method.

    We do not claim the two processes are the same. Section 3.1 presents the connection as an analogy and states the difference directly: our construction is “similar to the diffusion construction, with each fθlf_{\theta^l} now analogous to gθtg_{\theta}^t, only indexed by resolution rather than noise level.”

    By casting diffusion models in the testing-centric view, we are able to see a unique property (multi-stage supervision), that is missing from single-step models. As the sentence states, this is our belief: “what we believe is the reason for high-quality generation in SI models”. The analogy motivates transferring the property to IMLE, and we experimentally verify that doing so helps.

We have updated the relevant sections of the paper, with all changes highlighted in blue. Your input is valuable to us, and we would be happy to address any remaining concerns if you still have any. If you feel your concerns have been resolved, we would be grateful if you would kindly consider updating your score. Thank you again for taking the time and effort to review our paper.

[a] Yue, Kaiyu, et al. "Sphere Encoder 2." arXiv preprint arXiv:2610.02208 (2026).

[b] Karras, Tero, et al. "Elucidating the design space of diffusion-based generative models." Advances in neural information processing systems 35 (2022): 26565-26577.

[c] Song, Yang, et al. "Consistency models." arXiv preprint arXiv:2303.01469 (2023).

[d] Song, Yang, et al. "Score-based generative modeling through stochastic differential equations." arXiv preprint arXiv:2011.13456 (2020).

[e] Li, Ke, et al. "Multimodal image synthesis with conditional implicit maximum likelihood estimation." International Journal of Computer Vision 128.10 (2020): 2607-2628.

[f] Vashist, Chirag, Shichong Peng, and Ke Li. "Rejection sampling imle: Designing priors for better few-shot image synthesis." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.

[g] Yang, Jiawei, et al. "Representation Fr'echet Loss for Visual Generation." arXiv preprint arXiv:2604.28190 (2026).

Reviewer_4fKnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ROMS-IMLE, a convolutional-network-based single-step generative model. The method applies direct supervision to every upsampling stage of the model and trains on the aggregated loss across stages. Based on this mechanism, the authors attempt to introduce the multi-stage supervision from multi-step generative models into a single-step generative model. The paper validates the method on CIFAR-10, CelebA-HQ, and ImageNet 256.

Critical strengths
  1. The paper is well written and easy to follow.
  2. The ablation study effectively demonstrates the contribution of each training component.
  3. The proposed method has an advantage in parameter count.
Critical weaknesses
  1. The performance advantage of the proposed method is not obvious.
  2. The paper uses an arithmetic mean for the multi-stage supervision losses. The authors do not consider that loss magnitudes may differ across resolutions, nor do they attempt a weighted average.
Questions for authors
  1. Please see weaknesses.
  2. In Table 4, do the other baselines also adopt round-trip rejection?
  3. The proposed method does not show a clear performance advantage but has an advantage in parameter count. It is suggested to present a performance comparison under comparable parameter count.
11 comments

We thank the reviewer for taking the time to review our paper. We are glad that the reviewer found our paper well-written and our proposed approach effective. We address their concerns below:

UPDATED RESULTS

Since the reviewer had doubts about the roundtrip rejection, we decided to remove it all together. It turns out our model faced the same misalignment issue identified in RSIMLE [f]: the latent codes matched during training differ in distribution from those sampled at test time. Finetuning our existing model with RSIMLE objective (ϵ\epsilon = 6.0 in EQVAE latent space) resolves this, so no inference-time filtering is needed. Our aim in this paper is to propose a minimal recipe, a simple objective and a moderately-sized convolutional network, that achieves competitive performance. Modern state-of-the-art models, by contrast, combine many sophisticated techniques, such as advanced architectures, complex feature spaces and additional losses. To show that our minimal recipe can benefit from such techniques as well, we added one of them, FD-loss [g] to our existing model, which brings our model close to state-of-the-art performance. Here are the updated results:

ModelFID
ROMS-IMLE (with RS-IMLE training)2.86
ROMS-IMLE + FD Loss0.83

We discuss these results and address the concerns of the reviewer below:

W1 stands for Weakness 1, Q1 stands for Question 1

  1. [W1, Q3] Performance advantage of the proposed method is not obvious

    It is important to not that different generative models have vastly different training procedures. Almost all modern baselines build on a transformer backbone, while our generator is a convolutional network. Other techniques like Drifting use other advancements like training in a MAE feature space. A comprehensive comparison across methods with such different training strategies is difficult, and it is not the aim of this paper. Our goal in this paper is not a parameter-matched race for the best score, but to show that a minimal recipe is already competitive enough. To demonstrate that this minimal recipe is not a ceiling and our model can too benefit from recent advances, we added FD-loss to the same 310M model: it reaches FID 0.83 at 1 NFE. Other techniques, such as stronger architectures and feature encoders, could potentially be added in the same way. Our focus, however, remains on showing that the minimal recipe alone is sufficient for competitive generation.

  2. [Q2] Do other methods adopt round-trip rejection ?

    As mentioned above, we no longer need roundtrip rejection. Other methods reported in Table-4 do not use roundtrip rejection, but they do use other inference-time techniques to tradeoff quality against diversity. These included classifier-free guidance (which increase NFE) and truncations tricks.

  3. [Q3] Performance comparison under comparable parameter count

    With our updated results for ImageNet, we are competitive with other methods, both in terms of performance and parameter count. For our CelebA-HQ model, we have vastly better scores than a similarly sized Shortcut model (130M parameters), despite using far fewer evaluation steps.

  4. [W2] Arithmetic mean for the multi-stage supervision losses.

    We thank the reviewer for this astute observation. In our proposed method, each per-stage loss is averaged over the elements at that stage's resolution, so that the LL terms are on a comparable scale (Section 3.1). In the early stages of development, we also experimented with different weights for the per-stage losses, but none performed better than uniform weighting. In keeping with our minimal design, we therefore retain the unweighted average.

We have updated the relevant sections of the paper, with all changes highlighted in blue. Your input is valuable to us, and we would be happy to address any remaining concerns if you still have any. If you feel your concerns have been resolved, we would be grateful if you would kindly consider updating your score. Thank you again for taking the time and effort to review our paper.

[f] Vashist, Chirag, Shichong Peng, and Ke Li. "Rejection sampling imle: Designing priors for better few-shot image synthesis." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.

[g] Yang, Jiawei, et al. "Representation Fr'echet Loss for Visual Generation." arXiv preprint arXiv:2604.28190 (2026).

AuthorsOpenReview ↗

Rebuttal period closing soon; Feedback appreciated.

Thank you again for your review. As the discussion period closes soon, we wanted to check whether any concerns remain. If so, please let us know so we can respond in time. If our rebuttal has resolved your concerns, we would be grateful if you would consider raising your score.

Thank you for your responses. The responses have addressed many of my concerns. However, I believe the performance improvement of the proposed method in this paper is still relatively limited. For instance, in the updated results, iMeanFlow-M achieves a better FID (2.27 vs 2.86) with fewer parameters (174M vs 310M). Although the proposed method can also be combined with techniques like FD-Loss, the core of its performance improvement comes from FD-Loss, which does not change the assessment of the proposed method itself. It is indeed encouraging that the paper proposes good generation using a convolutional network generator, but it seems insufficient to reach the level of "a minimalist approach to competitive single-step generative modeling". Therefore, I will maintain my rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.