- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes an IMLE training recipe combining supervision at intermediate decoder resolutions with a Geman–McClure robust loss for matched real/generated pairs. It uses ConvNeXt generators for pixel-space and latent-space generation. Experiments cover CIFAR-10, CelebA-HQ, and ImageNet; on ImageNet, round-trip rejection improves the reported FID from 4.16 to 2.56.
Critical strengths
-
The Flowers ablation provides clear evidence of improvement within the tested setting. With the same backbone and training iterations, multi-stage supervision improves FID from 62.61 to 31.24, and adding the robust loss further improves it to 13.90, with three-seed statistics (Table 1). The attention-based variant also shows the same qualitative trend (Appendix F).
-
Demonstrating useful image generation with a non-adversarial, single-pass IMLE generator is valuable. The evaluation spans pixel and latent spaces, and Table 4 explicitly reports ImageNet results both before and after rejection, allowing readers to distinguish the underlying generator from the filtered sampling procedure.
Critical weaknesses
-
The contribution of the proposed training changes is insufficiently established. CHIMLE (Peng et al., NeurIPS 2022, Sections 2.2–2.3) already introduces intermediate output heads, downsampled targets, and supervision at multiple generator resolutions within IMLE. Its image-conditioned setting and hierarchical latent search differ from ROMS-IMLE, but this directly relevant precedent is not discussed, leaving the novelty of multi-stage supervision unclear. Empirically, the component ablations are confined to Flowers. The ImageNet experiment uses a different latent-space setup with EQ-VAE, without controlled evidence isolating the training changes there. Thus the substantial Flowers gains do not yet establish the contribution of these changes at the scale underlying the main performance claim. CHIMLE
-
The method specification and resource comparison leave the practical result difficult to assess. The Geman–McClure formula/scale and its application within the pixel-space LPIPS/pixel-loss combination are unspecified. Algorithm 1 uses one distance for matching and regression, whereas Appendix D describes LPIPS/L2 matching and a different regression loss; the implemented objective needs to be stated explicitly. Main-benchmark training budgets and measured matching costs are also missing. For ImageNet, the headline FID additionally uses rejection, whose encoder/decoder/LPIPS costs are not captured by generator NFE. Finally, the cited iMeanFlow reports a 174M-parameter, 1-NFE model with FID 2.27, while Table 4 includes only its 610M model. This smaller operating point is relevant to the claimed parameter efficiency of the 310M ROMS model. These reported results use different training setups, so they motivate a clearer comparison rather than establish a matched-budget ranking. iMeanFlow, Table 2
Questions for authors
-
What is the substantive distinction from CHIMLE's intermediate supervision? Can you provide a representative ImageNet ablation, holding the backbone, tokenizer, and training budget fixed, to establish the gains from multi-stage supervision and the robust loss before rejection?
-
What are the exact matching and regression objectives, including the robust-loss scale and pixel-space combination? Please also specify the ImageNet conditioning/matching procedure, learning rate, batch size, and training duration.
-
What are the measured training cost, including nearest-neighbour search, and sampling throughput before/after rejection? Please specify the rejection threshold and its selection, and contextualize the cost/quality trade-off against the smaller iMeanFlow models already available in the cited work.
7 comments
Rebuttal Part-2
-
[W2] Comparison to other baselines like smaller iMeanFlow
We thank the reviewer for pointing to iMeanFlow’s smaller models, which we have added to Table 4. It is important to not that different generative models have vastly different training procedures. In this instance, iMeanFlow builds on a transformer backbone, while our generator is a convolutional network. Other techniques like Drifting use other advancements like training in a MAE feature space. As the reviewer notes, a comprehensive parameter-matched comparison across methods with such different training strategies is difficult, and it is not the aim of this paper. Our goal in this paper is not a parameter-matched race for the best score, but to show that a minimal recipe is already competitive. To demonstrate that this minimal recipe is not a ceiling and our model can too benefit from recent advances, we added FD-loss (Yang et al., 2026), to the same 310M model: it reaches FID 0.83 at 1 NFE. Other techniques, such as stronger architectures and feature encoders, could potentially be added in the same way. Our focus, however, remains on showing that a minimal recipe alone is sufficient for competitive generation.
-
[Q2, W2] Robust Loss scale and configuration
We use the Geman–McClure loss:
Latent space (ImageNet). At each stage , is applied elementwise to the residual between the stage output and the downsampled target in the EQ-VAE latent space, and averaged over elements:
Pixel space (CIFAR-10, CelebA-HQ). The loss combines a perceptual (LPIPS) term and a pixel term with weights 1.0 and 0.1 respectively (Appendix D):
The pixel term applies elementwise, as in the latent-space case. For the perceptual term, we apply to the LPIPS distance of each image pair rather than to individual feature differences. LPIPS combines feature differences using learned weights calibrated for an distance, so applying to individual features would change what LPIPS measures.
-
[Q3, W2] Method specifications for ImageNet experiments.
We are happy to provide requested implementation details for ImageNet experiments. We have updated the Appendix with these additional details.
-
Training duration: We initially trained for 2 weeks on 8 L40s GPU. The finetuning with RSIMLE was done for an additional 2 days. For the FD loss finetuning, we trained for an additional 1 day for 50K iterations.
-
Learning rate: 2e-4 with cosine annealing
-
Batch size: 1024
-
Matching cost: For ImageNet, the nearest neighbour search happens within each class (condition). This is standard practice in conditional IMLE [e] literature. Nearest-neighbour search is highly optimized in modern libraries such as FAISS, which we use on GPU. In aggregate, nearest-neighbour search represents less than 1% of the total training time.
-
Conditioning: Each class has a separate learned class embedding vector , which is added to the noise vector . The sum is passed to the mapping network. We have updated Figure 8 to show this.
-
Sampling throughput: The results were measured on a single NVIDIA L40S GPU with batch size 256, bf16 and
torch.compilefor the generator, timed after warm-up. Sampling is a single forward pass of the generator (1 NFE) followed by one VAE decode. The VAE decode runs in fp32, the same setting used for the reported FID.Pipeline Throughput (img/s) Generator only (noise, class → latent) 410 Generator + VAE decode (noise, class → 256×256 image) 82 The VAE decoder accounts for ~80% of end-to-end time, while the generator takes ~2.4 ms per image. Latent diffusion models pay the same decode cost once per image, on top of 250 sampling steps (DiT, SiT). Since we no longer rely on roundtrip rejection, we omit the cost for it for the sake of clarity.
-
Rebuttal Part-3
-
[W2] Algorithm 1 uses one symbol d for matching and regression, but Appendix D describes different ones.
The matching phase depends only on the relative ranking of samples, not on the distance values themselves. As long as matching phase determines a sensible ranking between the generated samples, the IMLE procedure works fine. Hence, the matching distance need not be the same as the training loss; prior IMLE work also use different functions for the two phases(AdaIMLE [h], RS-IMLE [f]). We have updated Algorithm 1 to make this explicit: it now uses for nearest-neighbour matching and for the per-stage loss.
We have updated the relevant sections of the paper, with all changes highlighted in blue. Your input is valuable to us, and we would be happy to address any remaining concerns if you still have any. If you feel your concerns have been resolved, we would be grateful if you would kindly consider updating your score. Thank you again for taking the time and effort to review our paper.
[a] Yue, Kaiyu, et al. "Sphere Encoder 2." arXiv preprint arXiv:2610.02208 (2026).
[b] Karras, Tero, et al. "Elucidating the design space of diffusion-based generative models." Advances in neural information processing systems 35 (2022): 26565-26577.
[c] Song, Yang, et al. "Consistency models." arXiv preprint arXiv:2303.01469 (2023).
[d] Song, Yang, et al. "Score-based generative modeling through stochastic differential equations." arXiv preprint arXiv:2011.13456 (2020).
[e] Li, Ke, et al. "Multimodal image synthesis with conditional implicit maximum likelihood estimation." International Journal of Computer Vision 128.10 (2020): 2607-2628.
[f] Vashist, Chirag, Shichong Peng, and Ke Li. "Rejection sampling imle: Designing priors for better few-shot image synthesis." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.
[g] Yang, Jiawei, et al. "Representation Fr'echet Loss for Visual Generation." arXiv preprint arXiv:2604.28190 (2026).
[h] Aghabozorgi, Mehran, Shichong Peng, and Ke Li. "Adaptive IMLE for few-shot pretraining-free generative modelling." (2023).
Rebuttal Part-1
We thank the reviewer for their feedback. We are happy that the reviewer found our paper useful, valuable and clear.
UPDATED RESULTS
Since the reviewer had doubts about the roundtrip rejection, we decided to remove it all together. It turns out our model faced the same misalignment issue identified in RSIMLE [f]: the latent codes matched during training differ in distribution from those sampled at test time. Finetuning our existing model with RSIMLE objective (ϵ = 6.0 in EQVAE latent space) resolves this, so no inference-time filtering is needed. Our aim in this paper is to propose a minimal recipe, a simple objective and a convolutional network, that achieves competitive performance. Modern state-of-the-art models, by contrast, combine many sophisticated techniques, such as advanced architectures, complex feature spaces and additional losses. To show that our minimal recipe can benefit from such techniques as well, we added one of them, FD-loss [g] to our existing model, which brings our model close to state-of-the-art performance. Here are the updated results:
We discuss these results and address the concerns of the reviewer below:
W1 stands for Weakness 1, Q1 stands for Question 1
[W1,Q1] Comparision with CHIMLE
We thank the reviewer for bringing this up and we updated the main paper to cite CHIMLE. As the reviewer pointed out, CHIMLE does have some overlap with our contributions. However, there are key differences which separates our work from CHIMLE:
[W1, Q1] ImageNet ablations
We trained our model from scratch for 50K iterations, using the same training budget and backbone. Here are the results, evaluated over 50K generated samples:
Even early in training (50K iterations), ROMS-IMLE already improves FID, precision and recall over vanilla IMLE. In general, it is established practice to conduct ablation studies and hyperparameter tuning on smaller datasets, and finally transferring the best configuration to a large scale. Ablating on a large dataset like ImageNet is very computationally expensive, and repeating it per component till convergence is beyond our academic budget. This is the reason why we choose to do ablation over smaller datasets, which is consistent with the rest of the field (Sphere Encoder 2[a], EDM[b], Consistency Models[c], Score SDE[d]).
[W1] The ImageNet experiment uses a different latent-space setup than Flowers ablation. Claims do not transfer.
We would like to clarify that the Flowers ablation is comparable to ImageNet experiment: both are trained in the same EQ-VAE latent space. They differ in dataset size, conditioning (unconditional vs. class-conditional) and model size (95.8M vs. 310M parameters). We have updated the paper to clarify this. As mentioned above, ablating on a smaller scale is standard practice in the field.