acceptodds
Under review as a conference paper at ICLR 2027

Representation Distribution Matching for One-Step Visual Generation

Abstract

We elucidate the design space of Representation Distribution Matching (RDM), our name for the paradigm that trains a one-step image generator by matching generated and reference feature distributions under frozen pretrained encoders. We identify two design axes, how the distributions are compared and the representations they are compared in, and controlled studies along them yield three findings. First, the classical MMD, which could not train convincing generators a decade ago, becomes a strong and scalable objective once estimated right. Second, the generated batch is then the operative variable, with an optimum above 2048, far beyond customary batch sizes. Third, any single representation can be gamed, driven below the real score while images stay visibly fake, so we match against a balanced battery of encoders and evaluate with SWr14, a Sliced-Wasserstein distance over 14 encoders that is independent of the training loss and resists gaming. Combining the preferred choices yields improved RDM (iRDM): it sets the one-step state of the art on ImageNet at SWr14 1.30, corroborated by PickScore, a human-preference proxy our objective never optimizes, which prefers it over the prior best one-step generator on 71.2% of matched samples. The same recipe post-trains the four-step FLUX.2 [klein] into a one-step generator, surpassing the four-step version on GenEval, 0.826 to 0.794, and on PickScore, 22.76 to 22.58, in 90 H200 GPU-hours. Code will be released.

Primary area:
generative models
Keywords:
Image Generation, Few-step Generation, Distribution Matching

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_mBiSOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies representation distribution matching for one-step image generation along two axes: the distributional discrepancy and the feature representations. Its proposed recipe, iRDM, combines exact within-batch kernel repulsion with a frozen Nyström approximation of the reference attraction, large generated batches enabled by gradient caching, and adaptive weighting across ten frozen encoders. For text-to-image post-training, it additionally matches joint image–text feature distributions. The paper reports improvements over its ImageNet initialization on a fourteen-encoder Sliced-Wasserstein metric, including a four-encoder held-out subset, and on PickScore. It also reports one-step FLUX.2 results exceeding the four-step starting model on GenEval and PickScore. The main strengths are the practical distribution-matching recipe and its ablations; the main concerns concern evaluation independence, benchmark-conditioned reference curation, and the attribution of the reported gains.

Critical strengths
  1. Practical distribution-matching design: Separating the stationary reference attraction from the moving generated-sample repulsion is a useful design. The frozen reference removes per-step reference-resampling noise, while gradient caching enables investigation of substantially larger generation batches (§§3.1–3.2).

  2. Useful empirical decomposition: The batch-size, discrepancy, and encoder-weighting experiments provide actionable evidence about the recipe. In particular, the single-encoder experiment illustrates why matching one representation need not eliminate visible artifacts (§3.3; Appendices D–F).

  3. Evidence beyond the directly optimized encoders: Table 1 reports an improvement on the four held-out encoders, from 1.98 to 1.54 relative to FD-SIM. The ImageNet PickScore comparison provides additional supporting evidence. These results should be acknowledged, although they do not fully establish perceptual superiority or resistance to metric gaming.

Critical weaknesses
  1. Evaluation independence remains insufficiently established: Ten of the fourteen encoders in the primary metric also define the training objective. Switching from MMD to Sliced-Wasserstein changes the discrepancy but retains the same representations and their potential blind spots. The held-out improvement is encouraging; however, the training and held-out panels contain closely related model families (e.g., DINOv3 vs. DINOv2, SigLIP2 vs. SigLIP). This raises a legitimate question about evaluator redundancy rather than proving that the held-out results are invalid. As extensively discussed in prior literature on feature-space evaluation pitfalls (e.g., Stein et al., Exposing Flaws of Generative Model Evaluation Metrics and Their Unfair Treatment of Diffusion Models, NeurIPS 2023; Morozov et al., ICLR 2021), generative evaluation relying on correlated representations risks inheriting shared blind spots. Representation-similarity analysis, family-level holdouts, and blinded human evaluation would significantly strengthen the anti-gaming claim.

  2. The text-to-image evaluation is confounded by benchmark-conditioned curation: Appendix H.1 constructs part of the reference from the 553 GenEval evaluation prompts, filters generations using the benchmark detector, and includes these prompts in the training conditioning pool. Consequently, the reported GenEval improvement does not establish generalization to unseen compositional prompts. Moreover, PickScore ranks candidates for the perception reference, making the subsequent T2I PickScore evaluation dependent on a scorer already used in the training-data pipeline. The absence of a direct reward loss does not remove this dependence. A benchmark-disjoint reference and evaluation on unseen prompts are necessary to support the broader claims.

  3. Method, data-curation, and compute effects are not adequately separated: The T2I experiment changes both the sampling procedure and the target distribution, using selected teacher outputs rather than the teacher’s unfiltered distribution. The DMD2 baseline uses a different prompt pool and reaches its reported peak in approximately 10 H200 GPU-hours, versus 90 reported for iRDM (§4.2; Appendix H.2). These comparisons do not isolate the contribution of the matching objective. The paper should report end-to-end reference-construction costs and add controlled comparisons using matched prompt distributions, reference access where applicable, and compute budgets.

  4. Some methodological claims exceed the supporting evidence: The controller ablation shows a modest aggregate improvement and a larger worst-encoder improvement, but does not demonstrate prevention of perceptual shortcut learning (Appendix E). Likewise, the evidence supports an effective post-training recipe, not yet a competitive from-scratch generator. The paper should calibrate these claims and better contextualize its multi-encoder supervision relative to Vision-Aided GAN (Kumari et al., CVPR 2022), which already uses frozen pretrained feature ensembles with trainable adversarial heads and progressive model selection. The relevant distinction is direct kernel matching and adaptive weighting, rather than multi-encoder supervision itself.

Questions for authors
  1. Evaluator independence: Given the already reported held-out improvement, can you provide representation-redundancy analysis (e.g., CKA), family-level holdout experiments, or blinded human comparisons? Were held-out metrics used for hyperparameter tuning or checkpoint selection?
  2. Benchmark-disjoint generalization: How much of the GenEval improvement remains when the reference and conditioning pool exclude the evaluation prompts? Can you separately ablate GenEval-detector filtering and PickScore-based selection, and evaluate on unseen compositional prompts?
  3. Fair comparison and full cost: Does the reported 90 GPU-hour budget include teacher sampling, filtering, encoder feature extraction, and Nyström construction? How does iRDM compare with DMD2 under controlled data and compute settings?
  4. Metric interpretation: Equation (5) makes the selected validation baseline equal to one by normalization, but does not establish one as a universal lower bound. Can the authors clarify the “real floor” terminology and report uncertainty across evaluation samples, projections, and training seeds?
Reviewer_4DFNOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper compiles methods for training one-step generators through distribution matching in the feature space of frozen pre-trained encoders, and extracts two design dimensions: the comparison method and the representation space. Three findings are obtained under controlled experiments, and by combining the preferred options, a new scheme is derived. Experimental results demonstrate the effectiveness of the proposed scheme.

Critical strengths
  1. The research question is important, and the perspective is insightful; the organized design dimensions provide valuable guidance for future work.

  2. The experiments are thorough and yield highly competitive results, and the interpretation of the results is objective and accurate.

Critical weaknesses
  1. The generalizability of the proposed iRDM is unclear. Are the preferred options for each component generally applicable across scenarios? For example, a batch size of 2048 is only proven effective for post-training; if the model needs to be trained from scratch, would this still be the optimal choice? It is recommended to clarify which choices hold generally for RDM, and which are optimal choices specific to the current model.

  2. The evaluation metrics in the experiments could be supplemented. The compared methods all have publicly available FID scores, so a comparison should be conducted. Although FID as a metric may have its issues in this scenario, given that it remains the mainstream metric at present, reporting the corresponding results would better support the effectiveness of the proposed method, or, combined with other metrics, reveal the flaws of FID. There is no reason to completely ignore this metric.

Questions for authors

Apart from whether the issues mentioned in the weaknesses can be addressed, there are no further questions.

Reviewer_mtSDOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper unifies recent teacher-free one-step generators under a framework that matches the real data distribution in frozen-encoder feature spaces. It studies two design choices: how to compare distributions and which representation spaces to use. The proposed method, iRDM, uses MMD with a frozen Nystrom reference and ten encoders balanced by a Lagrangian controller. It reports SW_r14 of 1.30 on one-step ImageNet generation, versus the previous best of 2.05, and reduces FLUX.2 [klein] from four steps to one, improving GenEval from 0.794 to 0.826.

Critical strengths

The paper is a systematic study of two design axes. It explains the limitations of prior methods through specific design choices: drifting rebuilds its reference every batch, while FD-loss reduces the distribution to two moments. The separation between comparison methods and representation spaces makes these explanations testable.

The experiments are also thoughtful, including a synthetic estimator diagnostic, a batch-size sweep at equal wall-clock time, and a six-way comparison of training distances under two neutral evaluation distances (App. F, Table 6). The equal-time comparison is particularly useful and deserves wider adoption.

The gains are not limited to the authors' own metric. PickScore is never optimized, yet prefers iRDM over the previous best one-step model 71.2% of the time. The held-out-encoder score also improves, providing additional evidence beyond the training setup.

Critical weaknesses

My main concern is how robust the headline results are. SW_r14 depends on the authors' choices of encoder panel and normalization, but the paper does not test how sensitive the ranking is to these choices. The FLUX results also deserve a cleaner check, since the reference set is filtered using GenEval and PickScore is used both to rank the reference and to evaluate the result. These choices do not invalidate the gains, but make it difficult to judge how much they contribute. An encoder-panel sensitivity study and an unfiltered-reference ablation would help address these concerns.

Questions for authors

Can you provide a sensitivity analysis of the SW_r14 ranking across different encoder panels or subset study? Reporting FID alongside would also give readers an independent reference metric - nice to have.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response: evaluation, FID, scope and text-to-image

We thank the reviewers. Four questions recur across the reviews; short answers first, then the details.

  1. Is iRDM's ImageNet lead an artefact of evaluating with its own training encoders? No. PickScore, HPSv2.1, ImageReward and a blinded GPT-6-Astra comparison, none used in our ImageNet training, prefer iRDM to its initialization, and iRDM stays first on SW₍r14₎ when any encoder or encoder family is left out.
  2. Where is FID? The revised paper will report it for every generator. iRDM is 17th of 18 on FID and 2nd of 15 on FD_DINOv2, ahead of every multi-step model. FID is a poor optimisation target here: it can be pushed below real data with no gain in quality.
  3. Does the batch optimum above 2048 hold outside post-training? No. It is a post-training result, and the revised paper will say so. A from-scratch toy sweep shows the estimator itself works at every batch size from 16 to 1024.
  4. How much of the text-to-image gain comes from curating the reference? Each curation step adds 2 to 3 GenEval points; curation is part of RDM's result and of its cost. The released model's lead holds on 960 new prompts absent from GenEval, and HPSv2.1 and ImageReward, which our pipeline never uses, confirm the PickScore gain on COCO prompts.

Details

1. Evaluation (mtSD-W1, Q1; mBiS-W1, Q1, Q4)

  • iRDM is first in all 14 leave-one-encoder-out and all 9 leave-one-family-out panels.
  • PickScore, HPSv2.1 and ImageReward prefer iRDM to its initialization FD-SIM in 71.2%, 71.7% and 56.9% of 10,000 matched pairs; GPT-6-Astra, in a blinded two-alternative forced choice, in 67.2% of 400 judgments.
  • On five encoders from families absent from training, iRDM is closer to real data on I-JEPA and SAM, and FD-SIM on three SSL encoders that track Inception statistics. A controlled ablation traces this split to the gated controller.
  • Held-out metrics were never used for tuning or checkpoint selection.

2. FID (4DFN-W2; mtSD-Q2)

  • The two FD-trained models score below real training images, and the lowest-FID model is at chance or below against its own base under all three preference models.
  • iRDM has a worse FID than all three pMF-H models, yet all three preference models prefer it to each of them.
  • Our controller drops FID's Inception-v3 encoder from training once it reaches the real level.

3. Scope (4DFN-W1; mBiS-W4)

  • Trained from scratch on the toy task of Fig. 3, iRDM's estimator stays within 0.008 of the real level at every batch size from 16 to 1024.
  • A new appendix will state the scope of each design choice.

4. Text-to-image (mtSD-W2; mBiS-W2, W3, Q2, Q3)

  • Re-trained without each step: PickScore ranking adds 2.3 GenEval points, the detector-verified GenEval renders 2.1 and detector filtering 2.8.
  • On the 960 new prompts the released model beats the four-step model, 0.780 vs 0.764; a VQAScore from Qwen2.5-VL-7B agrees (+2.2 of 100).
  • HPSv2.1 and ImageReward prefer the one-step model on 59.8% and 62.1% of COCO-val images.
  • One of two re-runs reproduces the released checkpoint; the abstract will state the run mean (0.802) next to 0.826.
  • DMD2 trained on our exact prompts peaks at 0.774 GenEval, below the four-step model and below RDM.
  • Training takes 90 H200 GPU-hours; building the reference is a one-time cost of about 96 H100 GPU-hours.

Planned revisions. The revised paper will describe SW₍r14₎ as a different discrepancy checked on held-out and family-disjoint encoders, three preference models and GPT-6-Astra; report FID and FD_DINOv2 for every generator; scope the batch claim to post-training; rename "real floor" to "real-data reference level"; add the new text-to-image results and the full cost; and add Vision-Aided GAN, Stein et al. and Morozov et al. to related work.


From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.