acceptodds
Under review as a conference paper at ICLR 2027

Embedding Prediction Helps Image Generation

Abstract

In diffusion transformers, a class label or a text prompt is embedded once, and the same condition is reused at every denoising step. We ask whether predicted embeddings can serve as this condition instead. Next-Embedding Predictive Autoregression (NEPA) trains a Transformer to predict the next continuous embedding in a sequence. In generation, the clean image follows the noisy image, so its embeddings are the next embeddings after the condition and the noisy image. We train a NEPA model to predict them all at once with Multi-Embedding Prediction, and in Embedding Conditioned Generation, a DiT generator is conditioned on these predictions, recomputed at every denoising step, so the conditioning signal adapts to the current noisy state. Experiments on class-conditional ImageNet 256 × 256 study the condition of the generator, the design of Multi-Embedding Prediction, and the scaling of both models. The NEPA model adds a second network to every sampling step; with it, and combined with REPA, our final model, NEPA-DiT-XL, reaches an FID of 1.32 using about a third of the training compute of REPA.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
representation for generation, image generation, diffusion models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Gb1aOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces NEPA-based Embedding Conditioned Generation, where a model predicts clean-image embeddings from the current noisy image and uses them as dynamic conditions for DiT at each denoising step. On ImageNet 256×256, the method improves generation quality and reaches an FID of 1.32 when combined with REPA.

Critical strengths
  1. Clear methodological novelty. The paper introduces a simple but effective idea of replacing fixed diffusion conditioning with embeddings dynamically predicted from the current noisy image.
  2. Strong and well-controlled ablations. The experiments compare different prediction losses, conditioning strategies, model scales, and query frequencies, helping isolate where the gains come from.
  3. Competitive efficiency and performance. The final NEPA-DiT-XL + REPA model achieves an FID of 1.32 on ImageNet 256×256 while using substantially less training compute than strong REPA-based baselines.
Critical weaknesses
  1. Limited analysis of the predicted representations. While the paper shows that NEPA embeddings improve FID, it provides relatively little analysis of what information these predicted embeddings encode or why they are particularly effective for generation.
  2. Unclear necessity of the predictive embedding. Since the predicted embedding is a deterministic function of the class condition and the current noisy image, it does not introduce additional information. It remains unclear why an explicitly pretrained predictive representation is fundamentally preferable to letting the generator learn the same transformation internally.
  3. Unclear semantics of the clean-image embedding. The clean-image target is produced by NEPA’s own jointly learned embedding layer rather than by a fixed semantic encoder. It is therefore unclear what information this embedding is expected to represent and whether it captures meaningful clean-image structure or simply a latent space that co-adapts to the prediction objective.
Questions for authors

see above

3 comments
AuthorsOpenReview ↗

Reply to Reviewer Gb1a

We thank the reviewer for recognizing the novelty of the idea, the well-controlled ablations and the efficiency of the final model.

W1 and W3: What do the predicted embeddings and their jointly learned clean-image targets encode, and why do they work as a condition for generation?

The predictions form a rich, spatially organized representation of the clean image, and the generator reads this representation as its condition. We already describe the target space in Section 3.3. The targets are the predictor's own embedding of the clean image, and the contrastive loss keeps them distinct from one another, so they cannot collapse into a space that only fits the objective. The predictions estimate these targets from the class and the noisy image.

We add Appendix E to the revised PDF to show what the predictions encode. Figure 8 shows that the predictions recover the layout of the clean image before the noisy image shows it: under one PCA basis shared by all images and noise levels, they separate objects from backgrounds while the decoded noisy latent is still unrecognizable, and the objects of different images tend to take similar colors. Figure 9 shows the same along sampling trajectories. Figure 10 shows that the predictions of one region resemble each other: the similarity between one prediction and the targets of its image spreads over the region of the query at high noise and narrows to the query's own position as the noise decreases.

This representation is what makes the predictions a good condition. With the generator and its training recipe fixed, the predictions give 25.04 FID against 36.39 for the class embedding (Table 7).

In the revised version, we will describe the target space more prominently in Section 3.3.

W2: The prediction adds no information beyond the class and the noisy image, so why is an explicit predictor better than letting the generator learn the same mapping?

The predictor adds no new information, but it splits generation into two easier steps: first predict a good condition in embedding space, then generate the image from it. We already discuss this in Sections 4.1 and 4.2.

In the first step, MEP trains the predictor to predict the clean-image embeddings from the class and the noisy image. Its InfoNCE loss makes each predicted embedding distinguishable from the other embeddings of the same image, so the predictions keep the fine-grained detail of the clean image. The generator then receives a condition that already describes the clean image region by region, and its own job reduces to generating an image that matches it. When the same network is instead trained end to end with the generator, it receives only the generator's regression gradient, nothing pushes its outputs to be distinguishable, and the generator has to learn both steps at once from one loss. With the same architecture and FLOPs, the two-step design gives 25.04 FID against 29.52 for end-to-end training (Table 7).

In the revised version, we will make this two-step view more prominent in Sections 4.1 and 4.2, so that readers can follow why the explicit predictor helps.

Dear Reviewer,

Thank you again for your time. We would like to check whether our response has addressed the concerns raised in your review. Could you kindly let us know if any questions remain, so that we have enough time to address them? We would be happy to provide any further clarification, and we look forward to your feedback.

Best regards,

The Authors

concern solved. keep my positive score

Reviewer_q4qbOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Embedding Conditioned Generation (ECG), which uses clean-image embeddings predicted from the class label and the current noisy image to condition a diffusion Transformer. The predictor is trained with Multi-Embedding Prediction (MEP) to predict all clean-image patch embeddings at once using an InfoNCE loss. It is then frozen and used at each denoising step, passing its predictions to the generator through MM-DiT blocks. Experiments test design choices and model sizes, and show that the final model with REPA reaches image quality close to that of leading methods using less training compute than the REPA baseline.

Critical strengths
  1. The method has a clear two-stage structure: pretrain an embedding predictor with MEP, then freeze it and use its predictions to condition a diffusion generator.
  2. The method details are clearly described. This makes the technical design easy to follow.
Critical weaknesses
  1. The motivation is unclear. The introduction proposes updating the condition during denoising, but gives little explanation of the problems caused by fixed conditioning or the benefits expected from NEPA/MEP.

  2. Table 9 does not clearly show NEPA's contribution to the final results or its advantages over existing methods.

    1. The final generator also uses a REPA loss, but the table does not show results without REPA using the same model and settings. Earlier experiments test NEPA-DiT without REPA, but this comparison is missing for the final model. The separate effects of NEPA and REPA on the final results are therefore unclear.
    2. The appendix counts the NEPA predictor's parameters and pretraining cost. However, Table 9 does not show a comparison at the same total model size and training compute, which would better show the method's advantage.
    3. UDT+-XL/2 + REPA produces images of similar quality after the same reported total number of training epochs. The same number of epochs does not mean the same training compute, but the paper still needs to show a clear benefit over this baseline.
  3. The paper does not give a clear reason for using a two-stage design for image generation. MEP gives DiT NEPA-predicted embeddings as another conditioning input. Table 7 shows improvements, but the paper gives little explanation of the reasons for these improvements. The need to train the predictor separately and then freeze it is also unclear.

Questions for authors

See Critical Weaknesses.

7 comments
AuthorsOpenReview ↗

Reply to Reviewer q4qb

We thank the reviewer for the constructive review, and for finding the two-stage structure clear and the method details easy to follow.

W1: What problems does a fixed condition cause, and what benefits are expected from NEPA and MEP?

A fixed condition tells the generator what to draw but not what that means for the current image, and NEPA supplies this at every step by predicting the clean image embedding from the current noisy one. We already discuss this in the first four paragraphs of the introduction: a class label or text prompt is embedded once and reused at every denoising step, so the generator itself must work out what the condition implies for the current image.

NEPA is expected to fix this by computing the condition from the current noisy image, with MEP predicting all clean-image embeddings at once. Table 7 (Section 4.2) measures these benefits with the generator and its training recipe fixed:

ConditionFID
Class embedding36.39
Class token + x_t37.21
Predictor, trained jointly with generator29.52
Predictor, flow-matching pretrained, frozen30.86
Predictor, MEP pretrained, frozen (ours)25.04

Feeding the noisy embeddings of x_t themselves does not help (37.21 against 36.39), so the gain comes from the prediction rather than from extra access to x_t. MEP pretraining, with the predictor then frozen, is what makes this prediction a good condition: the same network gives 25.04 when trained with MEP, against 29.52 when trained jointly with the generator and 30.86 when pretrained with flow matching.

In the revised version, we will make this explanation more prominent in the first paragraph of the introduction and cite Table 7 there, so that readers can follow the motivation more easily.

W2: Does Table 9 show NEPA's contribution to the final results and its advantage over existing methods?

W2.1 What are the separate effects of NEPA and REPA in the final model?

ECG and REPA act on different parts of the generator, so they do not interfere and their gains add up. ECG sets the condition that the generator reads, and REPA aligns its features. At the same 400K iterations, adding ECG to REPA lowers FID from 7.97 to 7.59 without guidance and from 2.14 to 1.32 with guidance, and adding REPA to ECG lowers it from 8.59 to 7.59 and from 2.60 to 1.32.

ModelGuidanceFID
NEPA-DiT-XLnone8.59
REPA (our reproduction)none7.97
NEPA-DiT-XL + REPAnone7.59
NEPA-DiT-XLours2.60
REPA (our reproduction)REPA's best2.14
NEPA-DiT-XL + REPAours1.32

Each model uses its own best guidance, and our reproduction matches the 7.9 that REPA reports at 400K.

W2.2 How does it compare at the same total model size and training compute?

At the same total model size and training compute, MEP gives a better condition than other ways of training the same network, and our full model needs about a third of the training compute of SiT-XL/2 with REPA. Table 7 (shown in W1) gives the matched comparison. Its last three rows give the generator the same extra network as ours, with the same parameters and FLOPs per step, and differ only in how that network is trained. Pretrained with flow matching for the same 240 epochs as MEP and then frozen, this network gives 30.86 FID, against 25.04 with MEP, so the gain comes from the objective rather than from the added size or compute. At the system level, NEPA-DiT-XL + REPA is trained with 3.1e20 FLOPs in total, predictor included, and reaches 1.32, while SiT-XL/2 with REPA uses 1.0e21 FLOPs over 800 epochs and reaches 1.42 (Table 9). The two generators have nearly the same size (683M and 675M parameters, Table 13), and our total of 1.39B includes the shared 711M predictor.

W2.3 What is the benefit over UDT+, which reaches similar quality after the same total epochs?

UDT+ changes the generator architecture and ECG changes the condition that the generator reads, so the two are orthogonal. UDT+-XL/2 + REPA reports 1.38 after 320 epochs, and NEPA-DiT-XL + REPA reaches 1.32 after 240 epochs of the predictor and 80 of the generator. The predictor is trained once and reused, as a pretrained VAE or vision encoder is: one NEPA-XL model conditions the DiT-B, DiT-L and DiT-XL generators of Table 8 without retraining. Since it only replaces the condition input, we believe it can be combined with UDT+ in the same way.

In the revised version, we will add the results without REPA and add more discussion of training cost.

W3: Why a two-stage design, where do the gains in Table 7 come from, and why freeze the predictor?

Training the predictor separately gives a better condition, and freezing it lets one predictor serve many generators. The paper supports both reasons in Sections 3.4, 4.2 and 4.3.

The gains in Table 7 come from training the predictor separately: with identical architecture and FLOPs, MEP followed by freezing gives 25.04 FID, against 29.52 for joint training with the generator. Joint training passes only the generator's regression gradient to the predictor, which does not make its predicted embeddings distinguishable. MEP trains each predicted embedding to pick out its own clean embedding among those of the same image, so the predictions keep the fine-grained detail that helps generation.

Freezing then lets the predictor be trained once and reused, as a pretrained VAE or vision encoder is: one NEPA-XL model conditions the DiT-B, DiT-L and DiT-XL generators of Table 8 without retraining, and any new generator can use it in the same way. Its condition prefix also does not depend on the image, so its key-value cache is computed once per sample.

In the revised version, we will make these two reasons more prominent at the start of Section 3, so that readers can follow the design more easily.

Dear Reviewer,

Thank you again for your time. We would like to check whether our response has addressed the concerns raised in your review. Could you kindly let us know if any questions remain, so that we have enough time to address them? We would be happy to provide any further clarification, and we look forward to your feedback.

Best regards,

The Authors

Reviewer_HH7aOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper conditions a diffusion transformer on embeddings predicted by a second network rather than on a fixed class embedding. This second network is a NEPA-style Transformer trained from scratch with Multi-Embedding Prediction. Given a class token, a begin-of-image token and the patch embeddings of a noisy latent, it predicts the patch embeddings of the clean latent in one forward pass, using an InfoNCE loss with the other patches of the same image as negatives. The targets come from its own jointly trained patch embedding layer, and the noise level is sampled uniformly so that one model covers all timesteps. The network is then frozen, and an MM-DiT style generator is trained with flow matching while seeing the class only through these predictions, which are recomputed from the current noisy input at every sampling step.

Experiments are on class-conditional ImageNet 256×256 with SD-VAE latents. Ablations at the B scale cover the loss, stop-gradient, target normalization, timestep input, the negative bank and the patch size. A controlled comparison (Table 7) keeps the extra network fixed and changes only how it is trained: MEP followed by freezing gives FID 25.04, against 29.52 for joint end-to-end training and 30.86 for flow-matching pretraining, while the plain class-embedding baseline is at 36.39. FID drops when either network is scaled from B to XL. Combined with REPA, the final NEPA-DiT-XL reaches FID 1.32 with a 250-step SDE sampler and interval guidance (1.57 with a 96-step ODE sampler), at roughly a third of the training compute of SiT-XL/2 with REPA. The predictor adds about 92 GFLOPs per sampling step and has to be run at every step. Reusing its first prediction for all steps gives FID 242.63.

Critical strengths
  • The method is self-contained and cheap to query. The predictor is trained from scratch on standard SD-VAE latents, with no DINOv2, CLIP or multimodal LLM involved, and it connects to the generator only through a set of conditioning tokens, so the recipe is not tied to a particular pretrained model. The condition prefix never attends to image tokens, so its key-value cache is computed once before the denoising loop. The ablations also show that the predictor needs no timestep input and that patch size 4 cuts the predicted tokens by four at no cost in FID, 31.00 vs 31.36. The predictor is trained once and shared across generators. With REPA on top, the full system reaches FID 1.32 for about a third of the training FLOPs that SiT-XL/2 with REPA spends to reach 1.42.
  • Some findings are useful beyond this method, and the paper is open about costs and failure cases. In Table 1, InfoNCE gives the best FID but the lowest ImageNet accuracy after fine-tuning, while the regression losses show the reverse. This is a concrete data point that features that are better for recognition are not automatically better as a generation condition, which is relevant to recent work that brings pretrained representations into diffusion models, such as REPA and RAE. On cost, Table 13 lists parameters and per-step FLOPs for both networks, with guidance and VAE decoding included in the per-image totals.
Critical weaknesses
  • Why including REPA? The headline result combines the proposed conditioning with REPA, and the paper does not separate the two. NEPA-DiT-XL is trained with the REPA loss, but there is no run of the same MM-DiT generator with REPA and a plain class embedding, and no result for NEPA-DiT-XL without REPA under the final protocol. The paper also does not say why REPA is still needed if the predicted embeddings already give the generator a good representation. The numbers that are available suggest that the gain on top of REPA is small. Without guidance and with the 250-step SDE sampler, NEPA-DiT-XL with REPA reaches 7.59 after 400K iterations (Table 10), while the REPA paper reports 7.9 for SiT-XL/2 with REPA at the same iteration count and with the same sampler. The architectures differ, so this is only indicative, which is why the direct ablation is needed.

  • Only ImageNet experiments. The experiments are narrower than the motivation. The abstract and introduction frame the problem around both class labels and text prompts, and Appendix B argues that the approach suits text, where an encoding computed once cannot adapt to the evolving image. All results, however, are class-conditional ImageNet at 256×256 in a single latent space. A single class token out of 1,000 is about the simplest condition there is, and arguably not where an adaptive, image-aware condition should matter most. Even a modest text-to-image experiment, such as the COCO setup used by U-ViT and in the REPA appendix, or results at 512×512, would make the contribution easier to assess.

  • The predictor's behavior at high noise is not analyzed. Near t = 1 the noisy patches carry almost no information, so the prediction can only depend on the class token (and position), and picking the right patch among the other patches of the same image is largely guesswork. With t sampled uniformly, a good part of predictor training is spent in this regime, and the t distribution is not ablated. This range also matters at sampling time, since guidance is only applied for t in [0.4, 1], and it is unclear what the predictions add over a class embedding there. The query-once result shows how much the generator relies on p_t matching the current state, which makes the quality of these predictions across t important. Reporting the top-1 accuracy of the InfoNCE task, or the similarity to the true clean embeddings, as a function of t, and showing how p_t changes along a sampling trajectory, would make the mechanism much clearer.

  • Train and test may contain gap. There is a train–test gap in what the predictor sees. During training it only reads forward-noised real latents, x_t = (1−t)x_0 + tε, and the generator is trained on predictions computed from those same inputs. At sampling time the predictor instead reads states produced by the generator, which drift from the forward-process marginals through model, i.e. the exposure problem.

  • compute vs. gain. Frankly to me the design choice to re-generate the condition at every denoising step is computationally expensive and somewhat counterintuitive. As the community rapidly converges toward 1–4 NFE generation, adding per-step overhead requires strong empirical justification. Without experiments demonstrating clear gains on few-step diffusion models, it is difficult to assess the practical value of this trade-off.

Questions for authors

Please see above.

3 comments
AuthorsOpenReview ↗

Reply to Reviewer HH7a

We thank the reviewer for the detailed review, and for recognizing that the predictor is self-contained and efficient to query and that the finding in Table 1 is useful beyond our method.

W1: Why is REPA still needed, and how much does ECG add on top of it under the final protocol?

We include REPA because it has become a common component of recent class-to-image generators, and ECG is compatible with it. For example, REG [1] adds REPA's alignment loss to its own training objective, DDT [2] applies REPA's alignment to its condition encoder, and ReDi [3] reports that its joint image-feature synthesis is complementary to REPA. REG (1.36) and DDT (1.26) appear in Table 9 with REPA in their training, so NEPA-DiT-XL + REPA is compared with them under the same use of REPA. ECG sets the condition that the generator reads, while REPA aligns the generator's intermediate features with DINOv2, so the two act on different parts of the generator and can be used together.

The two do not interfere, and their gains add up. At 400K iterations, adding ECG to REPA lowers FID from 7.97 to 7.59 without guidance and from 2.14 to 1.32 with guidance, and adding REPA to ECG lowers it from 8.59 to 7.59 and from 2.60 to 1.32.

ModelGuidanceFID
NEPA-DiT-XLnone8.59
REPA (our reproduction)none7.97
NEPA-DiT-XL + REPAnone7.59
NEPA-DiT-XLours2.60
REPA (our reproduction)REPA's best2.14
NEPA-DiT-XL + REPAours1.32

In the revised version, we will add this table to the main text and state the protocol of the REPA reproduction.

[1] Wu et al. Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Think. NeurIPS 2025.

[2] Wang et al. DDT: Decoupled Diffusion Transformer. CVPR 2026.

[3] Kouzelis et al. Boosting Generative Image Modeling via Joint Image-Feature Synthesis. NeurIPS 2025.

W2: Does ECG help beyond class-conditional ImageNet 256, for example in text-to-image generation?

Thank you for this valuable suggestion. We agree that text-to-image generation is where a condition recomputed at every step should matter most, and Multi-Embedding Prediction fits it naturally. The NEPA model reads a condition prefix followed by the noisy image, as a language model reads a prompt, so the language model that a text-to-image system already uses to encode the prompt can be fine-tuned with Multi-Embedding Prediction to become this conditioner. As a first result, we trained a text-to-image model with embedding prediction: at every sampling step the predictor reads the prompt followed by the noisy image, and the generator reads its predictions. On MS-COCO it reaches an FID of 19.1 and a CLIP score of 24.9, better than the following text-to-image baselines:

ModelFID ↓CLIP ↑
DALL·E27.50-
Chameleon26.74-
CoDI22.26-
Transfusion (linear)20.324.0
Ours19.124.9

W3: How does the predictor behave at high noise, does uniform t waste training there, and what do the predictions add where guidance is applied?

Thank you for this valuable question. We ran new experiments to study it. We follow the predictor along the sampler's own trajectories, since these are the states it reads at test time. We draw 5,000 samples (5 per class) with the 96-step ODE sampler and the guidance of the paper, record the prediction at every sampling step, and compute the InfoNCE top-1 retrieval of each prediction among the 64 patches of the final sample (with the random sampling chance 1.6%). Steps are t = 1 − k/96, shown rounded.

t1.00.90.80.70.60.50.30.1
Sampling state9.085.997.299.299.799.999.999.9

W3.1 How do the predictions behave at high noise?

The predictions become informative almost as soon as sampling starts: they identify their own patch for 85.9% of patches at t = 0.9 and for more than 97% from t = 0.8. Retrieval is close to chance (9.0%) only at pure noise, t = 1.0. After about ten of the 96 steps, the predictions therefore already describe the layout of the image that the sampler will produce. We add Figure 9 to Appendix E of the revised PDF to show this: along each trajectory, the predictions, colored by PCA, separate the object from the background before the decoded state becomes recognizable.

W3.2 Does uniform t waste predictor training at high noise?

Training at high noise is not wasted, because it gives the predictor the condition for the first sampling steps, where the layout of the sample is set. Even at high noise, the predictions along the sampling trajectory identify their own patch for 85.9% of patches at t = 0.9, so they already describe the layout that the sample will take, and the generator is conditioned on this layout from the first steps.

W3.3 What do the predictions add over a class embedding where guidance is applied?

Guidance is applied only at low and medium noise, t ≤ 0.6, where the predictions identify their own patch for at least 99.7% of patches. Section 4.4 and Appendix C write the guidance interval as t ∈ [0.4, 1], but there t means sampling progress, which is 1 − t in the convention x_t = (1 − t) x_0 + t ε. The interval is therefore t ∈ [0, 0.6], and the first 39 of the 96 sampling steps, from pure noise down to t = 0.6, run without guidance. On the 57 guided steps, each prediction describes its own region of the image being generated, which a single class embedding cannot do.

In the revised version, we will correct the notation of the guidance interval and add this table for all steps, with cosine similarity and InfoNCE loss, to the appendix.

W4: Does the predictor suffer an exposure gap on the generator's own states?

Thank you for this valuable question! First, a diffusion denoiser is also trained only on forward-noised data and reads its own sampling states at test time, and it still generates well. The predictor reads the same states as the generator, so it is in the same position. To verify it quantitatively, we compare the sampling states of W3 with forward-noised real image at the same t:

t1.00.90.80.70.60.50.30.1
Real image + fresh noise2.88.120.944.769.489.299.3100.0
Generated image + fresh noise2.88.822.246.670.990.099.399.9
Sampling state9.085.997.299.299.799.999.999.9

At every t, retrieval on the sampling states is at least as high as on forward-noised real or generated images. We attribute the higher retrieval at high noise to the deterministic ODE sampler: a state early on the trajectory already fixes the sample that it is scored against, whereas fresh noise is independent of the image. In the revised version, we will add this comparison to the appendix.

W5: Is recomputing the condition at every step worth its cost, as generation moves to 1 to 4 NFE?

Thank you for this valuable question. ECG and few-step generation do not conflict, because the few-step models used in practice are distilled from a strong multi-step teacher, and ECG makes this teacher stronger. Recent text-to-image models still sample with 4 to 8 NFE rather than one, and they reach it through distribution matching distillation (DMD) [6]. Z-Image-Turbo [4] samples with 8 NFE after distillation with variants of DMD, and Qwen-Image-Flash [5] distills multi-step Qwen-Image teachers into a 4-NFE student under the DMD framework. In DMD the teacher supplies the score that the student follows, so the student depends on the quality of its teacher, and DMD2 [7] traces part of the remaining gap to approximation errors in the teacher's score, which propagate to the distilled generator. ECG lowers the FID of the multi-step model with the generator and its training recipe fixed (Table 7), so it gives distillation a better teacher to start from.

The two methods also act on different parts of the pipeline. Distillation reduces the number of steps, and ECG improves the condition that the generator reads at each step. The predictor is frozen, so a distilled student can keep reading the predicted condition at each of its few steps.

In the revised version, we will discuss combining ECG with few-step distillation in Section 4.4.

[4] Z-Image Team. Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformer. arXiv 2025.

[5] Wu et al. Qwen-Image-Flash: Rethinking the Training Recipe for Few-Step Distillation. arXiv 2026.

[6] Yin et al. One-step Diffusion with Distribution Matching Distillation. CVPR 2024.

[7] Yin et al. Improved Distribution Matching Distillation for Fast Image Synthesis. NeurIPS 2024.

Dear Reviewer,

Thank you again for your time. We would like to check whether our response has addressed the concerns raised in your review. Could you kindly let us know if any questions remain, so that we have enough time to address them? We would be happy to provide any further clarification, and we look forward to your feedback.

Best regards,

The Authors.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.