- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces NEPA-based Embedding Conditioned Generation, where a model predicts clean-image embeddings from the current noisy image and uses them as dynamic conditions for DiT at each denoising step. On ImageNet 256×256, the method improves generation quality and reaches an FID of 1.32 when combined with REPA.
Critical strengths
- Clear methodological novelty. The paper introduces a simple but effective idea of replacing fixed diffusion conditioning with embeddings dynamically predicted from the current noisy image.
- Strong and well-controlled ablations. The experiments compare different prediction losses, conditioning strategies, model scales, and query frequencies, helping isolate where the gains come from.
- Competitive efficiency and performance. The final NEPA-DiT-XL + REPA model achieves an FID of 1.32 on ImageNet 256×256 while using substantially less training compute than strong REPA-based baselines.
Critical weaknesses
- Limited analysis of the predicted representations. While the paper shows that NEPA embeddings improve FID, it provides relatively little analysis of what information these predicted embeddings encode or why they are particularly effective for generation.
- Unclear necessity of the predictive embedding. Since the predicted embedding is a deterministic function of the class condition and the current noisy image, it does not introduce additional information. It remains unclear why an explicitly pretrained predictive representation is fundamentally preferable to letting the generator learn the same transformation internally.
- Unclear semantics of the clean-image embedding. The clean-image target is produced by NEPA’s own jointly learned embedding layer rather than by a fixed semantic encoder. It is therefore unclear what information this embedding is expected to represent and whether it captures meaningful clean-image structure or simply a latent space that co-adapts to the prediction objective.
Questions for authors
see above
3 comments
Dear Reviewer,
Thank you again for your time. We would like to check whether our response has addressed the concerns raised in your review. Could you kindly let us know if any questions remain, so that we have enough time to address them? We would be happy to provide any further clarification, and we look forward to your feedback.
Best regards,
The Authors
concern solved. keep my positive score
Reply to Reviewer Gb1a
We thank the reviewer for recognizing the novelty of the idea, the well-controlled ablations and the efficiency of the final model.
W1 and W3: What do the predicted embeddings and their jointly learned clean-image targets encode, and why do they work as a condition for generation?
The predictions form a rich, spatially organized representation of the clean image, and the generator reads this representation as its condition. We already describe the target space in Section 3.3. The targets are the predictor's own embedding of the clean image, and the contrastive loss keeps them distinct from one another, so they cannot collapse into a space that only fits the objective. The predictions estimate these targets from the class and the noisy image.
We add Appendix E to the revised PDF to show what the predictions encode. Figure 8 shows that the predictions recover the layout of the clean image before the noisy image shows it: under one PCA basis shared by all images and noise levels, they separate objects from backgrounds while the decoded noisy latent is still unrecognizable, and the objects of different images tend to take similar colors. Figure 9 shows the same along sampling trajectories. Figure 10 shows that the predictions of one region resemble each other: the similarity between one prediction and the targets of its image spreads over the region of the query at high noise and narrows to the query's own position as the noise decreases.
This representation is what makes the predictions a good condition. With the generator and its training recipe fixed, the predictions give 25.04 FID against 36.39 for the class embedding (Table 7).
In the revised version, we will describe the target space more prominently in Section 3.3.
W2: The prediction adds no information beyond the class and the noisy image, so why is an explicit predictor better than letting the generator learn the same mapping?
The predictor adds no new information, but it splits generation into two easier steps: first predict a good condition in embedding space, then generate the image from it. We already discuss this in Sections 4.1 and 4.2.
In the first step, MEP trains the predictor to predict the clean-image embeddings from the class and the noisy image. Its InfoNCE loss makes each predicted embedding distinguishable from the other embeddings of the same image, so the predictions keep the fine-grained detail of the clean image. The generator then receives a condition that already describes the clean image region by region, and its own job reduces to generating an image that matches it. When the same network is instead trained end to end with the generator, it receives only the generator's regression gradient, nothing pushes its outputs to be distinguishable, and the generator has to learn both steps at once from one loss. With the same architecture and FLOPs, the two-step design gives 25.04 FID against 29.52 for end-to-end training (Table 7).
In the revised version, we will make this two-step view more prominent in Sections 4.1 and 4.2, so that readers can follow why the explicit predictor helps.