- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Representation-Augmented Diffusions (RAD). At each denoising step, the model predicts a semantic representation and feeds it back into the subsequent denoiser; the representation is used during both training and inference. The paper formulates this process as a sequential conditional latent-variable model and derives a variational objective with a conditional denoising term and a representation-policy KL term.
Critical strengths
- A simple, clear core mechanism with a large early-FID gain. Feeding the predicted representation back into the denoiser improves FID from 22.9 to 10.3, and the method can be combined with REG.
- Potentially useful trends in early training and low-step sampling. RAD reaches lower FID more quickly across model scales and training budgets, and its 10-step Euler result outperforms the reported 100-step REPA result. The exact efficiency multiplier requires stronger controls, but the trend is worth further investigation.
Critical weaknesses
- The main efficiency and generalization claims are not established under sufficiently matched conditions. RAD is trained for at most 400K steps and is not compared with REPA/SiT at mutual convergence. At 400K steps, RAD-B/L/XL recall is 0.49/0.49/0.50, whereas the corresponding REPA recall is approximately 0.65. The current evidence therefore shows faster attainment of a particular FID at a high-precision, low-coverage operating point. The GenEval improvement of XL/2 text-to-image generation from 0.258 to 0.266 is marginal. Evaluating gains at such low absolute performance levels does not provide compelling evidence for strong generalization.
- The theory strictly derives the structure of Eq. (27), not main-text Eq. (9) or the final algorithm. Eq. (9) replaces the ELBO denoising weight (w_t) with 1. The practical objective further replaces an unbounded low-noise SNR weight with a saturated weight and adds a mixed flow objective and per-sample stop-gradient adaptive reweighting. Consequently, “training maximizes a variational lower bound” does not apply to the best-performing experimental model.
Questions for authors
Can the authors provide RAD learning curves to at least 1M/4M steps and compare FID, precision, and recall after both methods approach convergence?
6 comments
Author Response (Contd.)
[W2] The bound vs. the trained objective.
We agree, and we have revised "Training maximizes a variational lower bound" in the abstract, the introduction and Sec. 2.2 to "The training objective is derived from a variational lower bound". The objective we train is a reweighted surrogate of the bound in Eq. (4). It differs from the bound in three ways:
- Following DDPM [Ho et al., 2020] and most diffusion models optimized for sample quality, we set the weight w_t of the denoising term to 1.
- We replace the alignment weight SNR(t)/2β², which grows without bound as t → 0, by its bounded counterpart λ_t = SNR(t)/(SNR(t) + β²).
- We add an adaptive per-sample weight on the alignment loss inspired by the literature on few-step models such as MeanFlow [Geng et al., 2025]. This has little effect on FID (8.5 vs. 8.3, Table 6).
Re. the mixed flow objective. We have added a theoretical result in App. A.9 ("Combining the Gaussian and flow policies", Eqs. 43–44), which shows that the flow likelihood term is a principled part of the variational bound and is not derived heuristically. Therefore, our practical objective is a reweighted version of the theoretical variational bound, and we have clarified this in the main text by revising our claims.
We hope our responses have addressed your concerns, and it would be great if you could consider improving your score. We are happy to answer any follow-up questions.
Thank you for the additional results and clarification. The 1M iteration result alleviates my concern that the FID gain is purely an early-training effect. However, RAD’s recall remains much lower than REPA’s, so the gain still comes with reduced sample coverage and diversity.
My main remaining concern is CFG. All ImageNet and text-to-image results are reported without CFG, and the paper does not explain why. This matters because CFG is standard in conditional generation and may further reduce RAD’s already low recall. Could you report RAD and the baselines with CFG, including FID, precision, recall.
Author Response
We thank the reviewer for the detailed feedback. We first list the relevant changes in the revision (highlighted in blue) and then answer each weakness and question.
Relevant changelog:
[W1, Q1] Efficiency and generalization claims.
As requested, we trained RAD-XL/2 for 1M iterations on class-conditional ImageNet and compared it with REPA at several training budgets in Table R1 below (also Tables 1 and 7 in the paper; full learning curves in Table 9). Apart from FID, REPA reports no metrics at 1M iterations (and we could not train a 1M REPA baseline ourselves due to compute limitations), so we also compare against REPA at 4M iterations. We make the following observations:
Table R1: REPA and RAD at different training budgets. ImageNet 256×256, FID-50k, Euler–Maruyama SDE with 250 steps, no CFG. REPA numbers from its paper (Tables 3 and 8). RAD leads in FID, IS and precision in every available comparison, whereas REPA has higher recall.