- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper investigates how internal representations are formed and used in pixel-space diffusion Transformers and proposes Representation Guidance (RG). Building on a decoupled encoder–decoder architecture, the method introduces a low-dimensional linear bottleneck between the two components and modifies the flow-matching target using the difference between EMA model predictions with and without the internal representation. The authors examine the components through layer-wise representation analysis and ablation studies, and report competitive results on ImageNet at 256×256 and 512×512 resolutions, as well as text-to-image generation.
Critical strengths
-
The method is easy to understand, with reasonably extensive component ablations.
-
RG retains standard Transformer blocks and improves generation by modifying the training objective. Compared with a model using the same architecture and cfg settings, it does not require additional computation of the RG prediction offset during sampling.
-
The generation results are competitive. RG-XL achieves FID of 1.55 and 1.74 on ImageNet at 256×256 and 512×512, respectively. Experiments across multiple model sizes and text-to-image generation also demonstrate the potential scalability of the approach.
Critical weaknesses
-
The abstract emphasizes that RG “directly forms and exploits internal representations,” but does not explicitly explain the role of REPA. Considered alongside the abstract’s discussion of the limitations of external alignment, it can easily lead readers to underestimate the method’s dependence on pretrained representation supervision and overestimate the contribution of this paper. In addition, the broad conclusion of the first insight—that better representation quality does not necessarily translate into better generation—has already been explored in prior work. PixelREPA [1] directly documents the degradation caused by REPA as JiT training progresses.
-
DDT [2] already uses an encoder to extract internal representations, conditions a decoder on these representations, and applies REPA supervision to intermediate encoder features. Thus, the use of internal representations alone does not distinguish RG from DDT. Moreover, Eq. (2) closely resembles Model Guidance (MG) [3], which modifies the training target using differences between model predictions. Although these works are cited, the incremental contribution of RG remains insufficiently isolated. Authors may distinguish the inherited design choices from the proposed advances and add a matched MG baseline using the same encoder–decoder architecture, bottleneck, REPA supervision, and training budget.
-
The model still uses cfg during sampling in addition to RG during training, and both affect generation quality. I suggest reporting results for the baseline and RG without cfg, to assess the benefit of RG itself and its dependence on sampling-time guidance.
-
Table 8 shows that, under the same bottleneck setting, adding RG reduces FID from 3.79 to 3.28 with REPA, whereas adding guidance increases FID from 4.71 to 8.99 without REPA. This indicates that, under the current experimental settings, the effectiveness of guidance clearly depends on external representation supervision. How should this phenomenon be explained?
[1] Representation Alignment for Just Image Transformers is not Easier than You Think. arXiv: 2603.14366
[2] DDT: Decoupled Diffusion Transformer. arXiv:2504.05741, 2025.
[3] Diffusion Models without Classifier-free Guidance. arXiv:2502.12154, 2025.
Questions for authors
-
Please report the training time of RG relative to the baseline, including the GPU-hours. The cost should include the additional EMA predictions.
-
RG introduces a guidance scale , a guidance timestep interval, and a warm-up setting, while also requiring additional EMA predictions during training. The ablations indicate that performance is sensitive to the guidance scale and timestep interval. Although Table 8 shows a meaningful FID improvement for B/16, it remains unclear whether these hyperparameters transfer directly across model sizes and tasks. Hyperparameter tuning can be particularly expensive at larger training scales.