- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates how to retrofit pretrained diffusion Transformers into recurrent models. At each denoising timestep, the method keeps the noisy input fixed and uses the previous prediction and additional scratchpad tokens as context for iterative updates through a weight-tied network. Experiments are conducted across three architectures and two model sizes.
Critical strengths
-
Retrofitting pretrained models with recurrent computation is a practically meaningful research direction.
-
The experiments cover three architectures: DiT, SiT, and FLUX.2, providing a certain degree of cross-architecture validation.
Critical weaknesses
-
The central idea is directly from TRM (arXiv:2510.04871). There is no new theoretical insight, no new architectural primitive, and no demonstration that the same recipe unlocks capabilities that pure parameter scaling or existing inference-time methods cannot.
-
Each loop processes 640 tokens instead of 256. Table 5 shows that a 4-loop forward pass already costs several times more FLOPs than the baseline or even the ×4 model under comparable conditions. In a fair iso-FLOPs comparison at inference, the advantage largely disappears. The paper never convincingly shows that the method is preferable to simply using a larger feed-forward model or more sampling steps under the same compute budget.
-
The paper presents the independent scratchpad as one of its main architectural designs, but Table 4 does not provide a direct ablation that retains the fixed noisy input and the previous prediction while removing only the scratchpad. The existing "w/o Previous Answer" experiment demonstrates the importance of prediction feedback, but does not establish the necessity of an independent scratchpad.
-
The core contribution and motivation of the paper are unclear, and the experimental results do not adequately support the claimed contributions. All the compared methods are implemented by the authors themselves, making it difficult to verify whether the ELT baseline faithfully reproduces the original method (arxiv:2604.09168). The authors should provide detailed baseline pseudocode and implementation details, or compare against the results reported in the original paper.
-
Table 1 should include results with classifier-free guidance. The results in Table 2 do not warrant a separate table, as precision and recall are generally not considered primary evaluation metrics for image generation. Furthermore, the paper lacks comparisons with current state-of-the-art image generation models, making the experimental results less convincing.
-
The computational cost analysis is insufficiently emphasized in the main paper, despite efficiency being one of its central motivations. Although the paper includes FLOPs vs. FID curves, the detailed comparisons of training and inference costs are relegated to Appendix. These substantial computational overheads are critical to evaluating the paper's efficiency claims and should be explicitly presented and discussed in the main text.
Questions for authors
- The AI Use Statement acknowledges that generative AI tools were used to produce the figures. However, several figures appear to lack sufficient manual refinement in terms of layout, visual consistency, and readability. Directly using AI-assisted figures without adequate polishing undermines the professionalism and credibility of the paper.