- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces LoopREG, a representation-spervised recurrent diffusion transformer that improves the image generation quality by looping on specific supervised representative block instead of sampling more steps, also this needs an alignment to work.
The paper evalutes this method on ImageNet using SiT-B/2 Sit-L/2 and SiT-XL/2 as backbone. At 400K training steps, LoopREG-B/2 reduces FID from 15.08 for the local REG baseline to 4.91, while LoopREG-XL/2 improves FID from 3.38 to 2.84 by increasing inference depth from two to three visits without changing model weights.
Critical strengths
Stengths:
-
The method is simple and easy implement, just looping on a shared block will improve the results without adding more parameters.
-
This paper introduces a really well ablation experiment. LoopSiT → LoopREPA → LoopREG is real informative.
Critical weaknesses
There do have some main issue concerns me.
-
Incremental novelty, the looped-block concept already exists. This paper claims the loop should be applied at the aligned block, but never explains why alignment makes it work. I see this as an experimental finding without any theoretical guarantee. I suggest the authors explain why alignment works, supported by some theoretical results.
-
It is also hard for me to be convinced whether if loop actually help in the final stage model, since the results showed on the XL part, the gain at 400K is already marginal, but 400K is considered to be early stage for XL. Published paper said XL converge around 4M steps, so more steps results 800K, 1M, 2,4M, 4M results need to be show to convince loop does help than sampling more.
-
The figure 2 is generated by AI, it is better to be redraw by author himself/ herself.
Questions for authors
Beside the weakness part, i have a question here in mind.
How does LoopREG compare with an ELT-style self-distillation loop, or a teacher-free alignment such as SRA, on the same backbone?
10 comments
Points 2-3: Training maturity, compute, and Figure 2 (Part 2/3)
2. Training maturity and increasing sampling steps.
The XL results reported in our manuscript are 400K training steps. With access to four H100 GPUs shared with other users, computational resources are relatively tight, making it impractical to train the XL model for more steps within the rebuttal period. We conduct the longer-training study on SiT-B/2 instead. Paired native REG/LoopREG continuations test whether the extra-loop benefit persists as training progresses, while measuring quality and training cost at the same milestones.
The native REG/LoopREG B/2 continuation now has completed 50K evaluations at 600K for both models and at 800K for LoopREG. At 600K, REG gives FID/Recall 12.9292/0.6297; LoopREG gives 12.5706/0.6343 at K_infer=2, 4.7137/0.5227 at K_infer=3, and 5.9406/0.4490 at K_infer=4. At 800K, the LoopREG values are 11.4055/0.6394, 4.4317/0.5246, and 5.6817/0.4610. The within-checkpoint K=2-to-3 FID reduction persists at 62.50% and 61.14%, respectively. All use EMA, seed 9201, 250 SDE steps, and image/semantic CFG=1. We are continuing both native B/2 models from 400K toward 1M total steps on four H100s at global batch 256 without gradient accumulation. The remaining evaluations are REG at 800K and both models at 1M: REG K_infer=1 and LoopREG K_infer=2/3/4, with 50K images per condition. This study tests whether extra-depth gains persist with training maturity and compares REG/LoopREG quality and incremental training cost at the same milestones.
The completed results and training costs are included in the revised main text and Appendix I.7. From 400K to 600K, recorded incremental training costs are 19.29 H100 GPU-hours for REG and 20.85 for LoopREG, or 0.08683/0.09382 seconds per retained successful update. LoopREG includes 0.71 GPU-hours of recorded restart waste; these are active training totals, excluding historical 0-to-400K training, setup, checkpoint I/O, and evaluation. As each remaining milestone completes, we will post its results in this reviewer thread and update the revised main text and appendix. Before the November 18 response deadline, we will also post a progress update listing any unfinished runs, even if no additional milestone has completed. Any remaining continuation and seed-replication results will be included in the final manuscript.
The main configurations currently use one training seed. Two additional independent K_train=2 seeds are scheduled after the paired 1M evaluations; neither has started. Each will train from scratch to 400K with fixed hyperparameters, then receive predefined K_infer=2/3 evaluations of 50K images each to test sensitivity to training randomness. We will first complete one seed and its evaluations. The second will run during discussion if enough time remains for the complete run; otherwise it will follow after discussion. We will report each seed and the mean/standard deviation across the original and two new seeds, posting completed results here and in the revised main text and appendix; the completed three-seed study will enter the final manuscript.
The next long-training target is 4M total steps per B/2 model. After the paired 1M evaluations and both independent seed replications, we will use the remaining shared-GPU training windows during discussion to continue toward this target. We will report completed milestones here and in the revised main text and appendix. Any training unfinished by the response deadline will continue afterward, with the 4M results included in the final manuscript.
The separate question of spending inference compute on more sampling steps can now be answered at B/2:
| Model | K_infer | Sampling steps | FID ↓ | Recall | Time (s) |
|---|---|---|---|---|---|
| REG | 1 | 289 | 15.24 | 0.6337 | 7.59 |
| LoopREG, K_train=2 | 2 | 268 | 14.73 | 0.6341 | 7.58 |
| LoopREG, K_train=2 | 3 | 250 | 5.34 | 0.5076 | 7.61 |
All three configurations use image CFG=1. These are four-H100 sampler-plus-VAE times for 256 images, averaged over two session medians. The largest matched-time gap is 0.634%. At this budget, extra recurrence gives lower FID and lower Recall than increasing sampling steps.
3. Figure 2.
We have redrawn Figure 2 in the revised paper. It explicitly separates training-time DINO supervision from inference, shows alignment at the terminal recurrent state, and preserves the shared block and joint image/semantic pathways. We had found that the AI-generated architecture diagram clearly conveyed the intended design and was considerably more aesthetically pleasing than a hand-drawn version; therefore, we adopted it directly in the paper before.
Additional question: ELT and teacher-free controls (Part 3/3)
ELT-style and teacher-free controls.
Both methods generate class-conditional ImageNet images at 256×256; ELT uses 512 sampling steps. We compare its published diffusion results with our completed 50K evaluations below. ELT values and settings come from Table 2 and Section 5.1. In ELT, N×L denotes N distinct layers repeated L times at inference; our K_train/K_infer count applications of one shared block. The prepared revised manuscript includes the L/2 comparison and the masked-model table in Appendix I.5 (Tables 16–17, p. 27).
LoopREG-L/2 reaches FID 3.2429 versus 3.16 for ELT 8N×4L (a 0.0829 difference, 2.62% higher), with about 13.26% fewer model parameters, 60% fewer generator-training image presentations, and 51.17% fewer sampler steps. This is a competitive quality–resource trade-off under the reported configurations.
| Model / inference depth | Parameters (M) | Training steps × batch | Sampler | Image CFG | FID ↓ |
|---|---|---|---|---|---|
| ELT 4N×8L | 271 | 500K × 512 | 512-step DDPM | 3 | 3.96 |
| ELT 8N×4L | 539 | 500K × 512 | 512-step DDPM | 3 | 3.16 |
| ELT 16N×2L | 1100 | 500K × 512 | 512-step DDPM | 3 | 2.83 |
| LoopREG-B/2, K_train=2, K_infer=3 | 139.03 | 400K × 256 | 250-step SDE | 1 | 5.3405 |
| LoopREG-B/2, K_train=2, K_infer=3 | 139.03 | 400K × 256 | 250-step SDE | 2 | 3.9672 |
| LoopREG-B/2, K_train=3, K_infer=5 | 139.03 | 400K × 256 | 250-step SDE | 1 | 4.9499 |
| LoopREG-L/2, K_train=2, K_infer=3 (new 50K) | 467.53 | 400K × 256 | 250-step SDE | 1 | 3.2429 |
| LoopREG-XL/2, K_train=2, K_infer=3 (historical 50K) | 685.02 | 400K × 256 | 250-step SDE | 1 | 2.8376 |
ELT diffusion uses width 2048 and the Stable Diffusion v1.4 VAE; LoopREG-B/2, L/2, and XL/2 use widths 768, 1024, and 1152 and sd-vae-ft-ema. Training exposure is 256M versus 102.4M image presentations, respectively: a 2.5× exposure difference, not a FLOPs ratio. Our parameter counts are computed from the actual model source, including the semantic pathway and projector and excluding DINO and VAE. Our semantic-token CFG is fixed at 1, retaining the conditional semantic prediction without extra guidance. Image CFG is a separate coefficient, active during sampling times [0, 0.85].
For LoopREG-L/2 at K_infer=3, analytical counting of all executed denoiser dense operators gives 179.59 GFLOPs/call and 44.90 TFLOPs/image over 250 calls (2 FLOPs/MAC, including the executed alignment projector; excluding VAE, normalization, softmax, activations, and other elementwise operations). For a common attention+MLP core comparison, ours is 43.798 TFLOPs/image. If ELT uses latent patch size 2 (256 tokens), standard dense attention/MLP, and two CFG branches at all 512 steps, its core is 862.02 TFLOPs/image, 19.68× ours. This ratio is conditional: ELT does not disclose its patch size or full CFG execution schedule, and these are analytical operation counts, not measured latencies. Appendix I.6 (Table 18, p. 28) gives the counting scope and patch-size sensitivity.
ELT also reports a separate masked-generation branch (BERT/MaskGIT with discrete tokens) in ELT's Table 1:
| Model / inference depth | Parameters (M) | Training | Sampling runs | FID ↓ |
|---|---|---|---|---|
| ELT-L 8N×3L | 101 | 270 epochs, batch 512 | 24 | 2.2 |
| ELT-L 12N×2L | 152 | 270 epochs, batch 512 | 24 | 2.1 |
| ELT-XL 7N×4L | 111 | 270 epochs, batch 512 | 24 | 2.0 |
Here 24 denotes sampling network runs. Section 5.1 and Table 5 specify 270 training epochs and batch 512, rather than per-row optimizer-step counts. The paper sweeps image guidance but does not list the chosen strength for each Table 1 row.
LoopREG uses frozen DINOv2 endpoint alignment and REG semantic denoising; ELT uses maximum-loop predictions to supervise intermediate loops through ILSD. Our supervision-site and alignment-loss interventions isolate the role of external alignment within REG. A same-backbone alignment-versus-ILSD experiment has not been run and is not scheduled in the remaining discussion-period GPU budget, which is allocated to paired continuation and independent training seeds.
The zero-alignment control retains DINO semantic targets and isolates direct hidden-state alignment within REG. A teacher-free SRA control has not been run and is not scheduled in the remaining discussion-period budget. The current objective-control evidence is the completed alignment-zero experiment with the semantic pathway retained.
Point 1: Why endpoint alignment matters (Part 1/3)
Thank you for asking for a sharper explanation of alignment and for distinguishing early from mature-model behavior.
1. Why supervision at the recurrent state matters.
The architectural analysis identifies the gradient path. For a recurrent endpoint h_K followed by a map T to the supervised layer, the alignment gradient is pulled back through the Jacobian of T. At co-location, T is the identity, so alignment directly shapes the recurrent endpoint. A distinct loop block placed after the alignment branch receives no direct gradient from that branch. In addition, weight sharing accumulates gradient contributions from all visited states into the same recurrent operator. These identities motivate the placement and sharing choices.
The site interventions test that prediction: fixing supervision at depth four favors the co-located loop, while moving supervision and recurrence together recovers extra-depth benefits at other sites. We have now added the requested objective-level control:
We trained a 400K-step control from scratch with alignment coefficient zero while retaining REG's noisy semantic-token input, semantic output, and semantic denoising objective. The latter still uses DINO targets. Architecture, training horizon K_train=2, and the semantic-loss coefficient are preserved. The positive comparison uses the original aligned checkpoint.
The alignment-free control deteriorates with extra recurrence while the aligned model improves, isolating alignment's contribution within REG's semantic architecture. Under its stated assumptions, the finite-horizon recurrence analysis explains how a stable trajectory can pass a task-preferred state before equilibrium. The location and objective interventions test this design principle in the trained model; the analysis does not predict the FID-optimal depth.