acceptodds
Under review as a conference paper at ICLR 2027

LoopREG: Representation Supervision Enables Test-Time Recurrent Depth in Diffusion Transformers

Abstract

Reusing Transformer blocks offers a way to increase diffusion-model computation at test time, but naive recurrence can degrade generation beyond its training depth. We introduce LoopREG, a representation-supervised recurrent diffusion Transformer that repeatedly applies a shared block at the state supervised by a pretrained visual encoder. Alignment is applied after the final recurrent visit during training; inference extends the same learned operator to additional visits. On ImageNet 256 x 256, LoopREG reduces SiT-B/2 FID from the local REG baseline's 15.08 to 4.91 at 400K training steps, a 67.41% reduction. Within a checkpoint trained with two visits, one extra inference visit improves FID from 14.71 to 5.34 with unchanged weights. The benefit persists on SiT-L/2 and SiT-XL/2. Controlled interventions show that useful recurrence follows the location of representation supervision, while REPA alignment produces a smaller extrapolation benefit. These results make representation-supervised recurrence a practical route to improving diffusion generation through additional inference depth.

Primary area:
generative models
Keywords:
diffusion, loop, supervision

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.20 over 5 reviews, above 49% of the others.
Reviewer_546VOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces LoopREG, a representation-spervised recurrent diffusion transformer that improves the image generation quality by looping on specific supervised representative block instead of sampling more steps, also this needs an alignment to work.

The paper evalutes this method on ImageNet using SiT-B/2 Sit-L/2 and SiT-XL/2 as backbone. At 400K training steps, LoopREG-B/2 reduces FID from 15.08 for the local REG baseline to 4.91, while LoopREG-XL/2 improves FID from 3.38 to 2.84 by increasing inference depth from two to three visits without changing model weights.

Critical strengths

Stengths:

  1. The method is simple and easy implement, just looping on a shared block will improve the results without adding more parameters.

  2. This paper introduces a really well ablation experiment. LoopSiT → LoopREPA → LoopREG is real informative.

Critical weaknesses

There do have some main issue concerns me.

  1. Incremental novelty, the looped-block concept already exists. This paper claims the loop should be applied at the aligned block, but never explains why alignment makes it work. I see this as an experimental finding without any theoretical guarantee. I suggest the authors explain why alignment works, supported by some theoretical results.

  2. It is also hard for me to be convinced whether if loop actually help in the final stage model, since the results showed on the XL part, the gain at 400K is already marginal, but 400K is considered to be early stage for XL. Published paper said XL converge around 4M steps, so more steps results 800K, 1M, 2,4M, 4M results need to be show to convince loop does help than sampling more.

  3. The figure 2 is generated by AI, it is better to be redraw by author himself/ herself.

Questions for authors

Beside the weakness part, i have a question here in mind.

How does LoopREG compare with an ELT-style self-distillation loop, or a teacher-free alignment such as SRA, on the same backbone?

10 comments
AuthorsOpenReview ↗

Point 1: Why endpoint alignment matters (Part 1/3)

Thank you for asking for a sharper explanation of alignment and for distinguishing early from mature-model behavior.

1. Why supervision at the recurrent state matters.

The architectural analysis identifies the gradient path. For a recurrent endpoint h_K followed by a map T to the supervised layer, the alignment gradient is pulled back through the Jacobian of T. At co-location, T is the identity, so alignment directly shapes the recurrent endpoint. A distinct loop block placed after the alignment branch receives no direct gradient from that branch. In addition, weight sharing accumulates gradient contributions from all visited states into the same recurrent operator. These identities motivate the placement and sharing choices.

The site interventions test that prediction: fixing supervision at depth four favors the co-located loop, while moving supervision and recurrence together recovers extra-depth benefits at other sites. We have now added the requested objective-level control:

We trained a 400K-step control from scratch with alignment coefficient zero while retaining REG's noisy semantic-token input, semantic output, and semantic denoising objective. The latter still uses DINO targets. Architecture, training horizon K_train=2, and the semantic-loss coefficient are preserved. The positive comparison uses the original aligned checkpoint.

Alignment coefficientK_infer=2 FID ↓K_infer=3 FID ↓K_infer=4 FID ↓
0.5, LoopREG (original sweep)14.715.346.20
0, semantic pathway retained24.7431.6851.54

The alignment-free control deteriorates with extra recurrence while the aligned model improves, isolating alignment's contribution within REG's semantic architecture. Under its stated assumptions, the finite-horizon recurrence analysis explains how a stable trajectory can pass a task-preferred state before equilibrium. The location and objective interventions test this design principle in the trained model; the analysis does not predict the FID-optimal depth.

AuthorsOpenReview ↗

Points 2-3: Training maturity, compute, and Figure 2 (Part 2/3)

2. Training maturity and increasing sampling steps.

The XL results reported in our manuscript are 400K training steps. With access to four H100 GPUs shared with other users, computational resources are relatively tight, making it impractical to train the XL model for more steps within the rebuttal period. We conduct the longer-training study on SiT-B/2 instead. Paired native REG/LoopREG continuations test whether the extra-loop benefit persists as training progresses, while measuring quality and training cost at the same milestones.

The native REG/LoopREG B/2 continuation now has completed 50K evaluations at 600K for both models and at 800K for LoopREG. At 600K, REG gives FID/Recall 12.9292/0.6297; LoopREG gives 12.5706/0.6343 at K_infer=2, 4.7137/0.5227 at K_infer=3, and 5.9406/0.4490 at K_infer=4. At 800K, the LoopREG values are 11.4055/0.6394, 4.4317/0.5246, and 5.6817/0.4610. The within-checkpoint K=2-to-3 FID reduction persists at 62.50% and 61.14%, respectively. All use EMA, seed 9201, 250 SDE steps, and image/semantic CFG=1. We are continuing both native B/2 models from 400K toward 1M total steps on four H100s at global batch 256 without gradient accumulation. The remaining evaluations are REG at 800K and both models at 1M: REG K_infer=1 and LoopREG K_infer=2/3/4, with 50K images per condition. This study tests whether extra-depth gains persist with training maturity and compares REG/LoopREG quality and incremental training cost at the same milestones.

The completed results and training costs are included in the revised main text and Appendix I.7. From 400K to 600K, recorded incremental training costs are 19.29 H100 GPU-hours for REG and 20.85 for LoopREG, or 0.08683/0.09382 seconds per retained successful update. LoopREG includes 0.71 GPU-hours of recorded restart waste; these are active training totals, excluding historical 0-to-400K training, setup, checkpoint I/O, and evaluation. As each remaining milestone completes, we will post its results in this reviewer thread and update the revised main text and appendix. Before the November 18 response deadline, we will also post a progress update listing any unfinished runs, even if no additional milestone has completed. Any remaining continuation and seed-replication results will be included in the final manuscript.

The main configurations currently use one training seed. Two additional independent K_train=2 seeds are scheduled after the paired 1M evaluations; neither has started. Each will train from scratch to 400K with fixed hyperparameters, then receive predefined K_infer=2/3 evaluations of 50K images each to test sensitivity to training randomness. We will first complete one seed and its evaluations. The second will run during discussion if enough time remains for the complete run; otherwise it will follow after discussion. We will report each seed and the mean/standard deviation across the original and two new seeds, posting completed results here and in the revised main text and appendix; the completed three-seed study will enter the final manuscript.

The next long-training target is 4M total steps per B/2 model. After the paired 1M evaluations and both independent seed replications, we will use the remaining shared-GPU training windows during discussion to continue toward this target. We will report completed milestones here and in the revised main text and appendix. Any training unfinished by the response deadline will continue afterward, with the 4M results included in the final manuscript.

The separate question of spending inference compute on more sampling steps can now be answered at B/2:

ModelK_inferSampling stepsFID ↓RecallTime (s)
REG128915.240.63377.59
LoopREG, K_train=2226814.730.63417.58
LoopREG, K_train=232505.340.50767.61

All three configurations use image CFG=1. These are four-H100 sampler-plus-VAE times for 256 images, averaged over two session medians. The largest matched-time gap is 0.634%. At this budget, extra recurrence gives lower FID and lower Recall than increasing sampling steps.

3. Figure 2.

We have redrawn Figure 2 in the revised paper. It explicitly separates training-time DINO supervision from inference, shows alignment at the terminal recurrent state, and preserves the shared block and joint image/semantic pathways. We had found that the AI-generated architecture diagram clearly conveyed the intended design and was considerably more aesthetically pleasing than a hand-drawn version; therefore, we adopted it directly in the paper before.

AuthorsOpenReview ↗

Additional question: ELT and teacher-free controls (Part 3/3)

ELT-style and teacher-free controls.

Both methods generate class-conditional ImageNet images at 256×256; ELT uses 512 sampling steps. We compare its published diffusion results with our completed 50K evaluations below. ELT values and settings come from Table 2 and Section 5.1. In ELT, N×L denotes N distinct layers repeated L times at inference; our K_train/K_infer count applications of one shared block. The prepared revised manuscript includes the L/2 comparison and the masked-model table in Appendix I.5 (Tables 16–17, p. 27).

LoopREG-L/2 reaches FID 3.2429 versus 3.16 for ELT 8N×4L (a 0.0829 difference, 2.62% higher), with about 13.26% fewer model parameters, 60% fewer generator-training image presentations, and 51.17% fewer sampler steps. This is a competitive quality–resource trade-off under the reported configurations.

Model / inference depthParameters (M)Training steps × batchSamplerImage CFGFID ↓
ELT 4N×8L271500K × 512512-step DDPM33.96
ELT 8N×4L539500K × 512512-step DDPM33.16
ELT 16N×2L1100500K × 512512-step DDPM32.83
LoopREG-B/2, K_train=2, K_infer=3139.03400K × 256250-step SDE15.3405
LoopREG-B/2, K_train=2, K_infer=3139.03400K × 256250-step SDE23.9672
LoopREG-B/2, K_train=3, K_infer=5139.03400K × 256250-step SDE14.9499
LoopREG-L/2, K_train=2, K_infer=3 (new 50K)467.53400K × 256250-step SDE13.2429
LoopREG-XL/2, K_train=2, K_infer=3 (historical 50K)685.02400K × 256250-step SDE12.8376

ELT diffusion uses width 2048 and the Stable Diffusion v1.4 VAE; LoopREG-B/2, L/2, and XL/2 use widths 768, 1024, and 1152 and sd-vae-ft-ema. Training exposure is 256M versus 102.4M image presentations, respectively: a 2.5× exposure difference, not a FLOPs ratio. Our parameter counts are computed from the actual model source, including the semantic pathway and projector and excluding DINO and VAE. Our semantic-token CFG is fixed at 1, retaining the conditional semantic prediction without extra guidance. Image CFG is a separate coefficient, active during sampling times [0, 0.85].

For LoopREG-L/2 at K_infer=3, analytical counting of all executed denoiser dense operators gives 179.59 GFLOPs/call and 44.90 TFLOPs/image over 250 calls (2 FLOPs/MAC, including the executed alignment projector; excluding VAE, normalization, softmax, activations, and other elementwise operations). For a common attention+MLP core comparison, ours is 43.798 TFLOPs/image. If ELT uses latent patch size 2 (256 tokens), standard dense attention/MLP, and two CFG branches at all 512 steps, its core is 862.02 TFLOPs/image, 19.68× ours. This ratio is conditional: ELT does not disclose its patch size or full CFG execution schedule, and these are analytical operation counts, not measured latencies. Appendix I.6 (Table 18, p. 28) gives the counting scope and patch-size sensitivity.

ELT also reports a separate masked-generation branch (BERT/MaskGIT with discrete tokens) in ELT's Table 1:

Model / inference depthParameters (M)TrainingSampling runsFID ↓
ELT-L 8N×3L101270 epochs, batch 512242.2
ELT-L 12N×2L152270 epochs, batch 512242.1
ELT-XL 7N×4L111270 epochs, batch 512242.0

Here 24 denotes sampling network runs. Section 5.1 and Table 5 specify 270 training epochs and batch 512, rather than per-row optimizer-step counts. The paper sweeps image guidance but does not list the chosen strength for each Table 1 row.

LoopREG uses frozen DINOv2 endpoint alignment and REG semantic denoising; ELT uses maximum-loop predictions to supervise intermediate loops through ILSD. Our supervision-site and alignment-loss interventions isolate the role of external alignment within REG. A same-backbone alignment-versus-ILSD experiment has not been run and is not scheduled in the remaining discussion-period GPU budget, which is allocated to paired continuation and independent training seeds.

The zero-alignment control retains DINO semantic targets and isolates direct hidden-state alignment within REG. A teacher-free SRA control has not been run and is not scheduled in the remaining discussion-period budget. The current objective-control evidence is the completed alignment-zero experiment with the semantic pathway retained.

Reviewer_sycyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces LoopREG, which enables test-time depth scaling in diffusion Transformers by repeatedly applying a shared block at a representation-supervised layer

Critical strengths
  1. Exploring test-time scaling through recurrent Transformer blocks without increasing model parameters is an interesting idea.

  2. The improvements in FID are substantial, with consistent gains across three model scales.

  3. The ablation studies are comprehensive, covering different baselines, recurrence locations, supervision positions, and representation analysis.

Critical weaknesses
  1. All experiments are restricted to ImageNet class-conditional generation. It remains unclear whether the proposed approach generalizes to other datasets and generation settings.

  2. While FID improves from 15.08 to 4.91, recall drops from 0.640 to 0.452. Do the gains primarily come from improved sample fidelity at the expense of diversity?

  3. Additional recurrent steps increase inference computation, but the paper does not compare LoopREG against baselines with equivalent computational budgets. End-to-end generation latency and training costs are also not sufficiently evaluated.

  4. The optimal inference depth is selected through an empirical FID sweep. It remains unclear whether the same depth generalizes across different models or datasets. Conducting extensive FID sweeps is impractical in real-world applications

Questions for authors

see weakness

5 comments
AuthorsOpenReview ↗

Weaknesses 1-2: Generalization and coverage (Part 1/2)

Thank you for the questions about practical compute, coverage, and depth selection. We have added direct measurements of the first two.

1. Generalization.

We build on REG and use ImageNet class-conditional generation, a shared benchmark in REG, REPA, and iREPA. This preserves a direct comparison with our baseline: within our REG/LoopREG experiments, we hold the SiT backbone, VAE representation, DINO supervision, and evaluation protocol fixed while changing recurrence. Our B/L/XL results examine this effect across model scales; the fresh L/2 evaluation reproduces FID 4.2763 to 3.2429 with one extra visit.

A controlled extension to another dataset requires preparing its data and conditioning, validating the latent/teacher-feature pipeline, training matched REG and LoopREG baselines, and establishing the evaluation reference. Moving to a different generation setting also requires adapting the conditioning architecture and baseline. Completing this work alongside the requested ablations is difficult within the short discussion period. We have completed the alignment, guidance, and measured-cost controls, and are prioritizing longer paired training and independent training seeds with the remaining compute.

After these continuation and replication studies, we aim to add a matched REG/LoopREG experiment on an additional dataset to the final manuscript, if compute and baseline adaptation permit. The planned comparison would fix the training budget and K_train=2, evaluate K_infer=2/3, and report FID, precision, and recall to test whether useful extra recurrence transfers beyond ImageNet. This extension is not yet started.

2. Fidelity and coverage.

With image CFG=2, fixed-depth LoopREG (K_train=2, K_infer=3) reaches FID 3.9672 at Recall 0.4960, 11.1% below the best FID in the measured REG guidance sweep (CFG=6: 4.4614, Recall 0.5952). Its latency is 13.14 s/256 images (mean of two session medians). REG CFG=6 improves both FID and Recall over the two unguided LoopREG anchors (5.3405/0.5076 and 4.9499/0.4465). The lowest measured REG Recall is 0.5712 at CFG=8, leaving a 0.0752 gap to guided LoopREG and 0.0636/0.1247 to the unguided anchors. These exceed the prespecified 0.01 tolerance; the results describe quality–coverage trade-offs, not matched Recall. Figure 10 (Appendix I, p. 25) plots the precision–recall and FID–Recall operating points.

The extra-depth anchors have higher Precision and lower sFID than REG CFG=6, and take 7.60/8.58 rather than 11.38 seconds per 256 images (33.21%/24.58% less time). These new-sweep times are single-session medians. Guided REG uses 461 batch-equivalent evaluations versus 250 for these unguided anchors, including conditional/unconditional processing. These are distinct quality–coverage–cost choices.

The complete REG sweep spans CFG=1, 1.5, 2, 3, 4, 6, 8; all points use 50K samples. Its Recall range is 0.5712–0.6339. The new CFG comparison also identifies a coverage-preserving recurrent option: at image CFG=1, REG obtains FID 15.26 and Recall 0.6339, while RandomLoop obtains FID 11.93 and Recall 0.6350. At CFG=2, the corresponding values are 9.73/0.6217 and 7.80/0.6344. RandomLoop requires approximately 30–31% more measured time, and its sFID is slightly worse. Thus the evidence supports distinct fidelity-focused and coverage-preserving operating points, rather than a claim that every metric improves together.

AuthorsOpenReview ↗

Weaknesses 3-4: Compute, training, and depth selection (Part 2/2)

3. Equivalent inference budget.

All new quality results below use 50K ImageNet-256 samples, EMA weights, and 250-step SDE sampling unless otherwise specified. Image CFG varies by row; the independent semantic-token CFG is fixed at 1, retaining conditional semantic predictions without extra guidance. Times are warm sampler-plus-VAE batch latencies on four H100 GPUs, 64 images per GPU. Except for the new guidance sweep, times are the mean of two session medians, each measured over five batches after two warmups. Model loading, disk I/O, and Inception evaluation are excluded.

ModelK_inferSampling stepsFID ↓RecallTime (s)
REG128915.240.63377.59
LoopREG, K_train=2226814.730.63417.58
LoopREG, K_train=232505.340.50767.61

All three configurations in this matched-time comparison use image CFG=1. The original sampling-step choices were retained for both timing sessions, with execution order reversed in the second session. The largest latency gap between either comparator and LoopREG K_infer=3 was 0.634%. At this matched-time operating point, extra recurrence lowers FID by 64.95% relative to REG with more sampling steps, and 63.75% relative to the same LoopREG checkpoint evaluated at its trained depth with more sampling steps. The corresponding Recall values are shown in the table.

The native REG/LoopREG B/2 continuation now has completed 50K evaluations at 600K for both models and at 800K for LoopREG. At 600K, REG gives FID/Recall 12.9292/0.6297; LoopREG gives 12.5706/0.6343 at K_infer=2, 4.7137/0.5227 at K_infer=3, and 5.9406/0.4490 at K_infer=4. At 800K, the LoopREG values are 11.4055/0.6394, 4.4317/0.5246, and 5.6817/0.4610. The within-checkpoint K=2-to-3 FID reduction persists at 62.50% and 61.14%, respectively. All use EMA, seed 9201, 250 SDE steps, and image/semantic CFG=1. We are continuing both native B/2 models from 400K toward 1M total steps on four H100s at global batch 256 without gradient accumulation. The remaining evaluations are REG at 800K and both models at 1M: REG K_infer=1 and LoopREG K_infer=2/3/4, with 50K images per condition. This study tests whether extra-depth gains persist with training maturity and compares REG/LoopREG quality and incremental training cost at the same milestones.

The completed results and training costs are included in the revised main text and Appendix I.7. From 400K to 600K, recorded incremental training costs are 19.29 H100 GPU-hours for REG and 20.85 for LoopREG, or 0.08683/0.09382 seconds per retained successful update. LoopREG includes 0.71 GPU-hours of recorded restart waste; these are active training totals, excluding historical 0-to-400K training, setup, checkpoint I/O, and evaluation. As each remaining milestone completes, we will post its results in this reviewer thread and update the revised main text and appendix. Before the November 18 response deadline, we will also post a progress update listing any unfinished runs, even if no additional milestone has completed. Any remaining continuation and seed-replication results will be included in the final manuscript.

The main configurations currently use one training seed. Two additional independent K_train=2 seeds are scheduled after the paired 1M evaluations; neither has started. Each will train from scratch to 400K with fixed hyperparameters, then receive predefined K_infer=2/3 evaluations of 50K images each to test sensitivity to training randomness. We will first complete one seed and its evaluations. The second will run during discussion if enough time remains for the complete run; otherwise it will follow after discussion. We will report each seed and the mean/standard deviation across the original and two new seeds, posting completed results here and in the revised main text and appendix; the completed three-seed study will enter the final manuscript.

4. Choosing inference depth in practice.

Depth selection is checkpoint-specific. A practical procedure is to calibrate a small set of depths once per checkpoint for the intended latency and coverage target, then fix the selected depth for deployment. This requires an evaluation sweep. Appendix C reproduces predetermined extra-depth comparisons on new generation seeds.

Thank you for the additional experiments. I have also read the comments from the other reviewers. I appreciate the matched-runtime comparisons and the RandomLoop results, which address part of my concerns.

However, generalization beyond ImageNet and practical inference depth selection remain open questions. The depth selection still requires a separate sweep for each checkpoint to identify an appropriate depth based on latency and coverage constraints

I also share Reviewer sSjz's concern that, compared with REG under stronger CFG, fixed-depth LoopREG achieves a lower FID but a lower recall and higher inference cost. practical advantage is unclear

I tend to maintain my current score

Reviewer_sSjzOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

LoopREG adds shared recurrence to REG. A single Transformer block is applied repeatedly at the representation-supervised layer, with DINOv2 alignment evaluated after the final training-time visit. The original joint image/semantic-token velocity objectives are retained. At inference, the same block can be applied beyond its training horizon without changing the weights or diffusion time grid. The paper reports substantial within-checkpoint FID improvements on ImageNet 256x256 across SiT-B/L/XL, studies recurrence/supervision placement, and analyzes representation geometry and image/semantic output paths. The useful inference horizon is finite, and the best FID operating points generally have lower recall.

Critical strengths
  1. The within-checkpoint depth sweeps reveal a substantial empirical effect. For B/2 trained at K=2, increasing inference depth to 3 improves FID from 14.71 to 5.34 with unchanged weights and sampler steps; beneficial extrapolation also appears at larger scales. This is a useful observation about allocating computation inside a diffusion denoiser.

  2. The supervision-site interventions are more informative than a simple baseline comparison. Holding supervision fixed and moving the loop, then moving both together, supports co-location in the tested settings. Appendix C's paired output-depth experiments further demonstrate contributions from both image and semantic updates. Reporting complete depth sweeps and the recall trade-off is also valuable.

Critical weaknesses
  1. The experiments do not yet establish an advantageous quality-diversity-compute trade-off. The headline B/2 FID improvement from REG's 15.08 to 4.91 accompanies a recall drop from 0.640 to 0.452 (Table 1). Within the K=2 checkpoint, recall falls from 0.631 to 0.508 when an extra visit improves FID. The authors acknowledge this, but acknowledgment does not establish whether recurrence offers a better operating point than existing ways of concentrating a generator's distribution. There is no guidance sweep for REG versus LoopREG, or comparison at comparable recall and measured inference cost. RandomLoop partly recovers coverage but reaches a much weaker best FID of 11.68. Moreover, the manuscript and supplement provide no generated-image examples: the quantitative curves do not show which visual defects improve or what variation is lost. These omissions materially limit the interpretation of the large headline gain.

  2. The incremental contribution over existing recurrent generation is insufficiently established. LoopREG keeps REG's objectives and adds parameter sharing at the supervised block. ELT[1], already available before May 2026 and cited in Section 2, also trains looped visual generators and reports useful inference beyond training depth. External endpoint alignment is a meaningful difference from ELT's intra-loop self-distillation, but there is no controlled comparison to establish its advantage. The plain SiT/REPA/REG controls are informative, yet do not isolate alignment's contribution within REG's joint semantic-token architecture: a recurrent REG control retaining joint denoising but removing the alignment loss is missing. The geometric correlations and scalar transient-state example help interpret the observations, but do not resolve this attribution. I therefore do not find the evidence sufficient for acceptance in its current form.

[1] Goyal S, Agrawal S, Anil G G, et al. ELT: Elastic looped transformers for visual generation[J]. arXiv preprint arXiv:2604.09168, 2026.

Questions for authors
  1. Does LoopREG improve the fidelity-coverage frontier over REG when guidance strength is varied? Please compare at comparable recall and report inference cost, accounting for any extra conditional/unconditional evaluations. A comparison with Internal Guidance[1] would also help distinguish the benefit from an existing use of internal depth information. Please include same-seed, same-class generated examples across matched, beneficial extra, and over-iterated depths, together with unselected sample grids to make the coverage loss interpretable.

  2. What does external endpoint alignment add relative to an ELT-style intra-loop self-distillation control in the same backbone/setup? Please clarify the specific new capability established beyond prior elastic/looped generation; a directly matched control would be more informative than comparing published FIDs from different training and sampling protocols.

  3. What happens when the alignment coefficient is set to zero while retaining REG's semantic-token input, output, and denoising loss? This would separate the contribution of direct hidden-state alignment from joint semantic denoising. Appendix C establishes an output-path effect for the trained model, but does not replace this training-objective control.

  4. Can the key larger-scale results be tied to complete evaluation manifests and confirmed in fresh full sampling runs? Appendix A states that L/2 metrics come from an owner summary without the original metric/configuration JSON, and that the historical resume path does not preserve all SDE random draws, with affected cells unidentified. The supplement supplies source and aggregate tables, but does not close those provenance gaps. Appendix C's fresh B/2 evaluations are useful; please distinguish their generation-seed variation from independent training-seed replication.

Clarifying these points, together with the targeted comparisons above, could change my assessment.

[1] Zhou X, Li Q, Hu X, et al. Guiding a diffusion transformer with the internal dynamics of itself[J]. arXiv preprint arXiv:2512.24176, 2025.

8 comments
AuthorsOpenReview ↗

Q3 and Q1: Alignment control and guidance sweep (Part 1/4)

Thank you for specifying the attribution control and the verification gap. We have completed both, together with controlled runtime measurements.

Q3. Removing alignment while preserving REG's semantic pathway.

We trained a 400K-step control from scratch with alignment coefficient zero while retaining REG's noisy semantic-token input, semantic output, and semantic denoising objective. The latter still uses DINO targets. Architecture, training horizon K_train=2, and the semantic-loss coefficient are preserved. The positive comparison uses the original aligned checkpoint.

Alignment coefficientK_infer=2 FID ↓K_infer=3 FID ↓K_infer=4 FID ↓
0.5, LoopREG (original sweep)14.715.346.20
0, semantic pathway retained24.7431.6851.54

The alignment-free control retains semantic supervision, yet additional recurrence worsens generation. Direct representation alignment therefore contributes beyond the retained semantic-token denoising objective in this B/2 setting. This training-objective intervention complements the submitted output-path interventions, which examine how an already trained model uses its semantic state.

Q1. Coverage, guidance, cost, and examples.

All new quality results below use 50K ImageNet-256 samples, EMA weights, and 250-step SDE sampling unless otherwise specified. Image CFG varies by row; the independent semantic-token CFG is fixed at 1, retaining conditional semantic predictions without extra guidance. The interval [0, 0.85] specifies when image guidance is active during sampling, not a range of CFG strengths. Times are warm sampler-plus-VAE batch latencies on four H100 GPUs, 64 images per GPU. Except for the new guidance sweep, times are the mean of two session medians, each measured over five batches after two warmups. Model loading, disk I/O, and Inception evaluation are excluded.

ModelK_inferImage CFGFID ↓sFID ↓PrecisionRecallTime (s)
REG1115.266.640.64780.63396.60
LoopREG, K_train=22114.746.710.64700.63447.09
LoopREG, K_train=2315.345.800.79860.50767.61
RandomLoop, K_train in {1,2,3}5111.936.800.65880.63508.56
REG129.735.870.68480.621711.38
LoopREG, K_train=2229.605.990.68430.624012.28
LoopREG, K_train=2323.975.310.82800.496013.14
RandomLoop, K_train in {1,2,3}527.806.040.69120.634414.91

The REG sweep varies image CFG over {1, 1.5, 2, 3, 4, 6, 8}; the independent semantic-token CFG stays at 1, retaining the conditional semantic prediction without extra guidance. The interval [0, 0.85] specifies when image guidance is active during sampling, not its strength. The table includes all seven REG strengths and both fixed-depth anchors, using 50K samples per point, 400K checkpoints, EMA, seed 9201, and 250 SDE steps. These times are from one new session: the median of five 256-image batches after two warmups on four H100s, including sampler and VAE and excluding loading, I/O, and metric evaluation.

ModelK_inferImage CFGFID ↓sFID ↓PrecisionRecallTime (s)
REG1115.26206.64320.647800.63396.59
REG11.512.18676.25000.664860.627111.42
REG129.72735.86760.684760.621711.39
REG136.61475.41050.709480.619811.43
REG145.09795.41550.723680.611011.40
REG164.46146.49260.720920.595211.38
REG185.12388.35150.701480.571211.41
LoopREG, K_train=2315.34055.79540.798620.50767.60
LoopREG, K_train=3514.94995.38530.851500.44658.58

With image CFG=2, fixed-depth LoopREG (K_train=2, K_infer=3) reaches FID 3.9672 at Recall 0.4960, 11.1% below the best FID in the measured REG guidance sweep (CFG=6: 4.4614, Recall 0.5952). Its latency is 13.14 s/256 images (mean of two session medians). REG CFG=6 improves both FID and Recall over the two unguided LoopREG anchors (5.3405/0.5076 and 4.9499/0.4465). The lowest measured REG Recall is 0.5712 at CFG=8, leaving a 0.0752 gap to guided LoopREG and 0.0636/0.1247 to the unguided anchors. These exceed the prespecified 0.01 tolerance; the results describe quality–coverage trade-offs, not matched Recall. Figure 10 (Appendix I, p. 25) plots the precision–recall and FID–Recall operating points.

AuthorsOpenReview ↗

Q1 continued: Compute trade-offs and generated samples (Part 2/4)

The two unguided fixed-depth LoopREG anchors (K_train/K_infer=2/3 and 3/5) have higher Precision and lower sFID than REG CFG=6, and take 7.60/8.58 rather than 11.38 seconds per 256 images (33.21%/24.58% less time). These new-sweep times are single-session medians. Guided REG uses 461 batch-equivalent evaluations versus 250 for these unguided anchors, including conditional/unconditional processing. These are distinct quality–coverage–cost choices.

At baseline-like coverage, RandomLoop at CFG=1 gives FID 11.93/Recall 0.6350 versus REG's 15.26/0.6339, with 29.74% more time. At nearly identical measured runtime, REG with 289 sampling steps gives FID 15.24 and trained-depth LoopREG with 268 steps gives 14.73, while extra recurrence with 250 steps gives 5.34. The maximum timing mismatch across the two sessions is 0.634%. Recall is approximately 0.634 for the first two and 0.508 for the latter.

Image CFG above one invokes additional conditional/unconditional processing: the 250-call sampler at CFG=2 uses 461 batch-equivalent network evaluations, included in the measured times. We have prepared unselected first-32-image grids at K_infer=2/3/4 from the same 800K checkpoint, seed, class stream, and sample indices. K=4 extends beyond the best tested depth: its FID is 5.68 versus 4.43 at K=3, while still improving on K=2. The revised appendix includes these grids. A direct Internal Guidance baseline has not been run and is not scheduled in the remaining discussion-period compute budget. That budget is assigned to completing paired continuation and independent training seeds; the present guidance evidence is the measured REG/LoopREG CFG comparison above.

AuthorsOpenReview ↗

Q2: ELT comparison and endpoint supervision (Part 3/4)

Q2. Contribution relative to ELT.

Both methods generate class-conditional ImageNet images at 256×256; ELT uses 512 sampling steps. We compare its published diffusion results with our completed 50K evaluations below. ELT values and settings come from Table 2 and Section 5.1. In ELT, N×L denotes N distinct layers repeated L times at inference; our K_train/K_infer count applications of one shared block. The prepared revised manuscript includes the L/2 comparison and the masked-model table in Appendix I.5 (Tables 16–17, p. 27).

LoopREG-L/2 reaches FID 3.2429 versus 3.16 for ELT 8N×4L (a 0.0829 difference, 2.62% higher), with about 13.26% fewer model parameters, 60% fewer generator-training image presentations, and 51.17% fewer sampler steps. This is a competitive quality–resource trade-off under the reported configurations.

Model / inference depthParameters (M)Training steps × batchSamplerImage CFGFID ↓
ELT 4N×8L271500K × 512512-step DDPM33.96
ELT 8N×4L539500K × 512512-step DDPM33.16
ELT 16N×2L1100500K × 512512-step DDPM32.83
LoopREG-B/2, K_train=2, K_infer=3139.03400K × 256250-step SDE15.3405
LoopREG-B/2, K_train=2, K_infer=3139.03400K × 256250-step SDE23.9672
LoopREG-B/2, K_train=3, K_infer=5139.03400K × 256250-step SDE14.9499
LoopREG-L/2, K_train=2, K_infer=3 (new 50K)467.53400K × 256250-step SDE13.2429
LoopREG-XL/2, K_train=2, K_infer=3 (historical 50K)685.02400K × 256250-step SDE12.8376

ELT diffusion uses width 2048 and the Stable Diffusion v1.4 VAE; LoopREG-B/2, L/2, and XL/2 use widths 768, 1024, and 1152 and sd-vae-ft-ema. Training exposure is 256M versus 102.4M image presentations, respectively: a 2.5× exposure difference, not a FLOPs ratio. Our parameter counts are computed from the actual model source, including the semantic pathway and projector and excluding DINO and VAE. Our semantic-token CFG is fixed at 1, retaining the conditional semantic prediction without extra guidance. Image CFG is a separate coefficient, active during sampling times [0, 0.85].

For LoopREG-L/2 at K_infer=3, analytical counting of all executed denoiser dense operators gives 179.59 GFLOPs/call and 44.90 TFLOPs/image over 250 calls (2 FLOPs/MAC, including the executed alignment projector; excluding VAE, normalization, softmax, activations, and other elementwise operations). For a common attention+MLP core comparison, ours is 43.798 TFLOPs/image. If ELT uses latent patch size 2 (256 tokens), standard dense attention/MLP, and two CFG branches at all 512 steps, its core is 862.02 TFLOPs/image, 19.68× ours. This ratio is conditional: ELT does not disclose its patch size or full CFG execution schedule, and these are analytical operation counts, not measured latencies. Appendix I.6 (Table 18, p. 28) gives the counting scope and patch-size sensitivity.

ELT also reports a separate masked-generation branch (BERT/MaskGIT with discrete tokens) in Table 1:

Model / inference depthParameters (M)TrainingSampling runsFID ↓
ELT-L 8N×3L101270 epochs, batch 512242.2
ELT-L 12N×2L152270 epochs, batch 512242.1
ELT-XL 7N×4L111270 epochs, batch 512242.0

Here 24 denotes sampling network runs. Section 5.1 and Table 5 specify 270 training epochs and batch 512, rather than per-row optimizer-step counts. The paper sweeps image guidance but does not list the chosen strength for each Table 1 row.

LoopREG uses frozen DINOv2 endpoint alignment and REG semantic denoising; ELT uses maximum-loop predictions to supervise intermediate loops through ILSD. Our supervision-site and alignment-loss interventions isolate the role of external alignment within REG. A same-backbone alignment-versus-ILSD experiment has not been run and is not scheduled in the remaining discussion-period GPU budget, which is allocated to paired continuation and independent training seeds.

Reviewer_5USnOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how diffusion Transformers can improve generation quality at inference by increasing the number of recurrent applications of a shared block. Building on REG's joint denoising of image and semantic tokens, the authors introduce LoopREG, placing the recurrent block at the location supervised by a pretrained visual encoder and applying alignment only to the state after the final training-time visit. At inference, the same block is applied more times while keeping the weights and diffusion sampling schedule unchanged. Experiments on ImageNet-256 with SiT-B/2, L/2, and XL/2 demonstrate generation gains beyond the training-time recurrence count, as well as a finite optimal depth when recurrence is extended further. The authors examine the relationship between useful recurrence and representation supervision through interventions that move the supervision and recurrence sites, compare SiT/REPA/REG supervision, and vary the image and semantic output depths. They also discuss random training depths and timestep-dependent inference-depth policies.

Critical strengths
  1. The substantial gains beyond training depth within the same checkpoint provide valuable empirical evidence for reusing learned computation. For SiT-B/2 trained with two recurrent visits, increasing the inference count to three reduces FID from 14.71 to 5.34 while keeping the weights and the number of denoiser calls at 250. L/2 and XL/2 also improve from 4.23 to 3.21 and from 3.38 to 2.84, respectively. Compared with best-result comparisons across models, these within-checkpoint comparisons more directly support the usefulness of the recurrent operator beyond its training depth. Appendix C reevaluates the matched and extra-visit endpoints for two training horizons using two generation seeds and 50K samples per condition, reproducing the main trends and providing replication at the sampling level.

  2. Interventions on supervision and recurrence locations provide fairly convincing support for the proposed design principle. In Section 4.3, when supervision is fixed at layer 4, only recurrence at that location yields a substantial extra-depth benefit; recurrence at the other tested sites degrades performance. Moving supervision and recurrence together restores the benefit at other depths. Together with the plain LoopSiT, LoopREPA, and LoopREG comparisons, these results support the importance of directly supervising the recurrent endpoint in the tested settings. Appendix C further selects the image and semantic output depths separately, providing targeted intervention evidence that semantic-state updates affect image generation through subsequent sampling steps. These analyses extend the contribution beyond a single best-FID configuration.

Critical weaknesses
  1. The most striking FID improvements come with a substantial loss of coverage, and a fuller comparison with existing guidance adjustments is missing. In Table 1, the REG-B/2 baseline has FID, precision, and recall of 15.08, 0.649, and 0.640, respectively, whereas the best-FID LoopREG configuration gives 4.91, 0.851, and 0.452. The train-two, infer-three configuration likewise reduces recall from approximately 0.631 to 0.508. Thus, the reported "67.41% reduction in FID" corresponds to a substantially different quality-coverage trade-off and cannot directly be interpreted as a balanced improvement across all aspects of distribution modeling. The authors explicitly acknowledge this behavior and report that RandomLoop achieves an FID of 11.68 at a recall of approximately 0.640, showing that the gains cannot be attributed entirely to reduced coverage. Nevertheless, comparing REG and LoopREG across CFG strengths and at similar recall is still needed to assess the independent value of extra recurrence relative to existing ways of adjusting quality and coverage.

  2. The practical advantage under equal compute budgets, and the incremental contribution over existing recurrent generation methods, have not been fully established. Fixing the number of denoiser calls does not fix computation: increasing B/2 from two to three recurrent visits raises block executions per call from 13 to 14, approximately 7.69%. Equal training steps across training-time recurrence counts likewise do not imply equal training FLOPs. The authors make these distinctions, but currently report mainly block-execution counts, without measured runtime, full-model FLOPs, or a direct comparison between increasing recurrence and increasing sampling steps under the same budget. Consequently, improvements in training-step counts cannot yet be interpreted as proportional savings in actual cost. In addition, ELT (Goyal et al., 2026) already studies shared recurrence for visual generation and reports gains beyond the training-time recurrence count. This paper cites that work but does not provide a direct experimental comparison. The supervision-location interventions are valuable, but the advantages over existing recurrent approaches still need to be clarified under comparable training and inference budgets.

Questions for authors
  1. How much of LoopREG's generation-quality advantage remains at comparable coverage? Could the authors provide FID, precision, and recall curves for REG and LoopREG across CFG strengths and inference recurrence counts, compare configurations with similar recall, and report their compute costs? In particular, when recall is close to the REG baseline's 0.640, how much advantage does fixed-training-depth LoopREG retain, and does RandomLoop improve this trade-off? This would affect my assessment of the practical significance of the best-FID improvement.

  2. Is extra recurrence the more effective choice under the same compute budget? Could the authors report measured inference time and full-model FLOPs for the key configurations, and compare against REG using the same extra budget to increase sampling steps? For ELT, are comparisons available at similar model scale, training budget, teacher settings, and inference cost, or can the scope of comparability of the existing results be clarified? These results would help assess the efficiency gains and the added value over existing recurrent generation methods.

5 comments
AuthorsOpenReview ↗

Q1: Guidance, coverage, and measured runtime (Part 1/2)

Thank you for distinguishing the within-checkpoint finding from coverage and compute. We now provide guidance sweeps, measured-budget comparisons, and continuation results through 800K.

Q1. Performance at comparable coverage and with guidance.

All guidance results use 400K EMA checkpoints, 50K ImageNet-256 samples, 250 SDE steps, and semantic-token CFG=1. Image CFG is listed separately. Times below are sampler-plus-VAE seconds per 256 images on four H100 GPUs, averaged over two session medians.

ModelK_inferImage CFGFID ↓sFID ↓PrecisionRecallTime (s)
REG1115.266.640.64780.63396.60
LoopREG, K_train=22114.746.710.64700.63447.09
LoopREG, K_train=2315.345.800.79860.50767.61
RandomLoop, K_train in {1,2,3}5111.936.800.65880.63508.56
REG129.735.870.68480.621711.38
LoopREG, K_train=2229.605.990.68430.624012.28
LoopREG, K_train=2323.975.310.82800.496013.14
RandomLoop, K_train in {1,2,3}527.806.040.69120.634414.91

At baseline-like Recall, trained-depth LoopREG gives a modest FID improvement. RandomLoop reduces FID by 21.84% at CFG=1 while retaining Recall 0.6350 versus REG's 0.6339, at 29.74% more inference time. With CFG=2, it reaches FID 7.80 and Recall 0.6344. Its sFID is slightly worse than REG's. Against REG CFG=1.5, RandomLoop CFG=1 gives FID 11.93 versus 12.19, Recall 0.6350 versus 0.6271, and time 8.56 versus 11.39 seconds, with slightly lower Precision.

We expanded REG image CFG to 1, 1.5, 2, 3, 4, 6, and 8. The following table reports all seven strengths and two unguided LoopREG anchors. Its times were measured together in one additional session, using five batches after two warmups; the image-guidance interval is [0, 0.85].

ModelK_inferImage CFGFID ↓sFID ↓PrecisionRecallTime (s)
REG1115.26206.64320.647800.63396.59
REG11.512.18676.25000.664860.627111.42
REG129.72735.86760.684760.621711.39
REG136.61475.41050.709480.619811.43
REG145.09795.41550.723680.611011.40
REG164.46146.49260.720920.595211.38
REG185.12388.35150.701480.571211.41
LoopREG, K_train=2315.34055.79540.798620.50767.60
LoopREG, K_train=3514.94995.38530.851500.44658.58

Guided LoopREG at K_train=2, K_infer=3 reaches FID 3.9672 with Recall 0.4960, an 11.1% lower FID than the best measured REG setting (CFG=6: 4.4614/0.5952). REG CFG=6 improves both FID and Recall over the unguided LoopREG anchors. REG's lowest measured Recall, 0.5712, remains 0.0752 above guided LoopREG. The large extra-depth gain therefore reflects a fidelity/coverage trade-off. Figure 10 in Appendix I plots the measured precision–recall and FID–Recall points.

AuthorsOpenReview ↗

Q2: Equal compute, ELT, and follow-up experiments (Part 2/2)

Q2. Equal inference budget, training cost, and ELT.

Times in this matched-budget table are sampler-plus-VAE seconds per 256 images on four H100 GPUs, averaged over two session medians.

ModelK_inferSampling stepsFID ↓RecallTime (s)
REG128915.240.63377.59
LoopREG, K_train=2226814.730.63417.58
LoopREG, K_train=232505.340.50767.61

At image CFG=1 and essentially equal measured time, extra recurrence reduces FID by 64.95% relative to REG using more sampling steps, and by 63.75% relative to trained-depth LoopREG using more steps. The maximum timing mismatch is 0.634%; the table reports the associated Recall. At 100 sampling steps, LoopREG K3 gives FID 5.70 and Recall 0.5103 in 3.19 seconds, adding a faster operating point.

The paired continuation also retains the extra-depth benefit. At 600K, REG gives FID 12.9292 and LoopREG K2/K3/K4 gives 12.5706/4.7137/5.9406. At 800K, LoopREG gives 11.4055/4.4317/5.6817. The K2-to-K3 reductions are 62.50% and 61.14%, with K3 Recall 0.5227 and 0.5246. These are 50K EMA evaluations with seed 9201, 250 SDE steps, and image/semantic CFG=1; Appendix I.7 gives all metrics.

From 400K to 600K, REG/LoopREG record 19.29/20.85 H100 GPU-hours at global batch 256. This incremental active-training cost excludes setup, checkpoint I/O, and evaluation; LoopREG includes 0.71 GPU-hours of recorded restart work. Removing that segment gives 0.08683/0.09062 seconds per update for REG/LoopREG.

Comparison with ELT. Both methods generate class-conditional ImageNet-256 images. The closest published ELT diffusion model by parameter count gives the following comparison (Goyal et al., 2026, Table 2).

Model / inference depthParameters (M)Training steps × batchSamplerImage CFGFID ↓
ELT 8N×4L539500K × 512512-step DDPM33.16
LoopREG-L/2, K_train=2, K_infer=3467.53400K × 256250-step SDE13.2429

LoopREG-L/2 reaches a similar FID with 13.26% fewer model parameters, 60% fewer generator-training image presentations, and 51.17% fewer sampling steps. ELT uses width 2048 and the Stable Diffusion v1.4 VAE; LoopREG-L/2 uses width 1024 and sd-vae-ft-ema. Image guidance and sampling settings are listed in the table.

Our L/2 denoiser requires 179.59 GFLOPs/call, or 44.90 TFLOPs/image over 250 calls (2 FLOPs/MAC; dense operators including the executed projector, excluding VAE and elementwise operations). On the common attention+MLP scope, LoopREG uses 43.798 TFLOPs/image. Assuming latent patch size 2, standard dense attention/MLP, and two CFG branches at all 512 steps for ELT gives 862.02 TFLOPs/image, a 19.68× ratio. Appendix I.6 details these assumptions and patch-size sensitivity.

The supervision differs: ELT uses maximum-loop predictions to train intermediate loops through ILSD, whereas LoopREG aligns the recurrent endpoint to frozen DINOv2 features. Our placement interventions move useful recurrence with supervision. Removing alignment while retaining REG's semantic pathway reverses the extra-loop effect: K2-to-K3 FID becomes 24.74 to 31.68, versus 14.71 to 5.34 with alignment. These interventions establish the role of directly supervising the recurrent state within REG.

Continuation and replication schedule. We are continuing both B/2 models(REG and LoopREG) from 400K to 1M total steps to assess training maturity, REG comparisons at the same milestones, and training cost. REG 800K and all 1M evaluations are pending. Two new training seeds are scheduled afterward; neither has started. Each will train K_train=2 from scratch to 400K with fixed hyperparameters and evaluate K2/K3 on 50K images each. We complete seed 1 first; seed 2 follows during discussion if a full run fits, otherwise afterward. We will report each seed and the three-seed mean/SD. Each completed milestone will update this thread and the revised main text/appendix. By November 18, we will report progress and remaining work even if unfinished; remaining continuation and replication results will enter the final manuscript.

Thank you for the additional experiments. The measured-runtime controls address an important part of my compute concern, and the RandomLoop results provide useful evidence of improved FID at approximately baseline recall. These additions strengthen the empirical contribution. I currently maintain my weak-accept recommendation.

Reviewer_oDKqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes LoopREG, a recurrent diffusion Transformer that repeatedly applies a shared block at a representation-supervised layer. The authors investigate whether applying the block more times at inference than during training improves generation. Experiments on class-conditional ImageNet show lower FID with additional recurrent visits across three SiT model scales, although the improvements are accompanied by reduced recall. The paper also examines how recurrence and supervision locations affect the results.

Critical strengths
  1. Interesting research question. The paper explores whether additional recurrent computation at inference can improve diffusion generation without increasing the number of model parameters.

  2. Strong within-checkpoint results. Increasing inference depth substantially improves FID while keeping the model weights fixed. The effect is observed across three SiT model scales.

  3. Informative analysis. The authors examine different recurrence and supervision locations and report both improvements and failure cases, including reduced recall and performance degradation at excessive depths.

Critical weaknesses
  1. The FID gains come with a substantial drop in recall. For the best B/2 result, FID improves from 15.08 to 4.91 relative to REG, but recall falls from 0.640 to 0.452. This makes the practical benefit difficult to judge without comparisons at similar recall or full precision–recall curves.

  2. The compute comparison is incomplete. The paper does not establish whether recurrent visits are a better use of additional inference compute than increasing the number of sampling steps. Comparisons at equal training steps also do not account for the extra compute required by recurrence.

  3. The proposed mechanism is not fully isolated. The location experiments are informative, but the controls do not clearly separate the effects of representation alignment, REG’s semantic pathway, and layer choice. The theoretical analysis provides intuition, but does not demonstrate why extrapolation works in the trained model.

  4. Robustness and generality remain uncertain. The main results lack multiple independent training seeds, and the best inference depths are selected through evaluation sweeps. Experiments are also limited to ImageNet, SiT backbones, and DINOv2 supervision.

  5. Some results are difficult to verify. The appendix notes that the original metric and configuration files for the L/2 results are unavailable in the evidence set, and the trained checkpoints are not included in the supplement.

Questions for authors
  1. Can the authors compare LoopREG with REG at similar recall levels, or provide precision–recall curves? This would help clarify how much of the FID improvement comes with reduced coverage.

  2. Under a matched inference compute budget, how does adding recurrent visits compare with increasing the number of sampling steps? Can the authors also report inference latency and training compute?

  3. Can the authors isolate the effect of representation alignment while keeping REG’s semantic pathway unchanged? How sensitive are the findings to the choice of supervised layer?

  4. How stable are the main results across independent training seeds? Do the authors have evidence that the selected inference depths generalize beyond the samples used to select them?

  5. Can the authors provide the original metric and configuration files for the L/2 results, along with the key trained checkpoints or another way to verify them?

4 comments
AuthorsOpenReview ↗

Q1: Guidance and coverage comparisons (Part 1/3)

Thank you for these concrete questions. We have completed new coverage comparisons, measured-time controls, an alignment ablation, and full L/2 reevaluations. We address each question below. Appendix I of the revised manuscript now includes these measurements.

Q1. Comparison at similar recall and precision–recall trade-offs.

The REG sweep varies image CFG over {1, 1.5, 2, 3, 4, 6, 8}; the independent semantic-token CFG stays at 1, retaining the conditional semantic prediction without extra guidance. The interval [0, 0.85] specifies when image guidance is active during sampling, not its strength. The table includes all seven REG strengths and both fixed-depth anchors, using 50K samples per point, 400K checkpoints, EMA, seed 9201, and 250 SDE steps. These times are from one new session: the median of five 256-image batches after two warmups on four H100s, including sampler and VAE and excluding loading, I/O, and metric evaluation.

ModelK_inferImage CFGFID ↓sFID ↓PrecisionRecallTime (s)
REG1115.26206.64320.647800.63396.59
REG11.512.18676.25000.664860.627111.42
REG129.72735.86760.684760.621711.39
REG136.61475.41050.709480.619811.43
REG145.09795.41550.723680.611011.40
REG164.46146.49260.720920.595211.38
REG185.12388.35150.701480.571211.41
LoopREG, K_train=2315.34055.79540.798620.50767.60
LoopREG, K_train=3514.94995.38530.851500.44658.58

With image CFG=2, fixed-depth LoopREG (K_train=2, K_infer=3) reaches FID 3.9672 at Recall 0.4960, 11.1% below the best FID in the measured REG guidance sweep (CFG=6: 4.4614, Recall 0.5952). Its latency is 13.14 s/256 images (mean of two session medians). REG CFG=6 improves both FID and Recall over the two unguided LoopREG anchors (5.3405/0.5076 and 4.9499/0.4465). The lowest measured REG Recall is 0.5712 at CFG=8, leaving a 0.0752 gap to guided LoopREG and 0.0636/0.1247 to the unguided anchors. These exceed the prespecified 0.01 tolerance; the results describe quality–coverage trade-offs, not matched Recall. Figure 10 (Appendix I, p. 25) plots the precision–recall and FID–Recall operating points.

The extra-depth anchors have higher Precision and lower sFID than REG CFG=6, and take 7.60/8.58 rather than 11.38 seconds per 256 images (33.21%/24.58% less time). These new-sweep times are single-session medians. Guided REG uses 461 batch-equivalent evaluations versus 250 for these unguided anchors, including conditional/unconditional processing. These are distinct quality–coverage–cost choices.

At near-baseline Recall, RandomLoop with image CFG=1 gives FID 11.93 at Recall 0.6350, compared with REG's 15.26 at 0.6339: a 21.84% FID reduction with a Recall difference of 0.0011. Precision also increases from 0.6478 to 0.6588. This is the random-depth training variant of LoopREG; the fixed-depth results are listed separately.

ModelK_inferImage CFGFID ↓sFID ↓PrecisionRecallTime (s)
REG1115.266.640.64780.63396.60
LoopREG, K_train=22114.746.710.64700.63447.09
RandomLoop, K_train in {1,2,3}5111.936.800.65880.63508.56
REG129.735.870.68480.621711.38
LoopREG, K_train=2229.605.990.68430.624012.28
RandomLoop, K_train in {1,2,3}527.806.040.69120.634414.91

All entries use B/2 checkpoints at 400K, 50K generated images, EMA, 250 SDE steps, and semantic CFG=1. Times in this near-baseline table are the mean of two session medians. At CFG=2, RandomLoop also improves FID over REG while retaining higher Recall (7.80/0.6344 versus 9.73/0.6217). Its sFID is slightly worse, and its measured inference time is about 30% higher at the same CFG. Fixed-depth LoopREG at its trained depth gives smaller FID gains at closely matched Recall.

AuthorsOpenReview ↗

Q2-Q3: Measured costs and alignment controls (Part 2/3)

Q2. Recurrent visits versus sampling steps at matched inference time.

ModelK_inferSampling stepsFID ↓RecallTime (s)
REG128915.240.63377.59
LoopREG, K_train=2226814.730.63417.58
LoopREG, K_train=232505.340.50767.61

All three configurations use image CFG=1. These are warm sampler-plus-VAE latencies for a batch of 256 images on four H100 GPUs. Each timing session uses two warmups and five measured batches; we report the mean of two session medians, reversing execution order in the second session. The largest comparator-to-target latency difference is 0.634%. At this matched time, the extra recurrent visit lowers FID by 64.95% relative to REG with more sampling steps, and 63.75% relative to the same LoopREG checkpoint at its trained depth with more sampling steps. Recall is included in the table.

At fixed 250-step sampling, K_infer=2→3 adds 7.37% measured time and reduces FID by 63.77%.

The native REG/LoopREG B/2 continuation now has completed 50K evaluations at 600K for both models and at 800K for LoopREG. At 600K, REG gives FID/Recall 12.9292/0.6297; LoopREG gives 12.5706/0.6343 at K_infer=2, 4.7137/0.5227 at K_infer=3, and 5.9406/0.4490 at K_infer=4. At 800K, the LoopREG values are 11.4055/0.6394, 4.4317/0.5246, and 5.6817/0.4610. The within-checkpoint K=2-to-3 FID reduction persists at 62.50% and 61.14%, respectively. All use EMA, seed 9201, 250 SDE steps, and image/semantic CFG=1. We are continuing both native B/2 models from 400K toward 1M total steps on four H100s at global batch 256 without gradient accumulation. The remaining evaluations are REG at 800K and both models at 1M: REG K_infer=1 and LoopREG K_infer=2/3/4, with 50K images per condition. This study tests whether extra-depth gains persist with training maturity and compares REG/LoopREG quality and incremental training cost at the same milestones.

The completed results and training costs are included in the revised main text and Appendix I.7. From 400K to 600K, recorded incremental training costs are 19.29 H100 GPU-hours for REG and 20.85 for LoopREG, or 0.08683/0.09382 seconds per retained successful update. LoopREG includes 0.71 GPU-hours of recorded restart waste; these are active training totals, excluding historical 0-to-400K training, setup, checkpoint I/O, and evaluation. As each remaining milestone completes, we will post its results in this reviewer thread and update the revised main text and appendix. Before the November 18 response deadline, we will also post a progress update listing any unfinished runs, even if no additional milestone has completed. Any remaining continuation and seed-replication results will be included in the final manuscript.

Q3. Alignment ablation and supervised-layer choice.

We trained a 400K-step control from scratch with alignment coefficient zero while retaining REG's noisy semantic-token input, semantic output, and semantic denoising objective. The latter still uses DINO targets. Architecture, training horizon K_train=2, and the semantic-loss coefficient are preserved. The positive comparison uses the original aligned checkpoint.

Alignment coefficientK_infer=2 FID ↓K_infer=3 FID ↓K_infer=4 FID ↓
0.5, LoopREG (original sweep)14.715.346.20
0, semantic pathway retained24.7431.6851.54

Removing alignment reverses the K_infer=2→3 effect: FID worsens from 24.74 to 31.68, while the original aligned LoopREG sweep improves from 14.71 to 5.34. At K_infer=4, aligned LoopREG retains FID 6.20, compared with 51.54 for the zero-alignment control. This separates direct alignment from the retained REG semantic-token denoising objective. The location experiments test layer choice: with supervision fixed at depth four, extra recurrence helps at the co-located block and hurts at blocks two, six, and eight. Moving supervision and recurrence together restores benefits at those other depths. Thus the favorable location follows supervision, rather than being intrinsic to Block4.

AuthorsOpenReview ↗

Q4-Q5: Training seeds and L/2 verification (Part 3/3)

Q4. Training seeds and validation on new generated samples.

The main configurations currently use one training seed. Two additional independent K_train=2 seeds are scheduled after the paired 1M evaluations; neither has started. Each will train from scratch to 400K with fixed hyperparameters, then receive predefined K_infer=2/3 evaluations of 50K images each to test sensitivity to training randomness. We will first complete one seed and its evaluations. The second will run during discussion if enough time remains for the complete run; otherwise it will follow after discussion. We will report each seed and the mean/standard deviation across the original and two new seeds, posting completed results here and in the revised main text and appendix; the completed three-seed study will enter the final manuscript.

For generalization beyond the samples used for depth selection, we already have new full 50K evaluations at generation seeds 2 and 3. The checkpoints and depth pairs were fixed before these evaluations:

K_trainGeneration seedTrained-depth FIDExtra-depth FIDFID reduction
2214.7089 (K_infer=2)5.3394 (K_infer=3)63.70%
2314.5669 (K_infer=2)5.2959 (K_infer=3)63.64%
3213.9237 (K_infer=3)5.2351 (K_infer=4)62.40%
3313.8477 (K_infer=3)5.3386 (K_infer=4)61.45%

The extra-depth gain persists in both new sample sets for each checkpoint, demonstrating reproducibility beyond the original generated samples.

Q5. L/2 verification and trained checkpoints.

The trained L/2 checkpoint is retained on our server and is available for sharing. We have repeated the full 50K evaluations at K_infer=2/3 using this same K_train=2 checkpoint: FID 4.2763→3.2429 (24.17% lower) and sFID 5.0138→4.4229. Both use 50K samples, seed 9201, EMA, 250 SDE steps, image/semantic CFG=1, and batch 16 per GPU. We have the complete configurations and raw logs for these new evaluations. The submitted appendix described missing historical evaluation files. The revised appendix separates those historical results from these fresh evaluations and reports their protocol and metrics. We have prepared the detailed evaluation records and configurations for the accompanying release. We will also share the key trained checkpoints, including L/2, together with the open-source code and reproduction instructions.

K_inferFID ↓sFID ↓IS ↑PrecisionRecall
24.27635.0138171.140.74680.6347
33.24294.4229263.310.85860.5137
AuthorsOpenReview ↗

Completed training replications, 1M continuation, and guidance comparisons

We have completed the two additional training seeds and the paired 1M continuation promised in our response. The full REG/LoopREG guidance grid and a same-backbone self-distillation control are also complete. We summarize the new evidence for Q1-Q4 below.

Q4. Two independent training replications with a preset inference depth.

Both new SiT-B/2 runs were trained from scratch to 400K with K_train=2 and fixed hyperparameters. The K_infer=2/3 pair was specified before training and evaluation. All entries use 50K samples, EMA, generation seed 9201, 250 SDE steps, and image/semantic CFG=1.

Training runFID K2FID K3FID reductionPrecision K2/K3Recall K2/K3
Original, seed 014.74245.340563.77%0.64696/0.798620.6344/0.5076
New seed 114.26795.674860.23%0.65460/0.795300.6320/0.5133
New seed 213.56274.996163.16%0.66530/0.809240.6256/0.5067
Mean ± sample SD14.1910 ± 0.59365.3372 ± 0.339462.39% ± 1.90 pp——

Both new runs reproduce the large FID improvement and increased Precision at the preset K3, with the same coverage trade-off. The result therefore extends beyond the original training run and the samples used to select its depth.

Q2. Completed 1M continuation and measured training cost.

At 1M, paired REG gives FID/Recall 10.6872/0.6364. LoopREG gives 10.4860/0.6407 at K2 and 4.4027/0.5355 at K3, a 58.01% within-checkpoint FID reduction. These evaluations use the same 50K protocol above. Recorded active training costs from 400K to 1M are 63.11 H100 GPU-hours for REG and 65.68 for LoopREG (+4.06%), at global batch 256. These include logged restart work and exclude pre-400K training, setup, checkpoint I/O, and evaluation.

Q3. Endpoint alignment versus same-backbone output self-distillation.

We trained an ELT-style output-ILSD control adapted to REG on SiT-B/2, using seed 1, 400K updates and training depth 2. It retains REG's semantic denoising objective, replaces direct patch alignment with distillation from the maximum-depth output to the intermediate output, and uses the same evaluation protocol as aligned seed 1.

Training objectiveFID K2Recall K2FID K3Recall K3
Output ILSD adapted to REG26.25660.629927.03300.6403
LoopREG endpoint alignment14.26790.63205.67480.5133

At K2, alignment lowers FID by 45.66% at nearly identical Recall. With the extra visit, its FID improves by 60.23%, while the ILSD control worsens by 2.96%. Together with the retained-semantic-pathway alignment-zero control, this comparison strengthens the evidence that the supervision objective matters for useful recurrence.

Q1. Completed symmetric guidance sweep.

REG and LoopREG K2/K3 have now been evaluated at image CFG 1, 1.5, 2, 3, 4, 6 and 8, with 50K samples per point and all 21 latencies remeasured under one protocol. At the original 400K checkpoint, LoopREG K3/CFG3 reaches FID 3.7811, Precision 0.83944 and Recall 0.4853 in 13.20 s/256 images. REG's best-FID point, CFG6, gives 4.4614/0.72092/0.5952 in 11.43 s. This is 15.25% lower FID and higher Precision with 15.48% more time and lower Recall. At closely matched Recall, K2/CFG6 gives FID 4.4867 and Recall 0.5955 in 12.31 s. The full grid therefore identifies the fidelity-oriented K3 operating points and the higher-coverage K2 operating points; K3's Recall remains below the measured REG range.

We will incorporate these completed replications, continuation costs, control results and guidance curves into the revised main text and appendix.

If you have any remaining concerns or questions, please let us know, and we will do our utmost to address them.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.