acceptodds
Under review as a conference paper at ICLR 2027

RELEVANCE-GUIDED TOKEN SUPERVISION ALLOCATION FOR DIFFUSION REPRESENTATION ALIGNMENT

Abstract

Recent work has shown that aligning intermediate diffusion features with pretrained visual representations can improve diffusion model training. However, existing methods typically use teacher representations to define alignment targets while allocating supervision uniformly, regardless of image content. To address this limitation, We propose Relevance-Guided Token Supervision Allocation (RGSA), which uses teacher-derived relevance cues to replace uniform aggregation of token-wise alignment losses with image-conditioned weighting.Specifically, RGSA derives proxy relevance cues from frozen teacher representations using patch-feature norms to characterize response strength and patch–CLS cosine similarity to capture local–global semantic consistency. These cues are fused into token relevance scores and normalized within each image to assign relative weights to the corresponding token-wise alignment losses. Experiments on ImageNet-1K show consistent improvements over iREPA across SiT-B/2, SiT-L/2, and SiT-XL/2, with similar gains across diverse visual teachers. Controlled ablations further show that RGSA outperforms random and spatially shuffled weighting in both FID and Recall. Together, these results highlight relevance-guided supervision allocation as an effective design choice for diffusion representation alignment.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
diffusion models, representation alignment, token-level supervision, visual representation learning, image generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_EdUTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Relevance-Guided Token Supervision Allocation (RGSA) to improve representation alignment in diffusion models. Built upon iREPA, RGSA replaces uniform token-wise loss aggregation with image-conditioned weights derived from frozen teacher representations. Specifically, it combines patch-feature norms and patch–CLS cosine similarity to estimate token relevance and adjusts the contribution of each token to the alignment loss. Experiments on ImageNet demonstrate improvements over iREPA across different model scales and visual teachers, with additional ablations supporting the effectiveness of the proposed weighting strategy.

Critical strengths
  1. The proposed method is simple and computationally efficient, requiring no additional learnable parameters or inference-time overhead. The reported experiments demonstrate consistent improvements over iREPA across different model scales and visual teachers.

  2. The paper provides relatively comprehensive experimental evaluations, including comparisons with alternative weighting strategies, ablation studies, and token-level alignment analyses. These experiments provide reasonable empirical support for the effectiveness of relevance-guided supervision.

Critical weaknesses
  1. Limited methodological novelty. My primary concern is that the technical contribution is relatively incremental. The proposed method essentially modifies the uniform token-wise loss aggregation in iREPA by introducing manually designed relevance weights. Both relevance signals, namely patch-feature norms and patch–CLS cosine similarity, are straightforward feature statistics, and their fusion involves only a simple weighted combination and normalization. While the resulting improvements are encouraging, the method does not introduce a substantially new representation alignment mechanism or optimization framework. Therefore, I find the methodological contribution insufficiently novel for ICLR.

  2. Limited significance of the empirical improvements. Although RGSA consistently improves over iREPA, the gains become quite marginal under longer training schedules. For example, at 400K training steps on SiT-XL/2, the FID improvement is only from 7.52 to 7.47 without CFG and from 1.93 to 1.91 with CFG. Moreover, the evaluation is primarily limited to ImageNet class-conditional generation, and the paper does not report variability across independently trained checkpoints. Given the simplicity of the proposed modification, stronger empirical evidence would be needed to establish its broader significance.

Questions for authors
  1. Beyond introducing heuristic token-wise weights into the existing iREPA objective, what is the main methodological novelty of RGSA compared with conventional spatially weighted feature distillation approaches?

  2. Could the authors provide results from multiple independent training runs, particularly for the 400K-step setting, to demonstrate that the relatively small improvements are statistically reliable?

4 comments
AuthorsOpenReview ↗

Response to Reviewer EdUT (Part 1 of 2)

We thank the reviewer for recognizing the efficiency of our method and the experimental analysis, and for pointing out the need to clarify our methodological contribution and long-training results. We have submitted the revised paper, with changes marked in blue.

Q1. What does RGSA contribute beyond conventional spatially weighted distillation?

FGD [1] uses foreground and background information from the detection task for spatial weighting. We focus on a question that has received relatively little attention in isolation in token-wise representation alignment for diffusion models. Once the teacher–student token correspondence is fixed, how much alignment supervision should each spatial token receive? To address this question, we propose RGSA, which uses patch feature norms and patch–CLS cosine similarities from a frozen teacher to compute per-image normalized weights without additional learnable parameters. The Related Work section of the revised paper now discusses how RGSA differs from conventional spatially weighted distillation.

We hold iREPA’s teacher, projector, token correspondence, per-token alignment loss, and global alignment coefficient fixed and replace only uniform aggregation with RGSA weighting. FID decreases from 110.87 to 108.92, and Recall increases from 0.4327 to 0.4526 (Table 5 of the original submission). The Shuffled control preserves the full set of weight values for each image and only permutes their spatial assignments. FID then rises to 112.58, and Recall drops to 0.4358. This shows that, in the tested setting, spatial allocation still affects generation performance even when the weight distribution is identical.

RGSA provides a simple baseline for studying spatial supervision allocation as an independent design dimension and for comparing and optimizing different allocation strategies.

Q2. Do the 400K-step improvements hold across independent training runs?

We added two independent training seeds to the comparison of iREPA and RGSA with SiT-XL/2 on ImageNet-1K. Table R1 shows that RGSA achieves lower FID and higher IS across all three seeds, both with and without CFG. With CFG, mean IS increases by 6.18 points (269.15 → 275.33). At 400K steps, the direction of the FID improvement is consistent across the tested seeds.

Table R1. Independent training results for SiT-XL/2 trained for 400K steps on ImageNet-1K. These results have been added to “Robustness to Training Randomness” (Appendix G, Table 13).

Sampling settingTraining seediREPA FID ↓RGSA FID ↓iREPA IS ↑RGSA IS ↑
No CFG07.527.47128.60129.49
No CFG17.687.34127.21128.73
No CFG27.497.42128.56129.33
No CFGMean ± SD7.56 ± 0.107.41 ± 0.07128.12 ± 0.79129.18 ± 0.40
With CFG01.931.91268.80275.15
With CFG11.941.89269.73277.56
With CFG21.961.92268.91273.27
With CFGMean ± SD1.94 ± 0.021.91 ± 0.02269.15 ± 0.51275.33 ± 2.15
AuthorsOpenReview ↗

Response to Reviewer EdUT (Part 2 of 2)

Q3. Generalization across datasets and generator architectures

Tables 3 and 4 of the original submission compare different visual teachers and SiT model sizes, respectively. RGSA achieves lower FID than iREPA in each of the tested settings.

To further assess RGSA across datasets and generator architectures, we added two comparisons with iREPA at a resolution of 256×256256\times256. Within each experiment, both methods use the same number of training steps and the same sampling settings.

Table R2 shows that RGSA reduces FID from 16.02 to 14.50 with SiT-B/2 on AFHQv2 and from 60.74 to 58.95 with JiT-B/16 on ImageNet-1K. Both results have been added to Section 4.1 of the revised paper.

Table R2. Experimental results across datasets and architectures. These results have been added to Section 4.1 of the paper.

Model and datasetMethodFID ↓sFID ↓IS ↑Precision ↑Recall ↑
SiT-B/2 · AFHQv2 256×256iREPA16.02105.1211.000.780.28
SiT-B/2 · AFHQv2 256×256RGSA14.50104.6311.100.790.30
JiT-B/16 · ImageNet-1K 256×256iREPA60.7419.2322.240.350.56
JiT-B/16 · ImageNet-1K 256×256RGSA58.9518.8924.100.360.56

References

  • [1] Yang et al. Focal and Global Knowledge Distillation for Detectors. CVPR 2022.
  • [2] Karras et al. Alias-Free Generative Adversarial Networks. NeurIPS 2021.
  • [3] Li and He. Back to Basics: Let Denoising Generative Models Denoise. CVPR 2026.

Thank you for the detailed rebuttal and additional experiments.

The new results across multiple training seeds, datasets, and architectures provide stronger evidence for the effectiveness and robustness of RGSA. I also appreciate the clarification regarding its differences from existing spatially weighted distillation methods.

However, my main concern remains the limited methodological novelty. RGSA essentially introduces heuristic relevance weights into the existing iREPA objective, and I am not fully convinced that this represents a sufficiently substantial technical contribution, especially given the relatively modest improvements under longer training schedules.

Overall, the rebuttal has addressed most of my empirical concerns. I am open to reconsidering my initial rating after further discussion with the other reviewers.

AuthorsOpenReview ↗

Response to Reviewer EdUT

Thank you for considering the additional evidence and for your willingness to reconsider your assessment during the reviewer discussion.

Reviewer_xrvwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes RGSA, a method for assigning different weights to tokens when aligning diffusion model features with a frozen visual teacher. It combines two teacher-derived signals: patch-feature L2 norms and cosine similarity between patch and CLS features. The scores are normalized in each image to weight the token-wise cosine losses. Built on iREPA, the method retains its alignment targets, projector, and model architecture, without adding learnable parameters or inference cost. Experiments on ImageNet-1K cover three model parameter sizes and three visual teachers. The reported improvements are more pronounced at 100K training steps, while the FID advantage becomes small at 400K steps. Ablation studies examine the choice of weighting signals, their spatial correspondence, and normalization.

Critical strengths
  • Negligible additional computational cost. RGSA reuses teacher features already computed during the pipeline of iREPA, and adds only simple scoring and aggregation operations. Table 8 reports a difference of less than 0.2% in training time and an increase of 0.01 GiB in peak memory. It introduces no additional model training or inference cost, making it an inexpensive modification to the existing iREPA training pipeline.
  • Thoughtful ablations of the weighting method. Table 5 examines alternative weighting signals, spatial correspondence, and normalization. The shuffled-weight control is particularly useful: it preserves each image's set of scores while breaking their correspondence with spatial positions. Appendix C and appendix F provide additional checks on the reported gains. These experiments help establish which aspects of the proposed approach matter.
Critical weaknesses
  • Limited methodological advance, with small FID gains after longer training. The contribution is a heuristic reweighting of an existing alignment loss, so its significance depends largely on the empirical results. The 100K-step results are encouraging, but the FID gains at 400K steps are small: Table 2 reports 7.52 to 7.47 without classifier-free guidance and 1.93 to 1.91 with guidance. Given the limited change to the method and the small FID gains after longer training, I am not yet convinced that the results establish a sufficiently strong advance over iREPA.
  • The comparisons do not establish the advantage of the specific weighting rule. Table 5 includes several useful alternatives, including a baseline that uses the teacher's CLS-to-patch attention as weights. However, the comparison with this baseline changes both the source of the scores and how concentrated the resulting weights are. In Figure 4, the top 20% of tokens receive approximately 21%-23% of the total weight under the RGSA allocations, compared with about 70% under CLS attention. The poorer performance of the CLS-attention baseline could therefore reflect excessive concentration, rather than an inferior source of relevance information. A stronger comparison would test whether another image-dependent signal, with a similarly mild weighting distribution, can achieve comparable gains. Without such evidence, it remains unclear how much the proposed combination of feature norms and patch-CLS similarity contributes beyond using a mild, image-dependent weighting scheme.
Questions for authors
  • How stable are the gains across independent training runs? Could the authors repeat iREPA and RGSA with different training seeds in a representative ImageNet-1K setting and report the mean and variability of FID and Recall? If intermediate checkpoints from the longer runs are available, FID curves over training would also help clarify when the advantage emerges and how long it persists.
  • What additional evidence supports the proposed weighting signals? One informative control would be to mix CLS-attention weights with uniform weights until their concentration is comparable to RGSA, for example by matching the total weight assigned to the top 20% of tokens (As mentioned in Weakness 2). The central question is whether the gains depend on the proposed RGSA or can be obtained with another weighting signal once its concentration is appropriately adjusted. Other controlled comparisons, or a theoretical justification would also be helpful to clarify the reasonability of RGSA.
4 comments
AuthorsOpenReview ↗

Response to Reviewer xrvw (Part 1 of 2)

We thank the reviewer for recognizing RGSA’s implementation efficiency and controlled ablations and for suggesting further tests of long-training stability and the effect of weight concentration. We have submitted the revised paper, with changes marked in blue.

Q1. Do the gains hold across training seeds and persist through 400K steps?

We added two independent training seeds, bringing the 400K-step SiT-XL/2 comparison on ImageNet-1K to three seeds. Table R1 shows that, with and without CFG, RGSA achieves lower FID and higher Recall and IS across all three seeds. With CFG, mean IS increases by 6.18. Table R2 shows that, without CFG, RGSA achieves lower FID than iREPA at all four checkpoints from 100K to 400K steps.

Table R1. Independent 400K-step training runs for SiT-XL/2 on ImageNet-1K. Summary values are the mean ± sample standard deviation. These results have been added to “Robustness to Training Randomness” (Appendix G, Table 13).

Sampling settingTraining seediREPA FID ↓RGSA FID ↓iREPA IS ↑RGSA IS ↑iREPA Recall ↑RGSA Recall ↑
No CFG07.527.47128.60129.490.6500.657
No CFG17.687.34127.21128.730.6480.660
No CFG27.497.42128.56129.330.6530.661
No CFGMean ± SD7.56 ± 0.107.41 ± 0.07128.12 ± 0.79129.18 ± 0.400.650 ± 0.0030.659 ± 0.002
With CFG01.931.91268.80275.150.6000.601
With CFG11.941.89269.73277.560.5900.592
With CFG21.961.92268.91273.270.6040.610
With CFGMean ± SD1.94 ± 0.021.91 ± 0.02269.15 ± 0.51275.33 ± 2.150.598 ± 0.0070.601 ± 0.009

Table R2. FID across training checkpoints for SiT-XL/2 on ImageNet-1K (without CFG).

Training stepsREPA FID ↓iREPA FID ↓RGSA FID ↓
100K19.3016.9015.79
200K11.25110.2589.69
300K8.8558.4088.21
400K7.907.527.47
AuthorsOpenReview ↗

Response to Reviewer xrvw (Part 2 of 2)

Q2. Can other weighting signals achieve comparable gains at similar weight concentrations?

As suggested by the reviewer, we mix normalized CLS attention with uniform weights (β≈0.020\beta\approx0.020). On a sample of ImageNet images, the mean Top-20% allocation mass is 21.3245% for CLS + Uniform and 21.2945% for RGSA. In Table R3, RGSA achieves lower FID (108.92 vs. 111.91) and higher Recall (0.4526 vs. 0.3300).

Shuffled preserves the full set of RGSA weights for each image and only permutes their spatial assignments. After shuffling, FID increases from 108.92 to 112.58, and Recall decreases from 0.4526 to 0.4358 (Table R3). This shows that spatial allocation still affects generation performance even when the weight distribution is identical. Figure 4(b) of the revised paper now includes weight statistics for Shuffled and Spatial Random.

Table R3. Comparison of RGSA, spatial shuffling, and spatially random weighting.

MethodGini coefficientTop-20% allocation massFID ↓Recall ↑
RGSA0.02184321.2945%108.920.4526
Shuffled0.02184321.2945%112.580.4358
Spatial Random0.33270035.8467%110.720.4230
CLS + Uniform0.01351021.3245%111.910.3300

Thank you for the thoughtful rebuttal and the additional experiments. The multi-seed results and controlled weighting comparisons help address my concerns. However, the FID gains after longer training remain modest, so my concern about the overall significance is not fully resolved. I therefore maintain my original score.

I also noticed a possible typo: RGSA’s Top-20% allocation mass is reported as 21.2945% in the text but 20.9192% in Table R3.

AuthorsOpenReview ↗

Response to Reviewer xrvw

Thank you for your feedback and for pointing out the discrepancy between the text and Table R3. Using the calculation described in the text, the value for RGSA in Table R3 should be 21.2945%.

Reviewer_pj7GOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how token-wise representation-alignment losses should be aggregated during diffusion training, the proposed RGSA builds directly on iREPA and keeps the teacher targets, spatial correspondence, projector, cosine loss, and generator architecture unchanged. It replaces uniform averaging with image-dependent token weights derived from two frozen-teacher statistics: patch-feature magnitude and patch-to-CLS cosine similarity. The fused scores are normalized within each image, so the method reallocates a fixed amount of alignment supervision without changing its overall scale. Experiments on ImageNet-1K cover three SiT capacities and three visual teachers, with controlled ablations on ImageNet-100. Notably, RGSA introduces no learned parameters and negligible measured training overhead.

Critical strengths
  1. The proposed method is well-motivated, technically sound, and easy to implement. The proposed RGSA modifies only the aggregation of token-wise losses. The alignment targets, projector, criterion, architecture, and training budget are held fixed. The shuffled control is particularly useful because it preserves the per-image score distribution while breaking spatial correspondence.

  2. Consistent results across model and teacher choices. At 100K steps, RGSA improves iREPA across SiT-B/2, L/2, and XL/2 and across DINOv2, CLIP, and MoCo-v3 teachers. This breadth reduces the likelihood that the gain comes from one particular backbone-teacher pairing.

  3. Useful ablations with negligible system cost. The cue, shuffling, normalization, alignment-weight, and sampling analyses cover the main design choices.

Critical weaknesses
  1. For evaluation, the long-horizon evidence is not yet convincing. At the longest reported budget of 400K steps, FID improves only from 7.52 to 7.47 without CFG and from 1.93 to 1.91 with CFG; the remaining metrics show relatively small or mixed changes. There is no substantially longer or demonstrably converged comparison (e.g., training for 4M steps at 800 epochs), and Table 12 varies sampling seeds from one checkpoint rather than repeating training. The current evidence therefore supports early-training acceleration more clearly than a reliable final-performance improvement.

  2. The methodological novelty is limited. RGSA applies a fixed heuristic weighting rule to the existing iREPA token-wise objective, using two standard teacher-feature statistics. It introduces neither a learned allocation mechanism nor a new alignment objective, and the analysis does not explain why this particular fusion should be preferable to other importance estimators.

  3. The empirical comparison and generalization study are not sufficiently systematic. Beyond REPA/iREPA, the paper does not provide matched comparisons with the related UniRA, sREPA, or MaskAlign methods discussed in Section 5, nor with recent alternatives such as SRA, Self-Flow, and Diffuse and Disperse. If these methods are not directly comparable, the technical differences should be explained. All experiments remain within class-conditional ImageNet latent SiT training, leaving generalization across datasets, generator architectures, and text-conditioned settings unclear.

  4. Generalization: Does the benefit persist beyond ImageNet class-conditional SiT, for example, on another dataset resolution (ImageNet 512x512), generator architecture (MeanFlow/JiT), or text-conditioned model (MS-COCO)?

Questions for authors

See the weakness, plz. For current version, the method is a relatively simple heuristic reweighting of iREPA, the 400K FID gains are only 0.05/0.02, and the experiments do not establish competitiveness against related methods or generalization beyond one ImageNet/SiT setting. These issues currently limit both the novelty and the strength of the empirical conclusion.

4 comments
AuthorsOpenReview ↗

Response to Reviewer pj7G (Part 1 of 2)

We thank the reviewer for recognizing our motivation, controlled ablations, and implementation efficiency. We have submitted the revised paper with changes marked in blue.

Q1. Long training and stability across seeds

Original Table 12 assesses sampling randomness alone. We added two independent training seeds, bringing the 400K-step SiT-XL/2 comparison on ImageNet-1K to three seeds. In Table R1, RGSA achieves lower FID and higher IS than iREPA for every seed, with and without CFG. With CFG, mean IS rises by 6.18 (269.15 → 275.33). Given the time and compute available for this rebuttal, our conclusions are limited to the tested training budget.

Table R1. Independent 400K-step SiT-XL/2 runs on ImageNet-1K. Added to “Robustness to Training Randomness” (Appendix G, Table 13).

SettingSeediREPA FID ↓RGSA FID ↓iREPA IS ↑RGSA IS ↑
No CFG07.527.47128.60129.49
No CFG17.687.34127.21128.73
No CFG27.497.42128.56129.33
No CFGMean ± SD7.56 ± 0.107.41 ± 0.07128.12 ± 0.79129.18 ± 0.40
With CFG01.931.91268.80275.15
With CFG11.941.89269.73277.56
With CFG21.961.92268.91273.27
With CFGMean ± SD1.94 ± 0.021.91 ± 0.02269.15 ± 0.51275.33 ± 2.15

Q2. Methodological novelty and contribution

We study a question rarely examined in isolation in diffusion representation alignment. Once teacher–student token correspondence is fixed, does spatial supervision allocation still affect generation? We hold iREPA’s alignment target, token correspondence, projector, per-token loss, and generator architecture fixed and replace uniform 1/N1/N aggregation with per-image normalized weights from frozen teacher features. This adds no learnable parameters.

The Shuffled control in original Table 5 retains each image’s weight values and permutes their assignment to spatial tokens. FID rises from 108.92 to 112.58 and Recall drops from 0.4526 to 0.4358, showing that spatial assignment affects generation even with identical weight distributions.

Q3. Why use this fusion rule?

Patch norms reflect local response strength, and patch–CLS cosine similarity measures consistency with the image-level representation. The mean Spearman correlation of 0.084 (original Figure 4) indicates weak agreement between their rankings. Our weighted sum (α=0.1\alpha=0.1) emphasizes norms and lets cosine similarity adjust the ranking of tokens with similar local responses.

In original Table 5, fusion raises Recall from 0.4175 to 0.4526 over norm-only weighting, with FID rising slightly from 108.65 to 108.92. Under the same settings, fusion also yields lower FID and higher Recall than CLS attention and loss-guided weighting.

AuthorsOpenReview ↗

Response to Reviewer pj7G (Part 2 of 2)

Q4. Comparison with related methods

sREPA, MaskAlign, and RGSA use external visual teachers. SRA and Self-Flow use internal representations for supervision, and Diffuse and Disperse uses dispersion regularization. Even with matched backbones and training budgets, performance differences may reflect the supervision source or loss formulation.

Table R2. SiT-XL/2 on ImageNet-1K (256×256), approximately 80 epochs, with CFG and DINOv2 as the external visual teacher for all methods.

MethodFID ↓sFID ↓IS ↑Pre. ↑Rec. ↑
REPA1.984.60263.00.800.61
iREPA1.934.59268.80.800.60
REPA + MaskAlign1.924.59270.10.810.60
sREPA2.254.65257.50.830.58
RGSA (Ours)1.914.52275.150.8130.60

We found no official public implementations of sREPA or MaskAlign. Their 80-epoch results come from the original papers and use the reported CFG and sampling settings.

Q5. Generalization across datasets and architectures

Original Tables 3 and 4 cover visual teachers and SiT model sizes. We added two 256×256256\times256 comparisons with matched training steps and sampling settings within each setting. Relative to iREPA, RGSA lowers FID from 16.02 to 14.50 on AFHQv2 (SiT-B/2) and from 60.74 to 58.95 on ImageNet-1K (JiT-B/16) (Table R3, revised Section 4.1).

Table R3. Cross-dataset and cross-architecture settings (revised Section 4.1).

Model and datasetMethodFID ↓sFID ↓IS ↑Precision ↑Recall ↑
SiT-B/2 · AFHQv2 256×256iREPA16.02105.1211.000.780.28
SiT-B/2 · AFHQv2 256×256RGSA14.50104.6311.100.790.30
JiT-B/16 · ImageNet-1K 256×256iREPA60.7419.2322.240.350.56
JiT-B/16 · ImageNet-1K 256×256RGSA58.9518.8924.100.360.56

References

  • [1] Choi et al. StarGAN v2: Diverse Image Synthesis for Multiple Domains. CVPR 2020.
  • [2] Karras et al. Alias-Free Generative Adversarial Networks. NeurIPS 2021.
  • [3] Xu et al. Beyond Point-Wise Matching: Structural Representation Alignment for Accelerating Diffusion Transformers. arXiv:2605.16949.
  • [4] Pang et al. MaskAlign: Token-Subset Representation Alignment for Efficient Diffusion Training. arXiv:2606.08788.
  • [5] Jiang et al. Representation Alignment for Diffusion Transformers without External Components. ICLR 2026.
  • [6] Chefer et al. Self-Supervised Flow Matching for Scalable Multi-Modal Synthesis. ICML 2026.
  • [7] Wang and He. Diffuse and Disperse: Image Generation with Representation Regularization. arXiv:2506.09027.
  • [8] Li and He. Back to Basics: Let Denoising Generative Models Denoise. CVPR 2026.
Reviewer_pj7GOpenReview ↗

Official Comment by Reviewer pj7G

Thanks for the detailed rebuttal, most of my concerns were addressed. However, the added generalization experiment on AFHQ and ImageNet-1K seems unconvincing to me, as AFHQ is an unconditional and easy dataset. It would be better to include more evaluation on datasets with higher resolution (e.g., ImageNet 512x512), generator architecture (MeanFlow/JiT), or simple text-conditioned generation tasks on MS-COCO. Therefore, I maintain my current rating.

AuthorsOpenReview ↗

Response to Reviewer pj7G

Thank you for the follow-up. We agree that evaluations at higher resolutions and on text-conditioned generation would provide stronger evidence of generalization. We would like to clarify the settings and results of two experiments.

Section 4.1 of the revised paper includes a JiT-B/16 comparison on ImageNet-1K at 256×256256\times256. We trained iREPA and RGSA for 100K steps each and evaluated them using the same sampling settings. Compared with iREPA, RGSA reduces FID from 60.74 to 58.95 and increases IS from 22.24 to 24.10. These results support the applicability of RGSA to JiT-B/16 at the tested resolution. Tables 3 and 4 also show that RGSA achieves lower FID than iREPA across the tested visual teachers and SiT model sizes, respectively.

The AFHQv2 experiment conditions generation on three class labels (cat, dog, and wild). We agree that this is a relatively simple task and treat its results as supplementary evidence of cross-dataset generalization.

Within the rebuttal period and our compute budget of one RTX 3090, we prioritized the JiT-B/16 and AFHQv2 comparisons. We plan to further evaluate RGSA on ImageNet-1K at 512×512512\times512 and on text-conditioned generation with MS-COCO.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.