acceptodds
Under review as a conference paper at ICLR 2027

SALD: Self-Referenced Advantage Learning for Diffusion Models

Abstract

Recent work on language-model adaptation has shown that single models can obtain informative training signals by evaluating their behavior in demonstrationor feedback-augmented contexts, with the help of a teacher network, which is driven by the students’ learned parameters. Inspired by this internal-reference principle, we investigate how diffusion models can identify self-referenced training signals without external demonstrations or teacher networks. We introduce SALD, a self-referenced training framework that evaluates each image-caption pair at two noise levels using the same model. The easier, lower-noise path is evaluated without gradient tracking to provide a reference, while the harder, higher-noise path provides the training gradient. Rather than directly distilling the easy-path prediction, SALD uses the difference between two path errors to adapt the hardpath objective. The proposed Advantage-Guided Diffusion (AGD) converts this relative error into a differentiable sample-level weight. Temporal Advantage Memory (TAM) accumulates relative difficulty across training and adapts the future gap between the two noise levels. Spectral Advantage Decomposition (SAD) further compares the residual power spectra of the two paths and constructs a differentiable, frequency-derived latent-element weight. All components share a single set of model parameters, requiring neither an external teacher network nor additional trainable parameters during training or inference, and no modification to the inference procedure. Experiments across multiple architectures and datasets demonstrate consistent improvements in generation quality, while component-wise ablations quantify the contributions of the proposed components.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Diffusion Models, Self-Referenced Learning, Teacher-Free Training, Generative Modeling

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_LigDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies self-referenced training for text-to-image diffusion models. It uses the model's own predictions to create training signals that (1) need no external teacher, demonstration dataset or architectural change, (2) adapt to each sample's difficulty over training history, and (3) adapt to the spectral structure of the model's prediction errors. Each image–caption pair is evaluated at two noise levels with shared weights. The low-noise (easy) path is computed without gradients and serves as a reference, and the high-noise (hard) path provides the training gradient.

Main contributions:

  • Advantage-Guided Diffusion (AGD): uses the difference between the prediction error at high noise and at low noise to compute a normalized mini-batch weight. This promotes the gradient of harder samples and down-weights easier ones. There is no direct distillation from easy to hard predictions.
  • Temporal Advantage Memory (TAM): an EMA-based per-sample curriculum that sets the gap between the low and high noise levels from the sample's difficulty across training history.
  • Spectral Advantage Decomposition (SAD): a frequency-derived spatial weight map applied to the velocity residual during loss computation.

Experiments report improvements over the base diffusion models and over training-acceleration/distillation baselines across backbones, resolutions, datasets and cross-dataset generalization.

Critical strengths

S1. The advantage-based idea is intuitive and well motivated. It brings training-history curricula, spectral bias, and sample-wise and noise-wise weighting into one framework. It is training-only, adds no parameters, and leaves inference unchanged.

S2. The experiments are broad: multiple backbones, resolutions and datasets, a wall-clock-matched comparison, and a longer-training (100k-step) comparison that rules out simple under-training of the baseline. The ablations are well designed and easy to read.

Critical weaknesses

W1. The motivation for SAD is inadequately discussed. It is unclear whether the term sg(P_e,i) in Eq. 13 is meaningful. Because of the coarse-to-fine nature of diffusion, the same frequency pattern may appear in the high-noise residual regardless of the easy counterpart. There is no visualization of M_i (Eq. 15) beyond Fig. 2, where the weight map looks similar to both the easy and hard residual maps. The gate is also positive at all frequencies and is mapped back with an inverse FFT that discards phase, so it is unclear how M_i encodes "which frequencies are hard". The paper itself acknowledges that M_i is not a localized map.

W2. The contribution of the self-referenced contrast is not isolated. In the ablation, "Abs. loss (no contrast)" already recovers most of the gain over uniform weighting (e.g., COCO FID 31.59 → 29.64, versus 29.31 for full SALD), so generic hard-example weighting may explain most of the improvement. This reading depends on Q1: the ablation appears to use |l_a| (the easy-path loss), not |l_b|, so it should be clarified or re-run. Differences are small, from single runs, and have no variance estimates.

W3. The advantage may reflect timestep and noise draw more than sample difficulty. The two paths use different timesteps and independent noise, so l_h − l_e is dominated by the noise-level gap and sampling variance. No evidence shows that A_i tracks per-pair difficulty, or that TAM's memory is populated, since samples may be seen only once on large datasets within 10k steps. Because the weights are not detached, the objective also differs from plain reweighting, and Theorem 1 is a generic SGD result that does not speak to SALD specifically.

W4. The baselines only partly match the claim. The compared methods are mostly training-acceleration or distillation approaches. Difficulty-aware weighting and curriculum methods that the introduction itself discusses (Min-SNR-style weighting, noise selection, curriculum learning, DRO), and a simple loss-weighted or focal baseline, are not compared.

W5. Training setting (post-training vs. from scratch) is not clearly stated. The main text does not say whether SALD is applied to pretrained checkpoints or used to train models from scratch. The learning rates (e.g., 1e-5 to 1e-4) and the baseline FIDs after 10k steps suggest fine-tuning of pretrained SANA, PixArt-Σ and FLUX.2 klein, but this should be stated explicitly. It affects how the results should be read: in a fine-tuning setting, gains may partly reflect domain adaptation or generic loss reweighting, and the claims should be scoped to post-training, not to "diffusion training" in general. It also affects the baseline comparison (what a 10k- or 100k-step budget means) and the interpretation of TAM's curriculum, whose gap schedule starts at 1 and grows over K steps. It is unclear whether the conclusions would hold when training from scratch.

W6. The evidence is limited and hyperparameters may be tuned on the test benchmark.

  • There are no seeds or confidence intervals, and the CLIP-Score gains are often within 0.1–0.3.
  • There are many hyperparameters (τ, τ_s, α, λ, Δmax, K), apparently chosen in ablations on the benchmarks used for reporting.
  • Results are on small datasets (e.g., CUB-200, Flowers) and FID-based, with no alignment benchmark such as GenEval or DPG-Bench.
  • The experiments fine-tune pretrained checkpoints, inferred from the learning rates and baseline FIDs. The paper should state the initialization explicitly, and the claims should be scoped to fine-tuning, not diffusion training in general.
Questions for authors

Q1. Is SALD applied to pretrained checkpoints (fine-tuning) or used to train from scratch? Please state the initialization for each backbone in the main text, and indicate whether the conclusions are expected to hold in the other setting.

Q2. At Line 436, is A_i = l_b,i − sg(l_a,i) the same as Eq. 6? If so, I expect the absolute-loss ablation to use |l_b,i|, not |l_a,i|. Could you clarify or re-run it?

Q3. Could you provide an ablation and/or visualization of the sg(P_e,i) term in Eq. 13? How does P_h,i − sg(P_e,i) compare with building the weight map directly from P_h,i? Please also visualize M_i.

Q4. How does SALD compare with hard-path-loss-only weighting, focal-style weighting and Min-SNR, with equal tuning effort and multiple seeds?

Q5. Does A_i correlate with an independent measure of sample difficulty? How often is a sample seen before its TAM memory is used?

Q6. What is the effect of detaching ω_i and M_i? How were the hyperparameter defaults chosen, and do they hold on a held-out benchmark?

9 comments
AuthorsOpenReview ↗

Author Rebuttal (Part 1/4)

Dear Reviewer LigD,

Thank you for your detailed review, for finding the advantage-based idea intuitive and well-motivated, and for appreciating how it unifies training-history curricula, spectral bias, and sample-wise weighting within a training-only framework, along with the wall-clock-matched and 100k-step comparisons.

The reviewer's comments raise several important questions about isolating SALD's components, the training setting, and the statistical evidence. We carefully address each of these questions below. All corresponding additions have been incorporated into the revised manuscript (Section 3 and Appendices C.2 and D.5–D.10).

W.1 & Q.3 Motivation for SAD and the Role of the Easy Spectrum

Response

We thank the reviewer for this careful analysis. Our clarification has three parts:

  • Why subtract sg(Pe,i)\mathrm{sg}(P_{\mathrm e,i}): We agree that Ph,iP_{\mathrm h,i} carries a noise-level-driven, coarse-to-fine profile shared across samples. This is precisely why we subtract the easy spectrum: Pe,iP_{\mathrm e,i}, from the same pair and model, captures the part of this profile already present at lower noise. SiS_i then keeps only the frequencies where this pair's error grows with noise, since Gi(κ)>12G_i(\kappa)>\tfrac12 if and only if Ph,i(κ)>Pe,i(κ)P_{\mathrm h,i}(\kappa)>P_{\mathrm e,i}(\kappa) (Eq. 20). A gate from Ph,iP_{\mathrm h,i} alone would largely re-encode the timestep profile.
  • Gate positivity: Since Ph,i≥0P_{\mathrm h,i}\ge0, a gate from Ph,iP_{\mathrm h,i} alone has Gi≥12G_i\ge\tfrac12 at every frequency. With sg(Pe,i)\mathrm{sg}(P_{\mathrm e,i}), Gi>12G_i>\tfrac12 holds on only 38–50% of bins, and at high noise it falls from about 60% (lowest band) to 25–30% (highest).
  • How MiM_i encodes hard frequencies: GiG_i is zero-phase, so MiM_i is the magnitude of the kernel whose frequency response is GiG_i. It encodes which frequencies are emphasized, not error locations (Section 2.4).
  • Visualization (new Appendix D.8): Figure 2 is a conceptual illustration of the SAD pipeline, not computed output. Figure 9 now shows the actual quantities for three pairs: the per-pixel error of each path (decoded with the VAE), both gates, and both MiM_i overlaid on the image. PeP_{\mathrm e}, PhP_{\mathrm h} and GiG_i are per-frequency quantities, so they are shown as spectra. Figure 10 gives the gate statistics over 200 pairs at three noise levels.
  • Ablation (Appendix D.9, Table 13): As requested, we build the gate directly from Ph,iP_{\mathrm h,i}, keeping everything else fixed.

Table: Ablation of the easy-path spectral reference on SANA 0.6B, COCO (512512).

SAD variantFID↓CLIP↑LPIPS↓
No SAD (AGD + TAM, Table 3)30.7826.600.7529
Gate from Ph,iP_{\mathrm h,i} only30.4626.730.7506
Gate from Ph,i−sg(Pe,i)P_{\mathrm h,i}-\mathrm{sg}(P_{\mathrm e,i}) (ours)29.3127.060.7374

The PhP_{\mathrm h}-only gate recovers only a small part of SAD's gain.

W.2 & Q.2 Isolating the Self-Referenced Contrast

Response

We confirm that Ai=ℓb,i−sg(ℓa,i)A_i=\ell_{b,i}-\mathrm{sg}(\ell_{a,i}) at Line 436 is Eq. 6, with aa and bb denoting the easy and hard paths; the revised paper now uses e\mathrm e and h\mathrm h throughout. The submitted ablation indeed used ∣ℓe,i∣|\ell_{\mathrm e,i}|, as written. As the reviewer suggests, we re-run it with ∣ℓh,i∣|\ell_{\mathrm h,i}| (Table 3, Case 6), selecting τ\tau from the same grid and held-out pairs as SALD and using 3 seeds on COCO (Appendix D.5, Table 10).

Note that both absolute-loss variants remove the contrast only from AGD's weight: TAM still sets the easy-hard gap and SAD still uses Ph−sg(Pe)P_{\mathrm h}-\mathrm{sg}(P_{\mathrm e}). They are therefore not generic hard-example weighting. Generic single-path weighting is represented by Min-SNR and DRO (Table 8).

Table: Isolating the easy-hard contrast on SANA 0.6B.

SettingContrast used inCOCO FID↓COCO CLIP↑MAGICK FID↓MAGICK CLIP↑
Uniform (Base)none31.5926.8038.5926.68
Min-SNRnone31.4926.8637.9326.65
DROnone31.7426.8138.2426.72
Abs. loss ∣ℓe,i∣\lvert\ell_{\mathrm e,i}\rvert (Table 3)TAM, SAD29.6426.9836.9526.54
Abs. loss ∣ℓh,i∣\lvert\ell_{\mathrm h,i}\rvert (best τ\tau)TAM, SAD30.2326.7037.4226.34
SALD (ours)AGD, TAM, SAD29.3127.0636.7426.85

The results provide two observations:

  • Contrast across components: The absolute-loss variants, which keep the contrast in TAM and SAD, close most of the gap between single-path weighting (Min-SNR 31.49, DRO 31.74 on COCO) and SALD (29.31).
  • Contrast in AGD: We agree that the AGD-specific margin is modest. Over ∣ℓh,i∣|\ell_{\mathrm h,i}| weighting it is 0.96 FID on COCO over 3 seeds, which exceeds the seed standard deviation (±0.08, Table 10).
AuthorsOpenReview ↗

Author Rebuttal (Part 2/4)

W.3 & Q.5 Does the Advantage Track Sample Difficulty?

Response

We agree that a single AiA_i is noisy; this motivates TAM's EMA (Section 2.3). We test pair-level signal, prompt specificity, and an exploratory simple–complex comparison (new Appendix D.10, Table 14 and Figure 11):

  • Pair-level signal: We measure AiA_i for 1,000 COCO image–caption pairs using trained SANA 0.6B. We keep the easy and hard timesteps fixed and repeat each measurement with 16 different noise pairs. Differences between image–caption pairs account for 73.3% of the variation in AiA_i; only 26.7% comes from changing the noise for the same pair. Thus, AiA_i contains a repeatable signal about the pair, rather than mainly random noise. Table 3, Case 6 also supports using the same pair as the reference: it gives better FID than using a different pair (29.31 vs. 30.18).
  • Prompt specificity: A pair is hard when its caption leaves the image underdetermined. On 100 DALL·E 3 pairs (pretrained SANA 0.6B), we fix the image, timesteps and 16 noise pairs, and only truncate the caption, so any change in AiA_i comes from the text.

Table: Increase of AiA_i over the full caption (share of images).

Captionσe=0.25\sigma_{\mathrm e}=0.250.500.500.750.75
Empty+0.0037 (82%)+0.0132 (80%)+0.0117 (62%)
First 5 words+0.0027 (80%)+0.0089 (76%)+0.0089 (56%)
First 10 words+0.0015 (76%)+0.0055 (64%)+0.0062 (57%)

AiA_i rises monotonically as the caption becomes less specific (Wilcoxon p<10−3p<10^{-3} up to 10 words at σe≤0.5\sigma_{\mathrm e}\le0.5), because the high-noise path relies more on the text (empty caption: ℓh\ell_{\mathrm h} +0.023 vs. ℓe\ell_{\mathrm e} +0.010).

Simple–complex pilot: We also evaluate five simple and five complex generated image–prompt pairs with pretrained SANA 0.6B, using the same 16 noise pairs across samples. At 25% easy-path noise (σe=0.25\sigma_{\mathrm e}=0.25, σh=0.40\sigma_{\mathrm h}=0.40), complex pairs have a higher mean AiA_i: 0.1466 vs. 0.0291. At 50% (0.500.50, 0.650.65), the means are 0.0385 vs. 0.0330, with substantial sample overlap. Figure 11 in Appendix D.10 shows three simple and three complex examples in a table with images, full generation prompts and both advantage values. These examples were selected to illustrate the higher advantage of complex pairs at both levels (shown-example means: 0.163 vs. 0.033 at 25%, 0.053 vs. 0.023 at 50%); the means above use all ten pilot pairs. This small pilot illustrates an association at these two noise levels, rather than a universal complexity ranking. It changes both image and text; the fixed-image specificity test above isolates the conditioning effect.

On the remaining points:

  • TAM usage: With B=8B=8, every CUB-200 image is visited about 7 times within 10k steps (68 at 100k). On COCO (five captions per image), 13.8% of images are revisited by 10k steps and all by 100k. An unvisited entry has mi=0m_i=0, so TAM reduces exactly to the base gap schedule (Eq. 10). Full statistics are in Appendix D.7 (Table 12).
  • Non-detached weights: The gradient through ωi\omega_i adds a term cj(ℓjSAD−ℓˉ)∇θℓh,jc_j(\ell^{\mathrm{SAD}}_j-\bar\ell)\nabla_\theta\ell_{\mathrm h,j} with cj≥0c_j\ge0, which emphasizes samples above the batch loss (derivation in Appendix C.2). Plain reweighting with detached ωi\omega_i performs worse (Appendix D.9, Table 13).
  • Theorem 1: We would like to clarify that Theorem 1 is stated for the SALD objective FmF_m itself: its stochastic gradient gsg_s includes all gradient paths through ωi\omega_i and MiM_i, so it covers the non-detached objective, not plain reweighting. It is a convergence guarantee, not a speed claim. The new Appendix C.1 (Lemma 5, Theorem 6) analyzes SALD with the online memory: each TAM update moves the objective by at most γ≤ℓmax⁡B2/N\gamma\le\ell_{\max}B^2/N, so the bound gains only a drift term 2γ/η2\gamma/\eta and recovers Theorem 1 for a fixed memory.
AuthorsOpenReview ↗

Author Rebuttal (Part 3/4)

W.4 & Q.4 Difficulty-Aware Baselines and Multiple Seeds

Response

We thank the reviewer for this suggestion. The revised manuscript adds Min-SNR and DRO on SANA 0.6B under the same 10k-step budget (Appendix D.5, Table 8). Noise selection is represented by SpeeD (Tables 1-2).

Table: FID↓ of difficulty-aware baselines on SANA 0.6B at 512×512512\times512 (Table 8).

MethodCOCOMultiGenFlickr8kCUB-200FlowersAvg.
Base31.5931.9328.8326.0630.6429.81
Min-SNR31.4931.3427.3925.5729.7529.11
DRO31.7431.5028.7025.1228.2429.06
SALD (ours)29.3129.1527.9123.7025.8327.18

SALD obtains the best FID on four of five datasets, with Min-SNR lower on Flickr8k, and the best or tied-best CLIP and the best LPIPS on all five. At 10241024, SALD also leads on MAGICK (36.74 vs. 37.93 / 38.24) and DALL·E 3 (11.98 vs. 13.69 / 12.82; Table 9).

For hard-path-loss-only weighting, which is the regression analogue of focal-style weighting (ωi\omega_i increasing in the sample loss), we select τ\tau from the same four values and held-out pairs as SALD and run 3 seeds on COCO (Appendix D.5, Table 10).

Table: 3-seed comparison on SANA 0.6B, COCO (512512) (mean ± std).

MethodFID↓CLIP↑LPIPS↓
∣ℓh,i∣\lvert\ell_{\mathrm h,i}\rvert weighting (best τ\tau)30.26 ± 0.0826.72 ± 0.060.7410 ± 0.008
SALD (ours)29.30 ± 0.0227.06 ± 0.020.7377 ± 0.005

SALD improves over hard-path weighting by 0.96 FID, beyond the seed standard deviation (±0.08), confirming that the gain is not a single-seed effect.

W.5 & Q.1 Training Setting and Scope of Claims

Response

We thank the reviewer for raising this concern. All experiments fine-tune public pretrained checkpoints; we do not train from scratch (Appendix E). The initializations are the official SANA 0.6B and 1.6B, PixArt-Σ\Sigma (512512 and 10241024) and FLUX.2 klein base (4B) checkpoints. Every baseline starts from the same checkpoint with the same data, global batch size (B=8B=8, Appendix D.1) and step budget. The revised Section 3 now states this initialization for every backbone, and the abstract and conclusion scope the claims to post-training of pretrained checkpoints.

On the related concerns:

  • Domain adaptation: All methods, including Base, are fine-tuned on identical data for identical steps, so domain adaptation is shared; the differences reflect the objective.
  • Generic reweighting: We compare against Min-SNR, DRO and hard-path loss weighting (W.4 & Q.4).
  • Budget: The 10k and 100k steps are fine-tuning steps. The gap persists at 100k (Table 7; e.g., 2.58 FID for SANA 1.6B on MultiGen at both 10k and 100k).
  • TAM curriculum: The ramp spans the first half of training (KK is half the training steps, e.g., 5k of 10k); afterwards δbase\delta_{\mathrm{base}} is sampled uniformly (Eq. 9).
  • From scratch: We do not claim transfer. Early in pretraining both paths have large errors and the advantage may be less informative; we leave this to future work.
Reviewer_AhGSOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SALD, a self-referenced training framework for diffusion models. The main idea is to compare prediction errors at different noise levels using the same model and use the difference to adapt the training objective. The method consists of three components: AGD for sample reweighting, TAM for adaptive noise-level scheduling, and SAD for frequency-based weighting. Experiments on SANA, PixArt-Σ, and FLUX.2 show improvements in FID across multiple datasets, with ablation studies supporting the effectiveness of the proposed components.

Critical strengths
  1. The overall idea is intuitive and practical. The method does not require an additional teacher model or introduce extra inference overhead, making it relatively easy to integrate into existing diffusion training pipelines.

  2. The experiments are fairly comprehensive, covering different model sizes, datasets, and resolutions. The improvements in FID are generally consistent. I also appreciate the ablation studies, wall-clock comparisons, and longer training experiments, which help support the empirical findings.

Critical weaknesses
  1. I have some concerns about the mathematical formulation of SAD. In Eq. 14, the frequency-domain gate is directly transformed back using IFFT, but the resulting weights are not necessarily non-negative. Since these weights are then applied to the squared residuals in Eq. 16, this could potentially affect the optimization objective and training stability. I think this deserves more discussion.

  2. The novelty over existing difficulty-based sample reweighting methods is not entirely clear to me. While the paper includes an absolute-loss ablation, I would like to see a more direct comparison with using the hard-path loss itself for sample reweighting. This would help clarify how much of the improvement actually comes from the proposed self-referenced advantage rather than standard hard-example weighting.

Questions for authors
  1. For SAD, can the weights obtained after IFFT become negative? If so, is there any additional constraint or numerical stabilization applied in the implementation?

  2. Could the authors compare SALD with a simpler baseline that directly uses the hard-path loss for sample reweighting? I think this would help better establish the importance of the easy-hard contrast.

2 comments
AuthorsOpenReview ↗

Author Rebuttal (Part 1/1)

Dear Reviewer AhGS,

Thank you for your review and for finding SALD intuitive and practical, with no teacher model or inference overhead, and for appreciating the breadth of the experiments, ablations, wall-clock comparison and longer training runs. Below, we clarify the non-negativity of the SAD weights and add a direct comparison with hard-path loss reweighting, which is included in the revised manuscript.

W.1 & Q.1 Non-Negativity of SAD Weights

Response

We thank the reviewer for this careful observation: the raw inverse transform can indeed be negative. Because GiG_i is real and zero-phase, Fr−1(Gi)\mathcal{F}_r^{-1}(G_i) is a real, symmetric kernel whose values can change sign. Therefore gradient can flip in the oposite direction during the update. To overcome this issue Eq. 14 therefore takes the elementwise magnitude, M~i=∣Fr−1(Gi)∣\widetilde{M}_i=|\mathcal{F}_r^{-1}(G_i)|, before normalization. The magnitude ∣⋅∣|\cdot| is written explicitly in Eq. 14. This guarantees:

  • Non-negativity: M~i,p≥0\widetilde{M}_{i,p}\ge0 and ∑pM~i,p+εm>0\sum_p\widetilde{M}_{i,p}+\varepsilon_m>0, so Mi,p≥0M_{i,p}\ge0 for every element (Eq. 15).
  • Fixed scale: The mean of MiM_i is ∑pM~i,p/(∑pM~i,p+εm)≈1\sum_p\widetilde{M}_{i,p}/(\sum_p\widetilde{M}_{i,p}+\varepsilon_m)\approx1, so Eq. 16 is always a non-negative weighted MSE on the same scale as ℓh\ell_{\mathrm h}.
  • Stable training: Together with the normalized ωi\omega_i (Eq. 8) and gradient-norm clipping at 1.0, training is stable; the convergence curves (Figure 4) and 100k-step runs (Table 7) show no divergence.

Hence the ∣⋅∣|\cdot| in Eq. 14 already enforces the constraint, and no further numerical stabilization is needed.

W.2 & Q.2 Comparison with Hard-Path Loss Reweighting

Response

We thank the reviewer for this suggestion. We add the suggested variant, ∣ℓh,i∣|\ell_{\mathrm h,i}| weighting, which replaces AiA_i by the hard-path loss in Eq. 7 and keeps everything else unchanged. For equal tuning effort, its temperature is selected from the same grid τ∈0.1,0.5,1.0,2.0\tau\in\\{0.1,0.5,1.0,2.0\\} and on the same held-out COCO training pairs as SALD (Appendix D.1). We also compare with two standard single-path difficulty-aware methods, Min-SNR and DRO (Appendix D.5; Table 8 at 512512, Table 9 at 10241024).

Table: Hard-example weighting vs. self-referenced advantage on SANA 0.6B.

SettingCOCO FID↓COCO CLIP↑COCO LPIPS↓MAGICK FID↓MAGICK CLIP↑MAGICK LPIPS↓
Uniform (Base)31.5926.800.740838.5926.680.7470
Min-SNR31.4926.860.739337.9326.650.7363
DRO31.7426.810.740138.2426.720.7326
∣ℓh,i∣\lvert\ell_{\mathrm h,i}\rvert weighting (best τ\tau)30.2326.700.740937.4226.340.7463
SALD (ours)29.3127.060.737436.7426.850.7230

Two observations follow:

  • Beyond standard hard-example weighting: SALD outperforms both single-path methods on both datasets. Across the five 512512 benchmarks of Table 8, it obtains an average FID of 27.18, compared with 29.11 for Min-SNR and 29.06 for DRO.
  • Contribution of the contrast: Using the advantage AiA_i instead of the raw hard-path loss improves FID by 0.92 on COCO (0.96 over 3 seeds, Table 10) and 0.68 on MAGICK.

Unlike single-path weighting, SALD's easy-hard contrast also drives the noise-gap curriculum (TAM) and the frequency emphasis (SAD), which hard-path loss alone cannot provide.

Happy to have further discussion!

AuthorsOpenReview ↗

A quick-follow-up

Dear, Reviewer,

Just a quick followup if there are any concern with the response.

Hope we addressed all the concern raised by the reviewer and it will help to raise the reviewers score.

We are happy to discuss if reviewer has any further concern.

Thanks

Reviewer_pZktOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes SALD, a self-referenced training framework for diffusion models. It constructs training signals by comparing prediction errors at two noise levels using the same model. SALD consists of three components: Advantage-Guided Diffusion adjusts sample weights based on relative errors, Temporal Advantage Memory adapts noise gaps using historical difficulty, and Spectral Advantage Decomposition introduces frequency-derived latent weights. Experiments show that this method has certain effectiveness.

Critical strengths
  1. SALD constructs training signals from predictions at different noise levels without requiring an external teacher, additional trainable parameters, or changes to the inference procedure.
  2. SAD extends the training signal to the frequency domain. By comparing residual power spectra between the two paths, it provides finer-grained latent weights that complement the sample-level weighting of AGD.
  3. The experimental evaluation is extensive.
Critical weaknesses
  1. Sections 2.2 and 2.5 explicitly state that ωi\omega_i remains differentiable and allows gradients to propagate through the hard path. However, Section 3.2 reports that allowing gradients through ωi\omega_i degrades performance. These statements appear contradictory, and the actual gradient computation needs clarification.
  2. TAM maintains a difficulty estimate for each training sample and uses it when the sample is revisited. However, MultiGen-20M contains a large number of samples, while training typically lasts only 10k steps. Reporting sample revisit rates and the fraction of updated memory entries would help establish how effectively TAM operates in large-scale training.
  3. The theoretical analysis does not fully account for the dynamic updates in TAM. Theorem 1 defines the objective FmF_m under a fixed memory state mm, whereas TAM continuously updates mm using the current advantage values. The analysis therefore does not establish how these changing memory states affect convergence.
Questions for authors

Please see Weaknesses.

5 comments
AuthorsOpenReview ↗

Author Rebuttal (Part 1/2)

Dear Reviewer pZkt,

Thank you for the positive assessment and for recognizing that SALD builds its training signal without an external teacher, extra parameters, or changes to inference, that SAD complements AGD with finer-grained spectral weighting, and that the evaluation is extensive. We address each point below; the corresponding clarifications are incorporated into the revised manuscript.

W.1 Gradient Flow Through the AGD Weight

Response

We are sorry for the unclear wording. The two statements refer to different gradient paths, and the description of Case 6 was mis-understood because of the unclear wording, please find the clarification below:

  • SALD (Sections 2.2, 2.5): ωi\omega_i is differentiable only through the hard path. The easy reference is stopped, so ∇θAi=∇θℓh,i\nabla_\theta A_i=\nabla_\theta\ell_{\mathrm h,i} (Eq. 6).
  • "Detach disabled" (Section 3.2, Case 6): removes sg(⋅)\mathrm{sg}(\cdot) from ℓe,i\ell_{\mathrm e,i}, so ∇θAi=∇θℓh,i−∇θℓe,i\nabla_\theta A_i=\nabla_\theta\ell_{\mathrm h,i}-\nabla_\theta\ell_{\mathrm e,i}. The optimizer can then lower a sample's weight by increasing its easy-path loss instead of reducing its hard-path loss, which is the exploitation we observe (COCO FID 30.05 vs. 29.31; MAGICK 37.43 vs. 36.74).

The gradient through the hard path is beneficial because it sharpens the difficulty focus. Differentiating Eq. 17 gives

∇θLSALD=1B∑j[ωj∇θℓjSAD+cj(ℓjSAD−ℓˉ)∇θℓh,j],cj=B,qj(1−qj)τ∑kqk≥0,\nabla_\theta\mathcal L_{\mathrm{SALD}}=\frac1B\sum_j\Big[\omega_j\nabla_\theta\ell^{\mathrm{SAD}}_j+c_j\big(\ell^{\mathrm{SAD}}_j-\bar\ell\big)\nabla_\theta\ell_{\mathrm h,j}\Big],\qquad c_j=\frac{B\\,q_j(1-q_j)}{\tau\sum_k q_k}\ge 0,

where ℓˉ=LSALD\bar\ell=\mathcal L_{\mathrm{SALD}} is the ω\omega-weighted batch mean of ℓSAD\ell^{\mathrm{SAD}}. Samples above the batch mean therefore receive additional weight on their hard-path loss, and those below it less. Removing sg(⋅)\mathrm{sg}(\cdot) adds −cj(ℓjSAD−ℓˉ)∇θℓe,j-c_j(\ell^{\mathrm{SAD}}_j-\bar\ell)\nabla_\theta\ell_{\mathrm e,j}, which rewards a larger easy-path loss for exactly these samples. The revised Section 3.2 states the Case 6 setting precisely, and the full derivation is in the new Appendix C.2.

W.2 TAM Revisit Rates in Large-Scale Training

Response

We thank the reviewer for this suggestion, and agree that TAM's memory helps only when pairs are revisited. With a global batch of B=8B=8 (Appendix D.1), a run makes 80k pair visits in 10k steps and 800k in 100k steps. TAM keeps one memory entry per image, shared by its captions, and a stored mim_i is used only from the image's second visit onward, so the key quantity is the share of entries revisited.

Table: TAM memory usage on SANA 0.6B (B=8B=8), 10k / 100k steps.

DatasetAvg. visits per entryUpdated ≥1 (%)Revisited ≥2 (%)
CUB-2006.79 / 67.87100 / 100100 / 100
Oxford Flowers9.77 / 97.69100 / 100100 / 100
Flickr8k9.89 / 98.88100 / 100100 / 100
COCO0.68 / 6.7651.7 / 10013.8 / 100
MultiGen-20M0.004 / 0.040.4 / 4.00 / 0

The table supports three observations:

  • Active memory: On CUB-200, Flowers and Flickr8k, every entry is revisited 7–10 times within 10k steps, so the memory sets the gap of every image. COCO has five captions per image, so 13.8% of its entries are already revisited within the first 80k visits, and all of them at 100k (about 7 visits each).
  • Web scale: On MultiGen-20M the memory is effectively inactive (at most 4% of entries touched, none twice). An unvisited entry has mi=0m_i=0, so tanh⁡(mi)=0\tanh(m_i)=0 and δi=δbase(s)\delta_i=\delta_{\mathrm{base}}(s) (Eq. 10): TAM reduces exactly to the base easy-hard gap schedule. The MultiGen gains (FID 31.93 to 29.15, Table 1), which persist at 100k steps (Table 7), therefore come from AGD, SAD and the gap schedule, not from per-image memory.
  • Memory contribution: Where the memory is active, disabling it (λ=0\lambda=0, schedule kept) on CUB-200 raises FID from 23.70 to 24.04.

The revised manuscript adds this table and analysis as Appendix D.7 (Table 12), and scopes TAM's per-image memory to datasets whose images are revisited during training. For web-scale data, a group-level memory (e.g., per caption cluster) is a natural extension.

AuthorsOpenReview ↗

Author Rebuttal (Part 2/2)

W.3 Theorem 1 Under Dynamic TAM Memory

Response

We agree that Theorem 1 holds the memory fixed (Remark 3). The revised paper analyzes the joint dynamics of (θs,ms)(\theta_s,m_s) in a new Appendix C.1, with complete proofs (Lemma 5 and Theorem 6). In brief:

  • How the adaptation enters: A step updates only the memory entries of the (at most BB) visited images, and a batch loss depends on mm only through the entries of its own images. The batch loss is a convex combination of per-sample losses bounded by ℓmax⁡\ell_{\max}, so each memory update moves the objective by at most γ=ℓmax⁡Bπmax⁡\gamma=\ell_{\max}B\pi_{\max} for every θ\theta, where πmax⁡\pi_{\max} bounds an image's inclusion probability; under uniform sampling, γ≤ℓmax⁡B2/N\gamma\le\ell_{\max}B^2/N (Lemma 5). This holds despite the rounding in Eq. 10, so no smoothness in mm is required. Under an optional Lipschitz assumption in mm, γ≤2(1−α)LmBℓmax⁡/N\gamma\le2(1-\alpha)L_mB\ell_{\max}/N, showing the role of the EMA rate α\alpha.
  • Result (Theorem 6): Assume, after the curriculum ramp (s≥Ks\ge K), LL-smooth objectives, fresh independent batches, an unbiased implemented gradient with variance ν2\nu^2, and bounded residuals, with F⋆=inf⁡m,θFm(θ)F^\star=\inf_{m,\theta}F_m(\theta). Then, for 0<η≤1/L0<\eta\le1/L,

1J∑s=KK+J−1E∣∇Fms(θs)∣22≤2(EFmK(θK)−F⋆)ηJ+Lην2+2γη.\frac{1}{J}\sum_{s=K}^{K+J-1}\mathbb{E}\\|\nabla F_{m_s}(\theta_s)\\|_2^2\le\frac{2\left(\mathbb{E}F_{m_K}(\theta_K)-F^\star\right)}{\eta J}+L\eta\nu^2+\frac{2\gamma}{\eta}.

The only new term is the drift 2γ/η2\gamma/\eta. This is an averaged-stationarity guarantee for FmsF_{m_s}, the objective SALD optimizes at step ss, and it recovers Theorem 1 (γ=0\gamma=0) when the memory is frozen or λ=0\lambda=0.

  • Size of the drift bound: With B=8B=8 and uniform sampling, γ/ℓmax⁡≤64/N\gamma/\ell_{\max}\le64/N: 5.4×10−35.4\times10^{-3} on CUB-200, 5.4×10−45.4\times10^{-4} on COCO and 3.2×10−63.2\times10^{-6} on MultiGen-20M. It is largest where the memory is revisited most (W.2) and far smaller at web scale.

As before, the result does not claim faster convergence than uniform flow matching.

Happy to have further discussion!

AuthorsOpenReview ↗

A quick-followup

Dear, Reviewer,

Just a quick followup if there are any concern with the response.

We are happy to discuss if reviewer has any further concern.

Thanks

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.