acceptodds
Under review as a conference paper at ICLR 2027

Controlling Polar Exposure to Delay Memorization in Diffusion Models

Abstract

Diffusion models can reach useful sample quality before copying training examples, but fast optimization can compress this generalization window by accelerating sample-specific fitting. We investigate this effect through update geometry and propose Quality-Gated De-whitening (QGD), a controller that retains a fast polar-update prefix and progressively restores fixed-gain momentum. Our random-feature analysis separates covariance-controlled, curvature-equalized and amplitude-controlled memorization clocks. Under aligned spectral assumptions, it establishes a finite-exposure condition under which a fixed-gain tail recovers a delay proportional to dataset size. QGD implements this principle with a confirmed quality gate, a bounded decay envelope and causal copy feedback. Immediate switching is the conservative limit; gradual control balances delayed copying against continued quality improvement. We pair QGD with Copy-Budgeted Selection (CBS), which applies simultaneous binomial calibration to a frozen checkpoint family, followed by a fresh evaluation of the released checkpoint. On 2,000-image CIFAR-10 subsets, QGD preserves the polar baseline's quality-arrival time while expanding its useful interval by 8.32x and reducing common-checkpoint copying by 75.9%. With identical calibration and independent quality evaluation, QGD achieves FID 75.56 versus 79.37 for SGD with the same selector. Exposure-matched controls, independent detector audits and transfer to flow matching and dance generation support adaptive exposure control as a practical way to improve the quality–copying tradeoff.

Primary area:
generative models
Keywords:
Diffusion Models, Memorization, Generalization, Training Dynamics

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_RyMDOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to optimize updated geometries actually affects the window between it about the useful sample quality and copy training data set in diffusion models.

Critical strengths

This paper provided a clear mechanistic frame. It also gives a careful experimental hygiene. There is also an informative ablation study to do the experiment.

Critical weaknesses

Seems like the practical benefit is small and rests on narrow evidence as the paper shows the final FID from 75.56 vs 79.37 is a modest gain from the regimen of the FID 75 - 85 and I do think there is a big gap remain from the theoretical to practice gap

Questions for authors

How does QGD compare to Muon to SGD/AdamW hard switch with separately tuned LR after the gate, and to Muon with a simple LR decay or early stop at the first copy warning?

1 comment
AuthorsOpenReview ↗

Response to Reviewer RyMD

Thank you for the positive assessment and for naming concrete alternatives to the gradual transition.

W. Size of the benefit and the theory–practice gap. The main comparison now separates quality arrival, the observed low-copy interval, common-checkpoint quality, and quality after checkpoint selection. On the 2,000-image setting, QGD shares Muon's 8.0k quality arrival, compared with 16.0k for SGD, and extends Muon's 20.0k window to 166.4k (Table 1). Under the same CBS rule, final FID is 75.56 for QGD, 79.37 for SGD, 81.94 for AdamW and 86.14 for Muon (Table 3). Table J.4 reports the differences within paired trajectories.

The comparisons also expose the tradeoff. Conservative alternatives can have lower copying and longer windows. QGD's quality advantage persists at the common checkpoint for n=10,000n=10,000 (FID 56.1 versus 61.0 for the best baseline in Table 1), flow matching (79.8 versus 82.8), and dance generation (kinetic FID 27.9 versus 30.2; Table 4). Appendix I evaluates an additional copying detector. Table N.1 adds the resolution/capacity setting, where QGD has FID 61.3 versus 64.2 for the hard switch, while the hard switch copies less.

The theory does not numerically predict these neural outcomes. Section 2 states its fixed-feature and alignment conditions; Section 4.2 tests qualitative consequences using matched controls. This distinction is retained throughout the revision.

Q. Tuned hard switching, learning-rate decay and early stopping. We added these comparisons to Section 4.2 and Table J.8. Both switches share Muon's prefix through the confirmed gate. Post-gate rates are selected by independent pilots from multipliers {0.5,1,2,4}\{0.5,1,2,4\}: the selected multipliers are 2 for the inherited SGD gain and 1 for the common AdamW matrix rate of 2×10−42\times10^{-4}. SGD retains its momentum buffer, whereas AdamW initializes its matrix-parameter moments at the switch. The decay comparator begins cosine decay at the gate and reaches zero at 200k.

The table reports means. Common-checkpoint audits are at 60k, except warning stopping, whose audit is at its stopping checkpoint. The full manuscript provides the standard deviations.

MethodWindow (k)Audit FID ↓Audit copy (%) ↓Final FID under CBS ↓
Muon → SGD, tuned post-gate rate143.679.83.0777.40
Muon → AdamW, tuned post-gate rate106.879.63.7278.83
Muon + gate-started cosine decay31.682.010.4685.39
Muon + first-warning stop2.887.12.8987.06
Same-gate hard switch, inherited gain186.081.52.4777.82
QGD166.477.83.8875.56

Tuning improves the SGD switch's released FID from 77.82 to 77.40 while shortening its window from 186.0k to 143.6k. QGD retains a longer window than that tuned switch and a lower released FID, while the tuned switch has lower audit copying. Warning stopping is conservative but ends soon after the quality gate: the mean stopping time is 10.8k and the mean observed window is 2.8k. Its truncated archive uses its own candidate count for CBS, so it is not penalized by the full archive's multiplicity allocation.

Proposition 8 in Appendix E supplies the mathematical comparison. In exact aligned polar flow, a scalar rate schedule re-parameterizes cumulative polar time; after a hard switch, a scalar gradient-rate schedule re-parameterizes the gradient path. The mixture leaves both pure-flow curves locally in the specified diagonal family. These statements do not establish a universal performance ordering, and stochastic momentum updates need not obey the same re-timing identity.

Similarly, an early-stopped model is a checkpoint of its continued trajectory, but that does not make CBS on the full archive automatically superior: the candidate family and calibration allocation also change. Table J.8 therefore compares the actual release procedures directly. The theory identifies why the baselines are relevant, and the evaluation determines their quality–copying tradeoffs under the stated protocol.

Reviewer_TCFyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies how optimizer geometry affects the interval between useful generation and memorization in diffusion models. It distinguishes covariance-controlled, curvature-equalized, and amplitude-controlled learning dynamics, and proposes Quality-Gated De-whitening (QGD), which gradually transitions from polar updates to fixed-gain momentum after reaching a quality target. The paper also introduces Copy-Budgeted Selection (CBS) for selecting checkpoints under a detector-defined copying budget. Experiments cover CIFAR-10 diffusion models, flow matching, and music-conditioned dance generation.

Critical strengths
  1. An interesting research question and a detailed theoretical analysis.
  2. Detailed ablation experiments.
Critical weaknesses
  1. Despite extensive theoretical analysis, the image experiments remain limited to small CIFAR-10 subsets and a compact UNet. The flow-matching experiments reuse the same dataset and architecture, while the dance experiments do not establish scalability in image generation. Without evaluation on a larger-scale or higher-resolution benchmark, such as ImageNet, it remains unclear whether the proposed mechanism and quality–copying improvements persist in more realistic settings.
Questions for authors

Could the authors clarify how the assumptions underlying the theoretical analysis relate to the observed neural training dynamics? In particular, what empirical evidence supports the spectral alignment and residual-margin conditions? Given that hard switching achieves a longer memorization delay in the analyzed model, what evidence connects the quality gains of gradual QGD to the proposed exposure-control mechanism?

1 comment
AuthorsOpenReview ↗

Response to Reviewer TCFy

Thank you for the questions on scale and on the relationship between the analysis and neural training.

W1. Scale. We added the resolution/capacity comparison in Section 4.4, Appendix N and Table N.1. It uses the same class-balanced 10,000-image CIFAR-10 subset, bicubically resized to 64 × 64, and a width-doubled DDPM-style UNet. Five paired trajectories use a 400k horizon, probes every 2k, a fixed FID target of 70, and a 5% copying budget. The table below reports means; Table N.1 includes between-trajectory standard deviations. FID and copying use independent audits at 120k.

MethodQuality arrival (k)Window (k)FID ↓Copy (%) ↓
AdamW19.6330.864.91.65
Muon14.070.466.714.11
Same-gate hard switch14.0297.264.22.63
QGD14.0255.261.33.50

QGD retains Muon's quality arrival, widens its window by 3.63× and improves audit FID from 66.7 to 61.3. The tradeoff also persists: the hard switch and AdamW retain longer windows and lower copying, while QGD has the lowest FID among these methods. Holding the image collection fixed isolates the joint change in resolution and model width; this is not evidence of transfer to a new image distribution.

The dataset-size comparison provides a complementary axis. At n=10,000n=10,000 in Table 1, QGD's window is 348.0k versus 84.0k for Muon, with common-checkpoint FID 56.1 versus 61.5 and copying 2.99% versus 12.37%. Flow matching and AIST++ evaluate another objective and modality (Table 4), but we agree that they do not independently establish scalability in image generation. Section 5 now states the evaluated regime and these limits.

Q1. How do the assumptions relate to neural dynamics? We distinguish an exact model check, finite-range diagnostics and neural interventions.

  1. Exact model check. Appendix E specifies every term of the aligned family, including the spectrum, target amplitudes, gate state and fixed gains. The admissible mixture has exposure margin 0.164813 and retains the theorem's lower and upper clock bounds at every size (Table E.1). Its post-gate finite-range slope is 0.986; the excess-exposure and positive-floor controls give 0.038 and 0.111. A nonpositive margin removes the sufficient guarantee; the latter outcomes describe these particular controls, not a converse theorem.
  2. Finite-range diagnostics. Appendix F and Table C.1 compare the clock summaries, their spectral dependence and the aligned crossover calculation. The differences between the reported exponents and the aligned values are stated explicitly. These comparisons do not identify the latent fluctuation margin of a trained neural network.
  3. Neural interventions. Section 4.2 and Table J.6 test the controller behavior suggested by the mechanism:
Prediction or controlInterventionExposure (k)Window (k)Copy at 60k (%)
ReferenceQGD4.21166.43.88
Conservative zero-exposure limitSame-gate hard switch0.00186.02.47
Persistent polar directionContinued MuonUnbounded20.016.10
Increased post-gate exposureQGD without feedback7.0470.04.77
Normalization can erase the delayQGD with per-step RMS4.5130.013.51

The normalization control is particularly informative because the exposure is close to QGD's while the window contracts sharply. The controller's exposure is observable; the theorem's fluctuation decomposition is not supplied by these measurements. We therefore present the interventions as tests of qualitative predictions, not verification of every theoretical assumption.

Q2. Why does gradual exposure help if the hard switch gives the longest delay? Proposition 7 establishes the hard-switch ordering under a common gate state and gain. The value of a gradual transition lies in quality within the retained interval.

At n=256,t=4n=256,t=4, finite exposure reduces normalized population risk from 0.06997 (hard switch) to 0.05471, although the hard switch achieves a lower minimum over longer horizons (Fig. E.1c). Neural results show a similar tradeoff: FID improves from 81.5 to 77.7–78.3 as exposure increases, while copying rises (Spearman ρ=−0.70\rho=-0.70 for FID, +0.81+0.81 for copying; Table J.7). These correlations do not establish causality or monotonicity. At matched exposure, feedback improves FID (77.8 vs. 79.0) and window length (166.4k vs. 124.0k) over pilot-fixed open loop (Table 2). Under the same CBS rule, QGD achieves FID 75.56, outperforming both the inherited-gain hard switch (77.82) and tuned SGD switch (77.40; Tables 3, J.8), while the baselines retain lower copying.

Reviewer_KxWPOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how optimizer choice affects when diffusion models start memorizing training data. By leveraging a linear random-feature model, it argues that gradient descent memorizes slowly because sample-specific information lies in weak-signal directions, which gives larger datasets a natural delay. In contrast, Muon’s orthogonalized updates move all directions at the same speed and largely remove this delay. Based on this, the authors propose QGD, which trains with Muon until a quality target is reached and then fades to momentum SGD. They also propose CBS, a checkpoint-release rule that keeps the detected copy rate under a budget.

Critical strengths
  • The idea behind the paper is interesting. Linking memorization timing to optimizer geometry, namely Muon’s equalization of update directions, is a useful perspective.

  • The random-feature analysis and the toy-model evidence from Section 2 are convincing.

Critical weaknesses
  • Clarity and presentation: The paper is very hard to follow for anyone outside this specific subarea. The abstract and introduction rely on paper-specific terminology that is used before being explained. Most sections begin without stating their purpose. Much of the supporting material is deferred to the appendix, including figures referenced as if they were in the main text (Figures 3 and 4). The method can be stated in simple terms as: train with Muon until a quality target, then fade to momentum SGD.

  • Strong assumptions and theory-experiment gaps: The analysis relies on strong assumptions: spectral separation between population and sample-specific structure, aligned singular directions, and fixed features, but robustness is not discussed. The empirical scaling exponents for Muon in the random-feature experiments (~0.2 and 0.6) also do not obviously match the theoretical prediction, and the paper does not reconcile them.

  • Novelty: In practice the Muon share is near zero within about 20k steps of the gate, so for most of training, QGD is simply SGD+momentum after a Muon warmup. The novelty seems to lie mainly in the quality and copy triggers.

Questions for authors

See Weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer KxWP

Thank you for the detailed presentation comments and for asking how the finite-range exponents relate to the theory.

W1. Clarity and presentation. We have rewritten the abstract and introduction around the training procedure: use Muon until a sample-quality target is confirmed, then reduce polar exposure while transitioning to fixed-gain momentum SGD; a rising copying probe accelerates this transition. The release decision is a separate stage: CBS calibrates a frozen checkpoint family and selects by development quality, followed by an independent final audit.

Quality and memorization clocks, polar updates, exposure and fixed gain are defined at first use. Table O.1 collects the terms and Algorithm 1 gives pseudocode. Sections 2–4 begin with their purpose, and Section 4.2 uses titled paragraphs. The geometry figure is now Figure 2 in Section 2. The aligned-diagnostic figure is Figure E.1 and is explicitly cited as appendix material.

W2. Assumptions and the exponent mismatch. Section 2, “Scope of the Assumptions,” and Table C.1 now separate three issues.

  • Spectral separation. Proposition 3 does not require polar alignment. Its linear-in-dataset-size conclusion requires fluctuation eigenvalue quantiles of order 1/n1/n and a uniformly controlled population–fluctuation cross term. Table F.2 documents the finite-range spectral pattern: the population eigenvalue varies by a factor of approximately 1.09, while the fluctuation eigenvalue decreases approximately as 1/n1/n. This supports the relevance of the analyzed regime within that comparison; it is not a universal spectral claim.
  • Alignment and rank. Proposition 5 gives an exact coefficient-distance clock under aligned, disjoint channels. With equal fluctuation singular values, the half-arrival time is s/(2η)s/(2\eta), independently of the positive covariance eigenvalues. The n/r\sqrt{n/r} dependence follows when the fluctuation energy and rank imply s∝n/rs\propto\sqrt{n/r}. Practical momentum, finite orthogonalization and learned features are not included in that exact-flow statement.
  • Finite-range exponents. For the crossover family s(n,r)=a1+n/rs(n,r)=a\sqrt{1+n/r}, the local slope is n/[2(n+r)]n/[2(n+r)]. Over the stated sizes, the fitted aligned slopes are 0.29 for r=64r=64 and 0.41 for r=16r=16. Table F.1 reports Muon clock slopes of 0.19/0.18 and 0.60/0.67, respectively. These are not equal to the aligned predictions, and the latter pair exceeds one half. We therefore use the aligned model to identify a mechanism and a finite-range rank effect, without claiming it quantitatively predicts those exponents. The pooled eigenvalue correlations in Table F.6, 0.13 for Muon and 0.99 for gradient flow, are descriptive associations rather than a direct verification of alignment.

The neural interventions in Table J.6 test consequences suggested by the analysis. They do not establish that the neural network satisfies the fixed-feature assumptions.

W3. Novelty and the role of the short fade. We agree that a Muon warm-up followed by SGD is the zero-exposure case and is not itself our contribution. The contribution is the exposure condition and the quality–delay tradeoff it motivates.

First, Proposition 1 identifies a sufficient quantity to bound: cumulative polar exposure below the remaining fluctuation margin. Proposition 7 gives the same-gate hard switch as the conservative reference. Second, the normalization control tests whether retaining a covariance-dependent tail matters. At similar exposure, per-step RMS normalization gives a 30.0k window and 13.51% copying, compared with 166.4k and 3.88% for QGD (Table 2). The theoretical counterexample establishes that normalization can erase the spectral delay; it does not prove fixed gain is necessary for every possible controller.

Third, the short fade improves quality relative to the hard switch: common-checkpoint FID changes from 81.5 to 77.8, and final FID under the same CBS rule changes from 77.82 to 75.56 (Tables 2–3). The hard switch retains the longer low-copy interval. Table J.7 examines how quality and copying vary with exposure, rather than treating the fade's duration alone as the contribution.

Finally, CBS integrates established exact binomial bounds and simultaneous calibration with quality-based checkpoint selection. Its guarantee concerns joint release and detector-budget violation; we do not claim a new binomial testing method or a population-FID guarantee.

I thank the authors for the clarifications provided. While the revised paper (especially abstract and introduction) is more amenable to a general ML audience, I will be maintaining my score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Changes in the Revised Manuscript

We thank the reviewers for their detailed feedback. The revised manuscript clarifies the method and its assumptions, adds the requested baselines, and expands the resolution and capacity comparison. New and rewritten passages are marked in blue. References below use the revised manuscript's numbering.

1. Presentation and organization. The abstract and introduction now explain the method before introducing the analysis: retain Muon until a predefined quality target is confirmed, then fade its polar direction into fixed-gain momentum SGD, with copying feedback shortening the fade. Sections 2–4 state their purpose at the outset, and Section 4.2 separates the aligned diagnostic, component controls, exposure analysis and simpler alternatives. Figure 2 places the optimizer geometry in the main text; Figure E.1 presents the aligned diagnostic. Table O.1 defines the terminology and Algorithm 1 gives the controller. Appendix references include their appendix letters throughout.

2. Theory and evaluation. Section 2 now distinguishes the assumptions needed for gradient flow, aligned polar flow and the delay-recovery theorem. Table C.1 separates the finite-range aligned calculation from the optimizer clock summaries. Section 4.2 and Table J.6 connect four consequences of the analysis to controller interventions; Table J.7 reports quality and copying against cumulative polar exposure. Proposition 8 in Appendix E distinguishes scalar re-timing of the exact polar and gradient flows from the path change produced by their mixture in the specified aligned family. We do not extend that re-timing identity to arbitrary discrete stochastic optimizers.

3. Requested baseline comparisons. Table J.8 adds switches from Muon to SGD and AdamW with independently selected post-gate rates, gate-started cosine decay, and warning-based stopping. Under CBS, their final FIDs are 77.40, 78.83, 85.39 and 87.06, respectively, compared with 77.82 for the inherited-gain hard switch and 75.56 for QGD. The warning-stop comparator is audited at its stopping checkpoint and calibrated with the actual size of its truncated archive. Section 4.2 explains the resulting quality–delay tradeoff.

4. Resolution and capacity. Section 4.4, Appendix N and Table N.1 add a width-doubled UNet comparison using the same 10,000 CIFAR-10 images resized to 64 × 64. QGD and Muon share a 14.0k quality-arrival time; QGD extends the window from 70.4k to 255.2k and improves audit FID from 66.7 to 61.3. The hard switch has a longer window and lower copying. The text distinguishes this resolution/capacity comparison from evaluation on a new image distribution.

5. Statistical scope. Proposition 2 explicitly bounds the joint event of release and detector-budget violation. Development probes, calibration and final audits have distinct roles. Appendix P provides trajectory-level values and aggregation conventions, including censored crossings and within-trajectory windows.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.