- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how to optimize updated geometries actually affects the window between it about the useful sample quality and copy training data set in diffusion models.
Critical strengths
This paper provided a clear mechanistic frame. It also gives a careful experimental hygiene. There is also an informative ablation study to do the experiment.
Critical weaknesses
Seems like the practical benefit is small and rests on narrow evidence as the paper shows the final FID from 75.56 vs 79.37 is a modest gain from the regimen of the FID 75 - 85 and I do think there is a big gap remain from the theoretical to practice gap
Questions for authors
How does QGD compare to Muon to SGD/AdamW hard switch with separately tuned LR after the gate, and to Muon with a simple LR decay or early stop at the first copy warning?
Response to Reviewer RyMD
Thank you for the positive assessment and for naming concrete alternatives to the gradual transition.
W. Size of the benefit and the theory–practice gap. The main comparison now separates quality arrival, the observed low-copy interval, common-checkpoint quality, and quality after checkpoint selection. On the 2,000-image setting, QGD shares Muon's 8.0k quality arrival, compared with 16.0k for SGD, and extends Muon's 20.0k window to 166.4k (Table 1). Under the same CBS rule, final FID is 75.56 for QGD, 79.37 for SGD, 81.94 for AdamW and 86.14 for Muon (Table 3). Table J.4 reports the differences within paired trajectories.
The comparisons also expose the tradeoff. Conservative alternatives can have lower copying and longer windows. QGD's quality advantage persists at the common checkpoint for n=10,000 (FID 56.1 versus 61.0 for the best baseline in Table 1), flow matching (79.8 versus 82.8), and dance generation (kinetic FID 27.9 versus 30.2; Table 4). Appendix I evaluates an additional copying detector. Table N.1 adds the resolution/capacity setting, where QGD has FID 61.3 versus 64.2 for the hard switch, while the hard switch copies less.
The theory does not numerically predict these neural outcomes. Section 2 states its fixed-feature and alignment conditions; Section 4.2 tests qualitative consequences using matched controls. This distinction is retained throughout the revision.
Q. Tuned hard switching, learning-rate decay and early stopping. We added these comparisons to Section 4.2 and Table J.8. Both switches share Muon's prefix through the confirmed gate. Post-gate rates are selected by independent pilots from multipliers {0.5,1,2,4}: the selected multipliers are 2 for the inherited SGD gain and 1 for the common AdamW matrix rate of 2×10−4. SGD retains its momentum buffer, whereas AdamW initializes its matrix-parameter moments at the switch. The decay comparator begins cosine decay at the gate and reaches zero at 200k.
The table reports means. Common-checkpoint audits are at 60k, except warning stopping, whose audit is at its stopping checkpoint. The full manuscript provides the standard deviations.
Tuning improves the SGD switch's released FID from 77.82 to 77.40 while shortening its window from 186.0k to 143.6k. QGD retains a longer window than that tuned switch and a lower released FID, while the tuned switch has lower audit copying. Warning stopping is conservative but ends soon after the quality gate: the mean stopping time is 10.8k and the mean observed window is 2.8k. Its truncated archive uses its own candidate count for CBS, so it is not penalized by the full archive's multiplicity allocation.
Proposition 8 in Appendix E supplies the mathematical comparison. In exact aligned polar flow, a scalar rate schedule re-parameterizes cumulative polar time; after a hard switch, a scalar gradient-rate schedule re-parameterizes the gradient path. The mixture leaves both pure-flow curves locally in the specified diagonal family. These statements do not establish a universal performance ordering, and stochastic momentum updates need not obey the same re-timing identity.
Similarly, an early-stopped model is a checkpoint of its continued trajectory, but that does not make CBS on the full archive automatically superior: the candidate family and calibration allocation also change. Table J.8 therefore compares the actual release procedures directly. The theory identifies why the baselines are relevant, and the evaluation determines their quality–copying tradeoffs under the stated protocol.