- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates numerical instability in classifier-free guidance (CFG) when combined with coarse DDIM sampling. The authors show that strong guidance can cause sign reversals and residual amplification along discriminative directions, potentially degrading generation quality.
The main contributions are:
- A theoretical analysis identifying instability thresholds and stable coefficient ranges under an ordered, commuting Gaussian model.
- Fitted CFG, a modified guidance coefficient derived by matching the exact contraction of a terminal Gaussian mode, without additional denoiser evaluations.
- A horizon-independent bound on accumulated relative contraction error under the stated Gaussian assumptions.
- A finite-scale crossover extension with a calibrated parameter.
Experiments on CIFAR-10 demonstrate improved FID and reduced residual amplification. On SDXL, the basic Fitted CFG method exhibits mixed results, while the calibrated crossover variant provides modest improvements in FID and KID with reduced text alignment.
Critical strengths
- Principled theoretical analysis of CFG instability (Sections 2-3).
The paper provides a useful mathematical explanation of why coarse DDIM steps under strong guidance can cause coordinate sign reversals and residual amplification. The distinction between stability, consistency, and contraction accuracy is meaningful, and Theorems 1-3 provide a rigorous framework for understanding these effects under the stated assumptions. This is the paper's strongest contribution.
- Simple implementation with carefully controlled experiments (Sections 3-4).
Fitted CFG requires only a modification of the guidance coefficient, without additional denoiser evaluations or retraining. The authors also provide substantial experimental controls, including matched seeds, different guidance strengths and sampling steps, closure comparisons, and local response analyses. The CIFAR-10 results show a notable FID improvement from 27.87 to 20.88, while the SDXL local-response experiments provide evidence of the predicted numerical instability.
Critical weaknesses
- Limited algorithmic novelty and unclear advantage over existing methods (Sections 3.2, 4.1, and 5).
The proposed update belongs to the existing deterministic CFG++ family and can also be expressed as standard CFG with an effective guidance coefficient. Therefore, the main algorithmic contribution is a particular coefficient selection rather than a fundamentally new sampling mechanism.
More importantly, the experiments do not convincingly demonstrate that contraction matching provides a practical advantage over simpler stabilization strategies. In Section 4.1, the sign cap achieves almost identical CIFAR-10 performance (FID 24.735 versus 24.733), and Table 4 reports nearly identical local-response improvements for the two methods.
Although the theoretical contraction-accuracy guarantee is novel and potentially valuable, its practical importance remains insufficiently demonstrated. The authors should establish settings in which contraction matching provides a measurable advantage over simpler clipping rules and stronger existing CFG-based methods.
- Restrictive theoretical assumptions and inconsistent empirical improvements (Sections 3-4).
The theoretical guarantees rely on ordered, commuting Gaussian covariance matrices and exact Gaussian denoisers. These assumptions are restrictive and do not directly establish stability or contraction accuracy for nonlinear learned diffusion models.
Furthermore, the empirical improvements are not consistent across evaluation metrics and models. On CIFAR-10 (Table 2), Fitted CFG improves FID but worsens KID from 0.0108 to 0.0177. On SDXL (Table 3), the basic Fitted CFG method worsens FID from 19.489 to 23.178, while also reducing CLIP alignment. The calibrated crossover extension improves FID only modestly (19.489 to 19.006), while still reducing text alignment.
These results suggest that stabilization and contraction matching do not necessarily translate into better overall generation quality. Stronger evidence of generalization and more favorable quality-alignment trade-offs would substantially strengthen the paper.
Questions for authors
-
Given that Fitted CFG belongs to the existing CFG++ update family, what practical benefit does the proposed contraction-matching coefficient provide beyond the simpler sign cap? Can the authors identify experimental settings where the two methods differ meaningfully in generation quality or numerical accuracy?
-
How applicable are the theoretical stability and relative-error guarantees to realistic learned diffusion models, where the Gaussian and commuting-covariance assumptions generally do not hold? Can the authors provide additional evidence connecting the theoretical contraction-matching property to empirical performance?
-
How do the authors explain the deterioration in KID on CIFAR-10 and the substantially worse FID of basic Fitted CFG on SDXL? Can they demonstrate a consistent advantage over tuned CFG, CFG++, and sign-cap alternatives when both image quality and conditioning strength are considered, under matched evaluation settings?
3 comments
Thank you for the detailed response and additional experiments. I appreciate the authors' efforts, particularly the new results demonstrating an advantage over the sign cap. However, some concerns remain:
- The simpler CFG baseline at g=2 still achieves substantially better results.
- Fitted CFG shows larger endpoint errors than the sign cap in 161 of 167 cases, raising questions about the practical relevance of its theoretical guarantees.
- The SDXL trade-offs remain, and the held-out comparisons with stronger baselines are still pending.
I appreciate the additional evidence and will carefully revisit the rebuttal and results to reconsider whether my original assessment and score should change.
Thank you for the thoughtful follow-up. We have now completed the further experiments. They show improved generation metrics at the CIFAR setting, better terminal diagnostics on the same endpoint-error cases, and a reproducible SDXL operating point against stronger baselines at .
1. Comparison with CFG at . We compared both schemes at on two fresh, independent 5k blocks, with paired seeds, the same mesh, 134 denoiser-branch evaluations, and matched closure.
| Method | FID, run 1 | FID, run 2 | Pooled KID | Pooled target accuracy | |
|---|---|---|---|---|---|
| CFG | 2 | 10.349 | 10.684 | 0.002577 | 99.26% |
| Fitted | 2 | 9.697 | 10.032 | 0.002268 | 99.15% |
Fitted improves FID and KID in both runs. Its pooled KID difference is (95% paired/shared-reference interval ). The target-accuracy difference is percentage points (95% paired interval ). At , fitted lowers FID and KID in both held-out runs, with 99.15% target accuracy and unchanged sampling cost.
2. Practical relevance on the endpoint-error cases. Theorem 3 motivates matching through relative-contraction accuracy in the ordered Gaussian model. To examine practical usefulness directly, we evaluated terminal outputs on the same 167 accepted cases underlying your endpoint-error comparison, at with matched closure:
| Endpoint | Target accuracy | Median pre-clamp out-of-range fraction |
|---|---|---|
| Converged reference | 81.44% | 37.43% |
| Sign cap | 94.61% | 10.81% |
| Fitted | 98.80% | 2.31% |
The out-of-range fraction counts scalar image values outside . Both reference integrators agree on all 167 target-classification outcomes. On these same cases, fitted combines the larger endpoint distances you noted with higher target accuracy and substantially less out-of-range output. Distance to the learned-field endpoint and terminal performance therefore rank the methods differently.
3. Completed SDXL comparisons with stronger baselines. At , where all six methods met the calibration CLIP floor, we evaluated each on two independent 5k blocks. Parameters were fixed using separate calibration data, with an equal five-candidate budget and KID selection subject to the CLIP floor. Methods share prompts, seeds, arithmetic and 60 denoiser-branch evaluations. Pooled 10k results:
| Method | Parameter | FID | KID | CLIP |
|---|---|---|---|---|
| CFG | 22.033 | 0.006677 | 0.31828 | |
| Fitted | 21.653 | 0.006341 | 0.31726 | |
| Sign cap | 22.012 | 0.006523 | 0.31800 | |
| Crossover | 21.782 | 0.006676 | 0.31769 | |
| CFG++ | 22.280 | 0.006584 | 0.31835 | |
| DPM-Solver++ | 21.633 | 0.006576 | 0.31741 |
The table uses native closures: for CFG/DPM-Solver++ and for the others. Fitted improves FID/KID over CFG and the sign cap in both blocks; these improvements also hold with matched closure. Against CFG, its KID difference is (pointwise 95% interval ), and CLIP changes by (), within the prespecified 0.002 tolerance. Fitted achieves the lowest point KID among the six selected settings and FID close to DPM-Solver++. This provides held-out evidence for fitted's practical use in a latent diffusion model at the 30-node budget.
Thank you for asking where contraction matching makes a measurable difference. We added a CIFAR comparison in which the fitted and capped coefficients separate.
Q1. Benefit beyond the sign cap. The stabilization band leaves the contraction magnitude free. Terminal matching selects βfit=1−r1+w within deterministic CFG++, and Theorem 3 bounds its accumulated log-relative error independently of the noise horizon under the stated Gaussian assumptions.
For the new CIFAR test, we chose r=0.85 and nominal g=9 before sampling. The pure-mode factors on untruncated steps are −0.35, 0 and 0.231617 for CFG, cap and fitted. We used two independent balanced 5k runs. Within each run, methods share checkpoints, initial noise, labels, 134 branch evaluations and conditional closure. Entries are run 1 / run 2; the g=2 row transfers the earlier calibration unchanged to this mesh.
Fitted improves all three metrics over cap in both runs. The accuracy gain is 2.17 percentage points (95% paired bootstrap interval [1.75, 2.59]); amplification p95 falls from 2.239 / 2.125 to 1.463 / 1.465. The paired guided closure preserves the FID/KID ordering. This gives an empirical reason to use the matching rule: it selects a coefficient with a Gaussian accuracy guarantee and delivers joint quality and conditioning gains over the cap in this setting, with no extra denoiser evaluations.
Q2. Applicability to learned denoisers. The denoiser-error margin can be quantified. With s=r1+w, Dc=ecx and Du=(1+eu)x, the fitted factor is s+(1−s)ec+(s−r)eu. It remains positive if (1−s)∣ec∣+(r−s)∣eu∣<s, so strong contraction can leave a small margin.
We also integrated 200 prespecified CIFAR cases against the same learned field. Of these, 167 pass two successive refinement checks and RK4 agreement at 10−4; 33 remain unresolved. At the last positive-noise node, median normalized endpoint errors on the accepted cases are 0.3782 / 0.5956 / 0.6832 for CFG / cap / fitted. Fitted has smaller error than cap in 6 cases and larger error in 161.
Q3. Metric disagreement and tuned alternatives. In the original 50k CIFAR comparison, FID falls from 27.8706 to 20.8797 and KID rises from 0.010827 to 0.017734. FID measures feature means and covariances through Gaussian approximations; KID uses a cubic-kernel discrepancy. They can rank distributions differently. The new comparison above improves both metrics over cap in both runs.
On SDXL, the reported trade-off persists with FP32 persistent states. On 5,000 archived pairs at g=12,N=12, crossover has FID 27.1793 versus CFG's 27.5896. The KID difference is -0.0002694 (95% interval [-0.0004034, -0.0001248]), with CLIP difference -0.0030005 ([-0.0033917, -0.0025958]). These use native closures, Dc for crossover and Dw for CFG. The cap's KID/CLIP advantage over fitted also persists in the first 1,000 pairs under stable arithmetic.
We evaluated CFG, fitted, cap, crossover, CFG++ and DPM-Solver++ on 256 calibration pairs at N=12,30, selecting by KID subject to CLIP being at least the CFG g=7.5 value minus 0.002. All six families have an eligible point at N=30; at N=12, only CFG and DPM-Solver++ meet the floor. Held-out evaluation of these selected settings is pending.