- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper introduces Guidance-Correlation Branch Transform (GCBT), a lightweight activation transform for post-training quantization of diffusion models with classifier-free guidance (CFG). The key observation is that the conditional and unconditional CFG activations form a strongly correlated two-dimensional source, whereas existing activation quantization methods typically quantize the two branches independently in their native coordinates. The paper formulates the conditional/unconditional pair as a two-dimensional branch space and applies a 2x2 orthogonal transform before activation quantization, followed by the inverse transform afterward. To determine the transform, the authors propose a guidance-weighted distortion surrogate that combines the per-layer second-moment matrix of matched CFG activations with the known CFG readout direction. Under an equal-rate quantization-noise assumption, the optimal rotation admits a closed-form solution, requiring neither gradient-based optimization nor angle search.
Critical strengths
- The method is simple, interpretable, and mathematically elegant.
- This paper provises good compatibility with existing PTQ methods.
- The broad and relatively careful empirical validation.
Critical weaknesses
- My main technical concern is the connection between the closed-form surrogate optimum and the actual quantizer optimum is insufficiently validated. The theoretical result assumes that the quantization-error variance of each transformed coordinate is proportional to its projected second moment, uses the same proportionality constant for both coordinates, and neglects cross-coordinate quantization-error terms.
- The use of the final CFG direction as a layer-local sensitivity measure deserves stronger justification.
- GCBT requires matched conditional/unconditional activations and therefore assumes active two-branch CFG. It does not currently cover distilled guidance, single-pass CFG variants, or dynamic/timestep-varying CFG schedules, as acknowledged by the authors.
- The CFG-aware baseline is informative but not a definitive comparison to prior CFG-aware quantization.
Questions for authors
- What happens when the conditional and unconditional branches are executed sequentially rather than jointly?
- Can the authors better characterize when GCBT is expected to help?
7 comments
W3 — Applicability to CFG Variants.
GCBT targets models with active two-branch CFG, where matched conditional and unconditional activations are required for branch-space rotation. While GCBT cannot be directly applied under an unchanged full-pass sequential execution schedule, a layer-wise interleaved schedule enables exact GCBT computation with near-sequential memory usage. Guidance-distilled and genuinely single-branch models remain outside its current scope. We validate this implementation and compatibility with dynamic CFG below.
(1) Sequential CFG Execution. We implement a layer-wise interleaved schedule that enables GCBT to operate with near-sequential memory usage. We temporarily buffer the unconditional activation until the corresponding conditional activation becomes available, reusing the buffer across layers. On PixArt-Σ W4A4, this implementation produces bitwise-identical outputs to joint GCBT, with only 53.3 MiB (3.7%) additional peak memory compared with Native full-pass sequential execution (Table R7).
Table R7. Peak denoiser memory and latency under sequential and interleaved CFG execution on PixArt-Σ/SVDQuant W4A4. Latency is the median of 20 runs per method on the same L40S GPU.
| Method | Peak denoiser memory ↓ | Latency ↓ |
|---|---|---|
| Native sequential | 1453.5 MiB | 5.726 s/image |
| Native interleaved | 1507.8 MiB | 5.827 s/image |
| GCBT interleaved | 1506.8 MiB | 5.712 s/image |
Under the same interleaved schedule, GCBT achieves nearly identical memory usage and latency to Native, demonstrating minimal additional execution overhead.
(2) Compatibility with Dynamic CFG. As demonstrated in Table R2 (Reviewer wcpw, W2, Q4), GCBT supports timestep-varying CFG by updating its closed-form rotation using the current scale . On CogVideoX, timestep-adaptive GCBT improves LPIPS from 0.445 to 0.409 and SSIM from 0.689 to 0.699 compared with Native.
W4 & Q2 — Distinction from Prior CFG-Aware Methods and When GCBT Helps.
(1) Difference from prior CFG-aware methods (W4). To the best of our knowledge, GAMP is the most directly related prior work on CFG-aware quantization. GAMP focuses on guided-output calibration and mixed-precision allocation, whereas GCBT introduces CFG branch-space transform coding and a guidance-aware, quantizer-independent objective for selecting the coding basis. These approaches address different aspects of quantization and are therefore complementary. Since GAMP has no public implementation, we evaluate GCBT on top of GAMP-inspired calibration under the same SVDQuant host. As shown in Table 1, GCBT further reduces LPIPS by 2.97% on MJHQ and 2.47% on sDCI relative to this calibration baseline, demonstrating additional benefits from branch-space coding.
(2) When GCBT Helps (Q2). GCBT is particularly beneficial when activation quantization error remains substantial and CFG branch correlations can be exploited. As shown in Table R3 (Reviewer wcpw, Q3), even fixed +45° and PCA/KLT rotations improve over Native, supporting the effectiveness of branch-space coding itself. Our Q-DiT/SANA A4/A8 experiments (Table R1) further show that GCBT substantially reduces local quantization error under both bitwidths, but denoiser-output MSE improves by 78.55% under A4 versus only 4.30% under A8, where the baseline is already close to the W4-only reference. These results help explain why GCBT can yield smaller output-level gains at higher activation precision.
Thank you again for your careful review and constructive feedback. We have added experiments to validate the connection between GCBT's closed-form optimum and the actual quantization-error optimum, justify the use of the final CFG direction as a layer-local sensitivity measure, and evaluate sequential execution and dynamic CFG. We also clarified GCBT's scope, its distinction from prior CFG-aware quantization methods, and when it is most effective.
We hope these additional results and clarifications help address your main concerns, and would welcome any further feedback or questions!
We appreciate the reviewer's careful reading and constructive feedback. We address each point below:
W1 — Validation of Eq. (6)’s Proportionality and Cross-Coordinate Error Term Assumptions.
We have clarified the assumptions behind Eq. (6) in the revised manuscript (highlighted in blue) and added empirical validation under the original quantization settings. Below, we summarize our findings and addressing for each assumption.
(1) Proportionality assumption. GCBT uses projected activation second moments to approximate quantization error. We test this assumption by fitting a single coefficient κℓ per site, shared across both transformed coordinates and all rotation angles. On held-out samples across 360 angles, the predicted quantization-error second moments closely match the measured values, with a median Pearson r of 0.999, median R2 of 0.999, and median/P90 normalized RMSE of 1.3%/4.3%. In total, 152/168 sites have an error below 5%. These results show that the proportionality assumption holds well at most sites.
(2) Cross-coordinate error terms. We further examine whether ignoring cross-coordinate error terms affects rotation selection. We find that directly minimizing the diagonal-only error Ddiag and the full error Dfull produces nearly identical angles at most sites: the median/P90 difference is 0.05°/2.72°, with only 5/168 sites differing by at least 5°. The resulting difference in held-out Dfull is just +0.129% (95% CI: [−0.006%, +0.254%]). Thus, ignoring these terms has little effect on rotation selection or actual quantization error.
Overall, these results support both approximations used by GCBT. Additional experimental results are provided in our response to Reviewer wcpw (W1, Q1–Q2).
W2 — Validation of the Final CFG Direction for Layer-Local Rotation Selection.
We validate using the final CFG direction for layer-local rotation selection by testing (1) whether local quantization error reflects final denoiser-output error and (2) whether CFG weighting improves rotation selection.
(1) Local-to-output correspondence. On SANA-1.6B, we compare CFG-weighted local quantization error with final denoiser-output error across candidate rotations. The two errors show strong agreement, with a median Spearman correlation of 0.863 and 80.85% pairwise ranking agreement. We further compare rotations selected by minimizing local error against those selected by directly minimizing final output error under the same CFG scale.
Table R5. Fraction of the oracle improvement in denoiser-output MSE recovered by local rotation selection on SANA-1.6B.
As shown in Table R5, local CFG-weighted selection recovers 90.92% of the improvement achievable by directly optimizing final output error. This shows that local error provides a reliable guide for rotation selection without requiring downstream output evaluations.
(2) Importance of CFG weighting. We test whether CFG weighting improves rotation selection over CFG-agnostic alternatives. On SANA-1.6B, we compare three local objectives: (i) equal weighting of the two branch errors, (ii) separate branch errors, and (iii) CFG-weighted branch errors. All three select rotations at six sites using leave-one-prompt-out evaluation. We then measure final denoiser-output MSE under the same CFG scale of 4.5 (Table R6).
Table R6. Final denoiser-output MSE under different local rotation-selection objectives on SANA-1.6B (CFG = 4.5), normalized to Native.
CFG weighting reduces final output MSE by 24.8% and 23.2% compared with the two CFG-agnostic objectives, respectively. This advantage persists on a fixed angle grid that excludes all GCBT-calibrated rotations.
Overall, these results support using the final CFG direction for layer-local rotation selection: local CFG-weighted error closely reflects final output error, and CFG weighting yields better rotations than CFG-agnostic alternatives.