- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2(+1)
- Soundness
- 2
Read review
Paper summary
The authors analyze how the error of the guided output in quantized models using classifier-free guidance (CFG) is affected not only by the individual quantization errors of the conditional and unconditional branches but also by the correlation between these errors. The authors decompose the guided error into the individual branch error terms and the mean squared difference between the two branch errors, which is amplified by , where is the guidance scale. This highlights the importance of reducing the discrepancy between the quantization errors of the two branches. Based on this, the authors propose PairRound, which uses a common scale for the quantizer across the two branches and shares the random number when stochastic rounding is applied. The common scale is set to the maximum of the two branches' scales and does not require additional training or calibration. The authors evaluate trajectory distance to the BF16 sampler and generation quality metrics across 11 image, video, and language models, and present ablations on the combination with existing quantization methods, the guidance scale, the sharing granularity, and the scope of application.
Critical strengths
S1. Clear problem formulation Since CFG combines the outputs of the conditional and unconditional branches, the quantization error of the guided output depends not only on the individual branch errors but also on the correlation between them. The authors explicitly explain these issues through the cross-branch error correlation and the difference between branch errors, and demonstrate that the guided quantization error can vary significantly depending on the correlation between branch errors, even when their individual MSEs are similar. Although the basic decomposition is simple, the analysis of quantization scale sharing and rounding randomness coupling provides useful insights for the quantization design and evaluation of CFG models.
S2. Broad applicability The authors propose a simple and clear methodology that shares the quantization scale and rounding randomness of the two branches. It also offers high compatibility, as it can be applied additionally without significantly modifying existing quantization methods. The authors demonstrate consistent performance improvements across various image, video, and language generation models and quantization settings, as well as additional performance gains when PairRound is combined with existing quantization methods. These results support the practicality and broad applicability of the proposed method.
Critical weaknesses
W1. Limited connection between theoretical analysis and practical performance improvement The authors present Proposition 4 as a sufficient condition for the shared scale to reduce CFG quantization error. However, PairRound applies scale sharing to corresponding quantizers without using these conditions as an application criterion. Although performance improvements are observed across all models, the link between the theory and these improvements remains unclear:
- The authors do not sufficiently analyze which layers or quantizers exhibit reduced CFG quantization error after scale sharing and which exhibit increased error.
- The authors do not distinguish between cases where the sufficient conditions of Proposition 4 are met and cases where they are not.
Analyzing the change in error before and after applying the shared scale in each case would more clearly show how well the theoretically presented mechanism explains the actual performance improvement.
W2. Limited clarity and interpretability of experimental results Although the scope of the experiments is broad, the presentation of the results makes it difficult to identify the key findings and their supporting evidence.
- Table 1 simultaneously presents theoretical validation using stochastic rounding, the evaluation of PairRound, and comparisons with existing PTQ methods. It also lists the trajectory distance for each backbone together with the normalized aggregate and the mean FID. As a result, the key results are difficult to understand intuitively.
- The number and types of evaluated backbones differ across methods. It is therefore unclear whether the resulting aggregate scores and mean FID values provide a meaningful basis for fair comparison.
- The appendix includes results for each backbone and detailed analyses of various settings. However, the relevant results are spread across multiple tables, which makes it difficult to find and synthesize the necessary evidence.
Categorizing the experiments by purpose, clearly linking the key conclusions of each analysis to the relevant tables, and summarizing results that are redundant with the main text would communicate the paper's contribution and empirical evidence more effectively.
Questions for authors
The authors are suggested to address the issues raised in the Weakness.
General Response 2: The problem, the fix and what is new, in plain terms
Dear AC and reviewers,
Thank you for the reviews and the discussion. As the discussion closes we restate in plain terms what the paper finds and what is new.
The problem. Classifier-free guidance runs the model twice, with and without the prompt, and extrapolates: the guided output is (1+s) times the prompted output minus s times the unprompted one. Of the 1,753 activation quantizers we surveyed in six inference stacks, 74.7% give each run its own dynamic scale, so the two runs round on slightly different grids. A value that is the same in both runs can then get two different codes: on these quantizers, 19% to 28% of the coordinates that hold the same value in both runs are rounded differently. Each run's error is small, but guidance takes the difference of the two runs and multiplies it. It is like measuring one board with two rulers whose marks are slightly offset: each reading is nearly right, but their difference mostly measures the offset, and guidance magnifies it.
What we find. The guided error has a part that no per-branch metric can see: the mismatch between the two runs' errors, which guidance multiplies by s(1+s) (Proposition 1). In practice it dominates. This mismatch alone accounts for 85% to 87% of the drift from the full-precision sampler (Figure 4a), and four rounding schemes whose per-branch errors lie within 8% of each other differ in their drift by up to 58 times (Figure 1).
What is new.
What it buys. On eleven image, video and language models, at matched bits and seeds, PairRound lowers the drift from the full-precision sampler by 60% to 68% and improves FID, VBench or perplexity on every one. It removes over 60% of the drift left by each of eight published quantization methods. On the methods that already use one fixed scale for both runs, it changes nothing, as the analysis predicts (Table R1).
What is classical, and what is new. Two concurrent preprints, GAMP and DSAQuant, also observe that guidance hurts quantized models. Both keep a separate quantizer per run and compensate around it: GAMP with calibration and mixed precision, DSAQuant with staged quantization-aware training and by turning guidance off late in sampling. PairRound removes the mismatch where it starts. The analysis builds on classical results, cited where they are used. What is new is the blind spot they reveal in guided sampling, the trade-off that a shared scale creates, and a fix that needs nothing but the shared grid. In the terms of the ICLR reviewer guidelines, this is new knowledge, theoretical and empirical, with a direct use for practitioners. Reviewer veSH finds the paper "extremely well motivated" and the fix "likely to be easily, widely deployed in practice", Reviewer qHEU confirms that "The central decomposition is correct", and Reviewer AYL3 finds that the analysis "provides useful insights for the quantization design and evaluation of CFG models".
We thank the AC and the three reviewers again for their time and careful reading.