acceptodds
Under review as a conference paper at ICLR 2027

On the Cross-Branch Correlation of Quantization Error in Classifier-Free Guidance

Abstract

Classifier-free guidance runs a network twice per step, with and without the condition, and extrapolates along the difference of the two predictions. Quantized models are judged per branch, yet the guided error equals , where and are the branch errors, is the guidance scale, and , the mean squared difference of the two errors, depends on a cross term that no per-branch metric sees. In six inference stacks, of the activation quantizers give the branches different scales or random numbers, and on these, to of the values held by both branches receive different codes. We propose PairRound, which lets the branches share what decides their codes: the scale, and the random numbers where rounding is stochastic. Codes then differ only where values differ, and a quadratic in states when the coarser grid that sharing gives one branch pays off, a condition our measurements meet. At matched bits and seeds, the shared scale lowers the trajectory distance to the BF16 sampler by to on eleven image, video and language backbones, improves FID, VBench or perplexity on every one, and composes with eight published methods and improves each of them.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
bit width

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_AYL3OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

The authors analyze how the error of the guided output in quantized models using classifier-free guidance (CFG) is affected not only by the individual quantization errors of the conditional and unconditional branches but also by the correlation between these errors. The authors decompose the guided error into the individual branch error terms and the mean squared difference between the two branch errors, which is amplified by s(1+s)s(1+s), where ss is the guidance scale. This highlights the importance of reducing the discrepancy between the quantization errors of the two branches. Based on this, the authors propose PairRound, which uses a common scale for the quantizer across the two branches and shares the random number when stochastic rounding is applied. The common scale is set to the maximum of the two branches' scales and does not require additional training or calibration. The authors evaluate trajectory distance to the BF16 sampler and generation quality metrics across 11 image, video, and language models, and present ablations on the combination with existing quantization methods, the guidance scale, the sharing granularity, and the scope of application.

Critical strengths

S1. Clear problem formulation Since CFG combines the outputs of the conditional and unconditional branches, the quantization error of the guided output depends not only on the individual branch errors but also on the correlation between them. The authors explicitly explain these issues through the cross-branch error correlation and the difference between branch errors, and demonstrate that the guided quantization error can vary significantly depending on the correlation between branch errors, even when their individual MSEs are similar. Although the basic decomposition is simple, the analysis of quantization scale sharing and rounding randomness coupling provides useful insights for the quantization design and evaluation of CFG models.

S2. Broad applicability The authors propose a simple and clear methodology that shares the quantization scale and rounding randomness of the two branches. It also offers high compatibility, as it can be applied additionally without significantly modifying existing quantization methods. The authors demonstrate consistent performance improvements across various image, video, and language generation models and quantization settings, as well as additional performance gains when PairRound is combined with existing quantization methods. These results support the practicality and broad applicability of the proposed method.

Critical weaknesses

W1. Limited connection between theoretical analysis and practical performance improvement The authors present Proposition 4 as a sufficient condition for the shared scale to reduce CFG quantization error. However, PairRound applies scale sharing to corresponding quantizers without using these conditions as an application criterion. Although performance improvements are observed across all models, the link between the theory and these improvements remains unclear:

  • The authors do not sufficiently analyze which layers or quantizers exhibit reduced CFG quantization error after scale sharing and which exhibit increased error.
  • The authors do not distinguish between cases where the sufficient conditions of Proposition 4 are met and cases where they are not.

Analyzing the change in error before and after applying the shared scale in each case would more clearly show how well the theoretically presented mechanism explains the actual performance improvement.

W2. Limited clarity and interpretability of experimental results Although the scope of the experiments is broad, the presentation of the results makes it difficult to identify the key findings and their supporting evidence.

  • Table 1 simultaneously presents theoretical validation using stochastic rounding, the evaluation of PairRound, and comparisons with existing PTQ methods. It also lists the trajectory distance for each backbone together with the normalized aggregate and the mean FID. As a result, the key results are difficult to understand intuitively.
  • The number and types of evaluated backbones differ across methods. It is therefore unclear whether the resulting aggregate scores and mean FID values provide a meaningful basis for fair comparison.
  • The appendix includes results for each backbone and detailed analyses of various settings. However, the relevant results are spread across multiple tables, which makes it difficult to find and synthesize the necessary evidence.

Categorizing the experiments by purpose, clearly linking the key conclusions of each analysis to the relevant tables, and summarizing results that are redundant with the main text would communicate the paper's contribution and empirical evidence more effectively.

Questions for authors

The authors are suggested to address the issues raised in the Weakness.

Reviewer_qHEUOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies quantization error across the conditional and unconditional branches of classifier-free guidance (CFG). It shows why each branch's error alone is insufficient to predict the guided error. PairRound shares quantization scales and, when applicable, random numbers between the branches. It needs no retraining or calibration. Tests on eleven image, video, and language backbones report 60%–68% lower trajectory distance, better generation quality, and compatibility with eight quantization methods.

Critical strengths

The diagnosis has direct experimental support. Proposition 1 separates the cross-branch error term. Figures 1–2 and Table 4 show that changing the rounding coupling can change guided error substantially even when each branch's error is similar. This explains why the two quantizers should be considered together. The central decomposition is correct; Propositions 2 and 4 also give valid results for fixed-marginal stochastic coupling and the condition for beneficial scale sharing.

A simple fix with broad tests and useful controls. PairRound changes the relation between quantizers without adding training or calibration. Its simple design is a strength. The paper explains the path from the error analysis to this change clearly.

The evaluation also checks when the method should do nothing. Table 1 covers eleven backbones, and Tables 2–3 examine compatibility and design choices. The no-change results for single-branch and already-shared setups in Figure 4c are particularly helpful. They test the proposed explanation as well as the average gains.

Critical weaknesses

Compatibility and reproducibility of the reported gains. Section 5 and Appendix B.4 use a common recipe with per-token dynamic activation scales. Seven published methods run at W4A8, while SVDQuant keeps its intended W4A4 setting. Their weight quantization and calibration are retained, and PairRound replaces the two activation scales with their maximum. However, Q-Diffusion, TFMQ-DM, and PTQ4DiT originally use calibrated or time-dependent parameters that need not follow this recipe.

A method that already uses one fixed activation scale for both branches should gain nothing from deterministic scale sharing. The reported 60%–85% reductions may therefore apply to adapted implementations. This is not necessarily unfair, but the paper needs to separate adaptation effects from improvements to each original method.

The definitions also need to be clearer. Section 4 and Algorithm 1 say per-coordinate sharing, while Section 5 describes per-token scales. The scale tensor, reduction axes, and pairing granularity need to be specified. Table 1 and Appendix B.3 also lack a complete formula for the normalized aggregate trajectory score. The language trajectory state, perplexity protocol, and judge setup need clearer definitions. Table 1 is dense, which makes these omissions harder to resolve.

The mathematical decomposition and shared-randomness principle have precedents. GAMP (arXiv:2607.08241) and DSAQuant (arXiv:2609.04031) overlap with parts of the diagnosis but use different fixes. I see the contribution mainly in applying this view to quantizer design and testing it across models.

The assumption in Proposition 3(b). On p.5, the proposition gives the nearest-rounding mismatch probability as p = |x|γ/((1+γ)σ_u), for p < 1. Appendix A.3 treats the interval length between normalized values as the probability of crossing a rounding boundary. Yet both the interval's position and its length depend on x. Uniform marginal phases do not establish the needed independence.

For example, take σ_u = 1, σ_c = 1.1, and x uniform on [-5.5, 5.5). Both normalized fractional parts are uniform. The actual mismatch probability is 5/22 ≈ 0.2273, but averaging the proposed formula gives 0.25. The equality seems to need a stronger conditional-phase assumption or an approximation qualifier. This affects the exact mismatch analysis, not Proposition 1 or the fact that equal inputs receive equal codes on a shared deterministic grid.

Overall assessment

I recommend 3/4 (Weak Accept), at the borderline. The error decomposition, tests of the mechanism, and broad reported gains support acceptance. The concerns above appear fixable, and the issue in Proposition 3(b) does not invalidate the central decomposition or the reported measurements. The authors should still clarify the native baseline settings, correct or qualify the proposition, and provide the missing definitions. I do not need more model families or benchmarks to assess these points.

Basis for confidence

I checked the mathematical arguments, experiments, and related quantization work. The concerns above are specific enough to check independently. I did not run the code or inspect experiment logs, so I cannot tell whether unclear reporting also reflects differences in the actual settings.

Questions for authors
  • For each published baseline, which activation settings are unchanged and which are replaced by dynamic per-token scales? Please distinguish results for original and adapted implementations. A representative unmodified dynamic-scale method and a fixed-scale no-op control would help.

  • What is the probability space in Proposition 3(b)? Please address the dependence of interval length and position on x, and revise the claim if it requires independent random phase alignment or an approximation.

  • Please give the scale tensor shapes, reduction axes, and pairing rule; the aggregate trajectory-score formula and normalization constants; and the language trajectory, perplexity, and judge protocols.

Reviewer_veSHOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

In this paper the authors propose an approach for effective post-training quantization of diffusion and language models that sample using classifier-free guidance. In classifier-free guidance, the model is run twice with and without conditioning, and the difference is amplified and added to the model’s prediction to push the sample further in the direction of the prompt. When quantization is applied in this setting, the quantization function (and corresponding scales) are calculated independently with respect to the two forward passes (which are treated as two distinct execution branches in dynamic quantization), then the relatively meaningless difference in the quantization errors is similarly upscaled and incorporated into the noise prediction. The authors demonstrate that 1) this is the case in 75% of activation quantizers used in guided sampling in software implementations from HuggingFace, TensorRT, vLLM, etc., which use dynamic quantization + classifier-free guidance and 2) this leads to deviation from the unquantized (BF16) sampler’s trajectory, and lower downstream sampled image quality.

To address this issue, the authors propose instead to share the quantizer scaling parameters and random numbers for stochastic rounding across the two branches, removing noise in the classifier-free guidance term that arises from just the mismatch between quantizations. They demonstrate that this change substantially reduces trajectory distance between the bf16 and quantized (W4A8) sampler across settings (seven image diffusion models, one video model, three LLMs) with corresponding improvements in FID, win rate, perplexity, etc.

Critical strengths
  • The paper identifies a relatively simple problem and corresponding solution in a common implementation of generative AI model inference.
  • The paper is extremely well motivated, including both a survey of actual implementations demonstrating that this is an issue, as well as a mathematical demonstration of the problem.
  • The proposed solution is computationally efficient and simple to implement requiring no calibration/training. It’s likely to be easily, widely deployed in practice to improve performance.
Critical weaknesses
  • The reported degradations in performance seem large relative to the literature. This may be artificially inflating the supposed claims of performance of the method, or at least mis-representing prior work as worse, and more subject to this issue, than it actually is. For example, for SDXL / 30 steps, the original SVDQuant paper reports: FP16 FID 16.6 -> INT W4A4 20.6 on MJHQ, while in this paper (appendix, Table 10) they report: FP16 FID 13.82 -> INT W4A4 25.53 (+11.71!). Similarly, Q-Diffusion replication results also seem quite different than what was reported in the paper. It’s unclear why the differences are so large, and further, this is sort of obscured by the way the results are reported (e.g. Table 1 mostly reports distances to the FP16 reference trajectory, while only reporting not-directly-comparable averages over actual end-task performance metrics; the end-task metrics are buried in the very long appendices).
  • It’s not clear to me whether just stochastic rounding + shared random numbers performs better than the proposed approach? In Table 1, we see that the “Stoch., shared rand.” line, which I believe is quantizers converted to use stochastic rounding + shared randomness, gets super low trajectory distance (lower than the proposed approach) while maintaining better avg end-task metrics. So, it seems better with respect to the main definition of better used in the paper. The first paragraph in 5.1 (main results) states that the relevant rows in Table 1 are (just?)“references, since they drop the deterministic rounding of 98.9% of the surveyed quantizers, which PairRound keeps”. But as far as I can tell, no explanation for why PairRound keeps the deterministic rounding is given. So it’s not really clear to me why PairRound is the proposed best approach vs. the stochastic rounding + fixed. This also seems to disagree fundamentally with Prop. 3.
  • Despite the problem and solution being relatively straightforward and easy to understand, the paper is very dense and not well written, making it harder to understand the approach (problem and solution) than it should be. Once I got it, it was clear, but the paper was not particularly helpful in explaining it. And per issues above, I may still be confused about some of the experimental results. While the paper including appendices is very long, some key experimental details seem to be missing. More below:
    • There’s a missed opportunity here for a figure clearly elucidating the problem here, not based in experimental results like the current figures (which are useful), but demonstrating the problem conceptually/visually, in terms of compounding errors arising from relatively small differences + different scaling factors leading to similar values getting quantized differently across the two branches. (And with differences in the induced grid coarseness.)
    • With respect to existing figures, e.g. Figure 1, a good rule of thumb (especially if you’re going with longer captions) is that the figure should be understandable basically on its own (with the caption). The opening sentence of the caption of figure 1: “The guided output separates couplings that per-branch errors cannot tell apart.” Is basically incomprehensible without context (and even the context of the paper never really clearly explains what that should mean, though I am able to infer it, with effort.) The subsequent sentence is a fragment. There are other parts of the paper also written using sentence fragments.
    • Terms used without being clearly defined, e.g. “couplings” throughout, referring to “codes” wrt quantization (I’m familiar with the terminology but it’s fairly standard to define the term wrt NN quantization).
    • In terms of missing experimental details: it’s not clear exactly where the “replicated” numbers come from, the results are not the same as published results so something must be different, but the exact experimental settings are not reported in order to compare. For the survey, it is also not clear exactly what was surveyed (i.e. which configuration was traced for each stack and when on what models and inputs were the mismatch rates measured). This experiment is not reproducible. The current status is enough to maybe convince me that the proposed issue is common in practice, but I’m not sure I can properly interpret the meaning of the percentages without more information.
Questions for authors
  • Why don’t the reproduced results from prior work match the published results? For each reproduced prior work, exactly what was kept from their method, what was replaced or changed in your implementation, and why?
  • What scaling factors (grids) do the stochastic rounding rows in Table 1 use, and more broadly why do these methods seem to work better than the proposed approach? If my interpretation of the results is correct, then why use the proposed approach over these?

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response 2: The problem, the fix and what is new, in plain terms

Dear AC and reviewers,

Thank you for the reviews and the discussion. As the discussion closes we restate in plain terms what the paper finds and what is new.

The problem. Classifier-free guidance runs the model twice, with and without the prompt, and extrapolates: the guided output is (1+s)(1+s) times the prompted output minus ss times the unprompted one. Of the 1,753 activation quantizers we surveyed in six inference stacks, 74.7% give each run its own dynamic scale, so the two runs round on slightly different grids. A value that is the same in both runs can then get two different codes: on these quantizers, 19% to 28% of the coordinates that hold the same value in both runs are rounded differently. Each run's error is small, but guidance takes the difference of the two runs and multiplies it. It is like measuring one board with two rulers whose marks are slightly offset: each reading is nearly right, but their difference mostly measures the offset, and guidance magnifies it.

What we find. The guided error has a part that no per-branch metric can see: the mismatch between the two runs' errors, which guidance multiplies by s(1+s)s(1+s) (Proposition 1). In practice it dominates. This mismatch alone accounts for 85% to 87% of the drift from the full-precision sampler (Figure 4a), and four rounding schemes whose per-branch errors lie within 8% of each other differ in their drift by up to 58 times (Figure 1).

What is new.

  1. A blind spot of per-branch quantization. Post-training quantization methods minimize the error of each run separately, but the guided sampler never uses a run alone. When two noisy outputs are averaged, their disagreement cancels, as the classical ambiguity decomposition shows; guidance extrapolates with a negative weight, so the same disagreement is amplified instead.
  2. A counter-intuitive trade-off. Sharing one scale, the larger of the two, puts the weaker run on a coarser grid, and its own error rises by 39% to 50%. A per-branch metric would reject this change. Yet the guided error falls by 61% to 67%. Propositions 3 and 4 and Corollary 1 explain when this pays, and show that the larger scale is the finest shared grid that clips neither run. Common random numbers, the classical idea behind sharing randomness, assume that each run's own error distribution stays fixed, so they cover stochastic rounding only.
  3. A fix at the source. PairRound makes the two runs round on one grid, and with the same random numbers where rounding is stochastic. The "Stoch., shared rand." row of Table 1 applies this random-number rule to a fully stochastic network; it is part of PairRound, not a competitor. PairRound needs no training, no calibration and no hyperparameter, keeps one bit width, keeps standard guidance at every step, and adds 12 µs per step and branch.

What it buys. On eleven image, video and language models, at matched bits and seeds, PairRound lowers the drift from the full-precision sampler by 60% to 68% and improves FID, VBench or perplexity on every one. It removes over 60% of the drift left by each of eight published quantization methods. On the methods that already use one fixed scale for both runs, it changes nothing, as the analysis predicts (Table R1).

What is classical, and what is new. Two concurrent preprints, GAMP and DSAQuant, also observe that guidance hurts quantized models. Both keep a separate quantizer per run and compensate around it: GAMP with calibration and mixed precision, DSAQuant with staged quantization-aware training and by turning guidance off late in sampling. PairRound removes the mismatch where it starts. The analysis builds on classical results, cited where they are used. What is new is the blind spot they reveal in guided sampling, the trade-off that a shared scale creates, and a fix that needs nothing but the shared grid. In the terms of the ICLR reviewer guidelines, this is new knowledge, theoretical and empirical, with a direct use for practitioners. Reviewer veSH finds the paper "extremely well motivated" and the fix "likely to be easily, widely deployed in practice", Reviewer qHEU confirms that "The central decomposition is correct", and Reviewer AYL3 finds that the analysis "provides useful insights for the quantization design and evaluation of CFG models".

We thank the AC and the three reviewers again for their time and careful reading.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.