acceptodds
Under review as a conference paper at ICLR 2027

When Timing Matters: Interleaved Cross-Block Post-Training Quantization

Abstract

Block-wise post-training quantization is making low bit-width compression of large language models increasingly feasible, especially under strict memory budgets. However, PTQ still lacks the global optimization view of quantization-aware training. While most recent PTQ work focuses on block-level improvements, cross-block variants use a moving window to capture inter-block dependencies, offering a step toward global behavior. We go further and propose Interleaved Cross-Block Quantization (ICBQ), a scheduling modification that interleaves short cross-block refinements with block-wise quantization so that errors are corrected before they propagate further, offering a new route toward more global optimization. The interleaved schedule naturally creates boundary overlap, which we analyze separately as a secondary effect. In the reported experiments, ICBQ reduces ternary-quantization perplexity relative to a matched Sequential CBQ baseline, yields finite perplexity in configurations where the baseline has severe degradation, and also transfers to 3-bit and 2-bit GPTQ.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Post-training quantization, large language models, weight-only quantization, low-bit inference

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_t3gbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies optimization scheduling in low-bit post-training quantization and proposes ICBQ, which inserts a short reconstruction sweep over pairs of adjacent Transformer blocks after each chunk of blocks is quantized and recomputes activations through the current prefix. The method also includes optional seam revisiting at chunk boundaries. Using ternary DBF and GPTQ as inner quantizers, the authors report perplexity and zero-shot accuracy across multiple language models and examine the effects of chunk size, seam revisiting, and prefit through ablation studies. The theoretical analysis derives recurrence-based error upper bounds under assumptions of a local representation-error envelope, per-sample midpoint contraction, and exact activation re-roll.

Critical strengths
  1. Simple and adaptable scheduling. Earlier correction and activation recomputation improve subsequent quantization inputs while preserving the two-block objective. Experiments with ternary DBF and GPTQ show applicability beyond a single weight representation.
  2. Fairly comprehensive evaluation. Multi-model results, seam/chunk-size/prefit ablations, and wall-clock timings reveal both benefits and limitations. These results could support a useful empirical contribution if the concerns below are addressed.
Critical weaknesses
  1. The current full theoretical bound does not support the emphasized geometric benefit of seam revisiting. Let A=(1−τL−1)/(1−τ)A=(1-\tau^{L-1})/(1-\tau). Substituting Eqs. (22) and (24) into Eq. (23) gives BLint=[1+ρ(1+γ+γ2)A]ϵsubB_L^{\mathrm{int}}=[1+\rho(1+\gamma+\gamma^2)A]\epsilon_{\mathrm{sub}}, with γS\gamma^S canceling completely. With shared constants fixed, this displayed bound is independent of KK and exceeds BLseqB_L^{\mathrm{seq}}. I acknowledge the tighter bounds discussed in Remark B.6 and Proposition B.15. However, with the explicit residual in Eq. (24), the main corollary yields no informative geometric gain, and the paper does not use the tighter residual to establish nontrivial conditions for an improvement in the full bound. The aggregate loss ratios in Appendix B.11 do not provide the per-sample midpoint certificate required by Assumption B.2 and therefore do not close this gap.
  2. The empirical gains are insufficiently contextualized in terms of computational cost and compression ratio. Full ICBQ uses more pair updates and activation re-rolls, taking approximately 21% longer on average, yet no sequential baseline with the same total optimization budget is provided. The dramatic improvement on Qwen3-8B also depends strongly on prefit (Table 9). Moreover, DBF's two ternary factors and scaling parameters mean that "1.58-bit" does not directly translate into effective bits per original weight. The paper does not report kk, actual storage requirements, or comparisons at matched compression ratios. These gaps make it difficult to assess the method's practical value.
Questions for authors
  1. Is the above simplification of Eqs. (22)–(24) correct? Using Proposition B.15 or another full upper bound, could you provide explicit, satisfiable conditions for BLint<BLseqB_L^{\mathrm{int}}<B_L^{\mathrm{seq}}, together with a numerical example, and explain how the main corollary's geometric gain holds under these conditions?
  2. Could you report per-sample midpoint errors, local representation envelopes, and Lipschitz diagnostics for the actual CBQ calls, rather than only aggregate MSE ratios at the two-block output? With seam revisiting enabled, how are the relevant assumptions verified when the first boundary update uses an FP16 neighboring block?
  3. Under the same total optimization-step or wall-clock budget, how does ICBQ compare with a baseline that allocates more budget to sequential refinement? Please clarify the exact prefit objective and implementation in light of Table 9, and report the per-layer kk values and actual model storage size for ternary DBF.
4 comments

Thank you for recognizing the scheduling contribution and empirical evaluation, and for carefully examining the theory.

Weakness 1 / Question 1: Theoretical bound. Your simplification is correct. Substituting Eqs. (22) and (24) into Eq. (23) removes the seam-count dependence from the displayed full bound. It therefore does not establish the claimed geometric improvement over the sequential bound. We will correct the interpretation of Corollary 4.2 and the associated claims. Proposition B.15 requires a separate comparison retaining all residual terms.

The seam analysis is conditional. Our main contribution is correcting the prefix and recomputing calibration inputs before later blocks are quantized. Table 3 supports this independently of seam revisiting: Mistral-7B improves from 31.10 PPL with Sequential CBQ to 16.95 without revisiting, versus 16.80 with revisiting.

Weakness 1 / Question 2: Midpoint contraction and FP16 neighbor. Aggregate two-block output-MSE ratios do not certify the per-sample midpoint contraction in Assumption B.2. The uniform midpoint certificate, local representation envelopes, and Lipschitz diagnostics have not been established for the reported runs. We do not claim that the aggregate ratios validate the theorem.

The first seam visit updates the quantized left block through a fixed FP16 right block. The second updates quantization parameters in both blocks after the right block is quantized. The appendix's description of joint updates during the first visit needs correction. Quantizing the right block does not itself change the midpoint, which depends on the left block and its input, but this does not establish contraction at either visit.

Weakness 2 / Question 3: Optimization budget. We evaluated a second sequential CBQ pass on all seven DBF models, keeping epochs per pair and calibration samples unchanged. Two full passes refine every adjacent pair twice, whereas ICBQ K=4 revisits only boundary pairs. The sequential baseline therefore receives a larger CBQ optimizer-step budget, with the same prefit budget.

C4 PPL (lower is better), DBF with prefit and C4 calibration. One-pass Sequential CBQ and ICBQ values are from the submitted paper.

ModelSeq. CBQ, 1 passSeq. CBQ, 2 passes (new)ICBQ K=4
Mistral-7B31.1027.1016.80
Llama-2-7B13.6013.4112.50
Llama-2-13B10.4810.4610.09
Llama-3.2-3B40.6341.0437.01
Llama-3-8B48.9850.3942.32
Qwen3-8B5752.9040216.1229.90
Qwen3-14B44.8245.9424.67

ICBQ achieves lower PPL on all seven models despite the larger sequential refinement budget, supporting the value of interleaving beyond repeated sequential refinement.

Weakness 2 / Question 3: Prefit. Prefit adapts the floating-point student block before DBF, with the teacher frozen, using

Lpre(ϕ)=1N∑n=1NMSE⁡(fbϕ(X^b(n)),fb⋆(Xb⋆,(n))).\mathcal L_{\mathrm{pre}}(\phi)=\frac{1}{N}\sum_{n=1}^{N}\operatorname{MSE}\left(f_b^{\phi}(\widehat X_b^{(n)}),f_b^{\star}(X_b^{\star,(n)})\right).

Student and teacher inputs come from their respective prefixes, which Eq. (5) should distinguish. Prefit uses up to 50 AdamW steps at learning rate 10−410^{-4} in float32.

Table 9 shows that prefit improves ICBQ across all seven models, while its effect on Sequential CBQ is less consistent. ICBQ remains effective in a setting where the sequential pipeline becomes unstable. On Qwen3-8B, with prefit enabled under both schedules, Sequential CBQ yields 5752.90 PPL while ICBQ achieves 29.90. Our new two-pass sequential result reaches 40216.12 PPL, exceeding the published one-pass reference despite the larger refinement-step budget. Additional sequential refinement therefore does not resolve this failure. Without prefit, ICBQ also improves on Sequential CBQ, from 47.28 to 41.59 PPL. The interaction with prefit highlights the practical value of the scheduling change, with benefits observed both with and without prefit.

Weakness 2 / Question 3: Effective precision and storage. The ternary factor alphabet does not imply 1.58 effective bits per original weight. Accounting must include both factors, their intermediate dimensions, scales, encoding, and unquantized weights. We will clarify the labeling. Our scheduling comparisons keep DBF factorization settings fixed but do not establish superiority across quantizers at matched compression ratios. Verified per-layer ranks and serialized-storage figures remain unavailable, so this accounting is unresolved.

As the discussion period ends in 2 days, we would like to kindly follow up and ask whether our rebuttal has addressed your concerns. Our response includes clarifications, new results, and planned changes to the paper. We would be grateful for your feedback and happy to address any remaining questions.

Best, Authors

Thanks for adding the two-pass baseline. The rebuttal does not report the two-pass runtime, so performance at a matched wall-clock budget remains unclear. My main concern is Corollary 4.2. With the residual from Eq. (24), the seam count cancels from the full bound in Eq. (23), so this bound does not establish the claimed geometric improvement. I’m keeping my rating at 2.

Thank you for your reply. We provide two brief clarifications for completeness.

Optimization budget. The two-pass runs were performed on different hardware from the original runs, with otherwise identical settings, so their wall-clock times are not directly comparable. As a hardware-independent measure, the table below reports total optimizer updates (CBQ refinement and prefit).

ModelICBQ K=4Seq. CBQ, 2 passes2-pass / ICBQ
Mistral-7B196,110318,9901.63×
Llama-2-7B196,110318,9901.63×
Llama-2-13B247,710401,3101.62×
Llama-3.2-3B85,830139,5901.63×
Llama-3-8B196,110318,9901.63×
Qwen3-8B221,910360,1501.62×
Qwen3-14B247,710401,3101.62×

The two-pass baseline performs about 1.6× more optimizer updates than ICBQ. Update counts also track runtime well in our setting, making them a reasonable proxy for cost: relative to one-pass Sequential CBQ, ICBQ performed about 1.23× more CBQ updates, and its measured wall-clock overhead was 1.21× on average (Table 5).

Corollary 4.2. As stated in our response, we agree that the full bound with the residual from Eq. (24) does not establish a geometric improvement. In the camera-ready version, we will restate Corollary 4.2 so that it claims only what the full bound supports, and present the seam-count dependence at the recurrence level where it holds (Lemma B.13, Proposition B.15). This concerns the analysis of the optional seam revisit. The main claims about interleaving rest on the empirical comparisons and ablations and are unaffected.

We hope these clarifications, together with the planned revisions, are helpful for your final assessment.

Reviewer_pYLdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Interleaved Cross-Block Quantization (ICBQ), a post-training quantization strategy based on cross-block refinement. Instead of quantizing the entire model before refinement, ICBQ alternates between quantization, refinement, and activation recomputation, aiming to reduce the propagation of quantization errors. Experiments on multiple LLMs, under ternary (~1.58-bit) quantization, show consistent improvements in perplexity and downstream performance, with more frequent interleaving yielding larger gains. While the empirical results support the ICBQ, the ablations suggest that most benefits arise from early error correction and activation recalibration rather than the seam-revisiting mechanism from the theoretical analysis.

Critical strengths
  • ICBQ is a lightweight scheduling that is potentially useful and can be easily integrated into PTQ pipelines

  • ICBQ demonstrates improvements over Sequential-CBQ across many LLMs in ultra-low-bit (ternary) settings.

  • The paper shows that the approach transfers beyond DBF quantization to GPTQ.

  • The ablations show that chunk-size experiments support the central hypothesis that more frequent refinement and activation recalibration improve quantization quality.

Critical weaknesses
  • The core contribution is a scheduling modification of existing cross-block quantization. Even though the contribution provides good results, it is arguably too small for ICLR

  • The theoretical analysis emphasizes seam revisiting, but ablations suggest most gains come from early refinement and activation re-computation.

  • Most results are in the extreme ternary quantization results, leaving it unclear how impactful the method is at more commonly used bit-widths.

  • ICBQ incurs additional runtime, and the paper does not disentangle the benefits of improved scheduling from those of increased optimization effort.

Questions for authors
  1. How does ICBQ differ from existing iterative calibration or block-coordinate optimization schemes beyond changing the scheduling of refinement?

  2. The theory emphasizes seam revisiting as a key mechanism; can the authors quantify how much of the observed gain is attributable to seam revisits versus early activation recalibration?

  3. Most of the results are reported for ternary quantization. How does the benefit of ICBQ scale at higher, more commonly used precisions such as 4-bit and 8-bit quantization?

  4. Since ICBQ incurs additional runtime, how much of the performance improvement remains when compared against a baseline with a matched computational budget?

3 comments

Thank you for recognizing the method's ease of integration, improvements across models, transfer to GPTQ, and supporting chunk-size ablation.

Critical Weakness 1 and Question 1: Novelty and significance. The contribution is deliberately simple: it changes when cross-block refinement occurs relative to quantization of later blocks. ICBQ retains the existing two-block objective, but corrects the prefix and recomputes activations before subsequent blocks are quantized. This changes the inputs on which those quantization decisions are made. The specific contribution beyond the general idea of iterative calibration or block-coordinate updates is this interleaving of refinement with ongoing quantization.

We believe the significance lies in identifying and validating this scheduling effect across models and two inner quantizers. By changing when cross-block refinement occurs, ICBQ introduces a distinct scheduling perspective that delivers substantial low-bit improvements through a straightforward modification to existing PTQ pipelines. This combination of insight, effectiveness, and ease of adoption is a strength of the contribution. More broadly, a simple method that provides a new insight and strong empirical results can offer greater practical value than a more sophisticated approach with comparable benefits, precisely because it is easier to understand, implement, and adopt.

Critical Weakness 2 and Question 2: Theory versus seam ablation. We began with the idea of correcting errors before quantizing later blocks. Once we implemented this with chunks, the boundaries raised a natural follow-up question: could revisiting them improve the result further? We explored that question theoretically and then checked it through ablation. The extra revisit contributed less to PPL than we had anticipated, but we retained the result because it helps distinguish the main scheduling benefit from this optional addition.

Table 3 quantifies this distinction. Mistral-7B changes from 31.10 PPL (sequential) to 16.95 (no revisit) and 16.80 (revisit). Qwen3-8B changes from 5752.90 to 30.02 and 29.90, respectively. These comparisons isolate the marginal effect of revisiting, although they do not separately quantify early refinement and activation recomputation. We will clarify the intended scope of the seam analysis and distinguish it from the empirical evidence for interleaving.

Critical Weakness 3 and Question 3: Higher bit widths. Our focus is aggressive low-bit quantization, stated in the abstract and the introduction's opening sentence. Table 2 extends the evaluation beyond ternary DBF to GPTQ W2g128 and W3. At W3, ICBQ improves on Sequential CBQ in all seven models, but GPTQ-only remains better in several cases. We therefore do not claim that refinement is uniformly beneficial across precisions. We will make this scope and limitation more explicit. Benefits at 4 and 8 bits remain unestablished by our current experiments.

Critical Weakness 4 and Question 4: Matched computational budget. We agree that the additional runtime requires a direct control. We evaluated a second full sequential CBQ pass on all seven DBF models, keeping the epochs per pair and calibration samples unchanged. Two sequential passes refine every adjacent pair twice, whereas ICBQ K=4 revisits only the pairs at chunk boundaries. The two-pass baseline therefore receives a larger CBQ optimizer-step budget, with the same prefit budget. This control compares refinement-step budgets.

C4 PPL, DBF with prefit and C4 calibration. The one-pass Sequential CBQ and ICBQ values are from the submitted paper.

ModelSequential CBQ, 1 pass (paper)Sequential CBQ, 2 passes (new)ICBQ K=4 (paper)
Mistral-7B31.1027.1016.80
Llama-2-7B13.6013.4112.50
Llama-2-13B10.4810.4610.09
Llama-3.2-3B40.6341.0437.01
Llama-3-8B48.9850.3942.32
Qwen3-8B5752.9040216.1229.90
Qwen3-14B44.8245.9424.67

ICBQ retains lower PPL on all seven models despite the sequential baseline's larger refinement-step budget. This supports the value of interleaving beyond simply repeating sequential refinement.

Reviewer_pYLdOpenReview ↗

Rebuttal comment

Thank you for the response and the new experiments. The two-pass Sequential-CBQ baseline is a strong control and resolves my concern that ICBQ’s gains could be explained by additional optimisation. The rebuttal also clarifies that seam revisiting is a secondary effect and that the method is specifically targeted at aggressive low-bit quantisation. My remaining concerns are that the contribution is still a modest scheduling modification and that the theory analyses a mechanism that appears to explain only a small part of the empirical gains. Nevertheless, the rebuttal has improved my assessment, and I am inclined to raise my rating to 3 (weak acceptance).

Thank you for your thoughtful reassessment and constructive feedback. We are glad that the additional experiments resolved your concern about the optimization budget, and we will incorporate these results into the revised paper.

We agree that the scheduling change is simple. We see its strength in combining a distinct temporal perspective with empirical effectiveness and ease of adoption in existing PTQ pipelines.

Regarding the theory, we agree that the current analysis concerns seam revisiting and does not explain the main empirical gains from interleaving. We will make this scope explicit in the revised paper. We see a theoretical treatment of interleaving’s effect on reconstruction error on the calibration data as a promising direction for extending the analysis. Establishing suitable assumptions and validating the argument would require further work beyond the remaining discussion period. If this yields a sound result, we would include it in the camera-ready version. Our present claims about interleaving rest on the empirical comparisons and ablations.

Reviewer_jBZFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines the optimization schedule of cross-block post-training quantization (CBQ). Standard sequential CBQ quantizes all blocks first and then performs a single left-to-right refinement pass over adjacent two-block windows, which allows early quantization drift to accumulate across layers. The authors propose ICBQ (Interleaved Cross-Block Quantization), which partitions the model into chunks of KK blocks, interleaves short cross-block refinement passes during progressive quantization, and re-rolls intermediate activations from stored layer-0 inputs. The paper provides theoretical bounds showing seam-amortized error contraction, where chunk boundaries receive additional contractions. Evaluated on seven models using ternary DBF (1.58-bit) and low-bit GPTQ (W3/W2g128), ICBQ prevents divergence and improves perplexity over sequential CBQ.

Critical strengths
  1. Fresh Scheduling Perspective: The paper tackles PTQ error propagation from an orthogonal dimension—the temporal scheduling of refinement—demonstrating that interleaving local optimizations can curb compounding quantization drift.
  2. Empirical Recovery from Quantization Collapse: On aggressively quantized models where Sequential CBQ diverges or degrades heavily (e.g., Qwen3-8B and LLaMA-3-8B under ternary DBF), ICBQ consistently stabilizes reconstruction and yields finite, competitive perplexities.
Critical weaknesses
  1. Severe Disconnect Between Theory and Empirical Findings: The paper frames its entire mathematical justification around the concept of seam-amortized geometric contraction via boundary revisits (Theorem 4.1 and Corollary 4.2). However, the author's own ablation study in Table 3 demonstrates that revisiting seams yields negligible and inconsistent gains compared to the no-revisit schedule. This implies that the primary theoretical contribution does not explain the true empirical behavior of the method.
  2. Unfair Compute Budget Comparison: ICBQ introduces repeated activation re-rolling and extra refinement passes, leading to a 20% to 30% increase in wall-clock time compared to Sequential CBQ (Table 5). The paper fails to provide an iso-compute baseline (e.g., allowing Sequential CBQ multi-pass refinement or more optimizer iterations) to verify whether the gain comes from the scheduling itself or simply from additional optimization budget.
Questions for authors
  1. Given that the ablation in Table 3 shows seam revisiting provides negligible empirical gain over no-revisit, how should we reconcile the primary theoretical focus on seam-amortized contraction with the actual empirical driver of ICBQ?
  2. If Sequential CBQ is allowed multiple sequential passes (or an increased optimization iteration budget) to match the total compute/runtime of ICBQ, how much of the performance gap remains?
2 comments

Thank you for recognizing the fresh scheduling perspective and the empirical recovery from severe quantization degradation.

Critical Weakness 1 and Question 1: Theory and the seam ablation. We first asked whether correcting errors before quantizing later blocks would improve the result. Implementing this with chunks naturally created boundaries, which raised a second question: would an extra refinement at those boundaries help? We explored that possibility theoretically, then tested it instead of assuming that it would translate into practical gains. The ablation showed that most of the PPL improvement was already present without the extra revisit. We kept this result because it clarifies which part of the method matters, even though the revisit helped less than we had anticipated.

For example, on ternary DBF, Mistral-7B improves from 31.10 PPL with Sequential CBQ to 16.95 without seam revisiting, versus 16.80 with revisiting. Qwen3-8B improves from 5752.90 to 30.02 without revisiting, versus 29.90 with it. These results support interleaving independently of the extra seam updates.

The abstract and Section 3.4 already identify boundary overlap as secondary, but we acknowledge that the theory section's prominence can suggest a broader explanation of the gains. We will clarify its intended scope and distinguish the seam analysis from the empirical evidence for interleaving.

Critical Weakness 2 and Question 2: Compute-budget comparison. We agree that equal per-window settings do not establish equal total compute. The no-revisit variant already uses one visit per adjacent pair, as does Sequential CBQ, showing that extra pair visits alone do not explain the gains. However, activation re-rolling still adds cost.

We evaluated a second full sequential CBQ pass on all seven DBF models, keeping the epochs per pair and calibration samples unchanged. Two sequential passes refine every adjacent pair twice, whereas ICBQ K=4 revisits only the pairs at chunk boundaries. The two-pass baseline therefore receives a larger CBQ optimizer-step budget, with the same prefit budget. This control compares refinement-step budgets.

C4 PPL, DBF with prefit and C4 calibration. The one-pass Sequential CBQ and ICBQ values are from the submitted paper.

ModelSequential CBQ, 1 pass (paper)Sequential CBQ, 2 passes (new)ICBQ K=4 (paper)
Mistral-7B31.1027.1016.80
Llama-2-7B13.6013.4112.50
Llama-2-13B10.4810.4610.09
Llama-3.2-3B40.6341.0437.01
Llama-3-8B48.9850.3942.32
Qwen3-8B5752.9040216.1229.90
Qwen3-14B44.8245.9424.67

ICBQ retains lower PPL on all seven models despite the sequential baseline's larger refinement-step budget. This supports the value of interleaving beyond simply repeating sequential refinement.

As the discussion period ends in 2 days, we would like to kindly follow up and ask whether our rebuttal has addressed your concerns. Our response includes clarifications, new results, and planned changes to the paper. We would be grateful for your feedback and happy to address any remaining questions.

Best, Authors

Reviewer_8tycOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes Interleaved Cross-Block Quantization (ICBQ), a scheduling strategy for improving low-bit LLM quantization considering cross-block dependency and global optimization. ICBQ first groups multiple transformer blocks into chunks and then conducts progressive quantization with chunks, which interleaves cross-block refinement. The authors study the effect of boundary overlapping through both theoretical analysis and empirical evaluation results. This paper also demonstrates that the proposed method is compatible with existing block-wise PTQ methods, such as GPTQ.

Critical strengths

The experiment shows significant performance improvement for ultra-low-bit compression across different model families, especially in perplexity reduction. It shows that progressive quantization and cross-block refinements are crucial in ultra-low-bit compression.

Critical weaknesses
  • ICBQ integrates two concepts into cross-block quantization: progressive quantization and error compensation (seam-revisit in this paper). However, both have already been proposed in previous works, such as GPTQ and QEP, respectively. Seam-revisit can be viewed as an alternative error-compensation approach using cross-block refinement: the second block in boundary pairs is affected by the first block's quantization error during the Adam update. Therefore, in my opinion, ICBQ is an excellent combination but contains limited novelty for this problem.

  • The relation between chunk size and performance improvement needs further verification and discussion. The chunk-size ablation was conducted only on ternary DBF and lacks consideration of different quantization and bit-width settings. The same ablation on GPTQ would help evaluate the impact of refinement frequency more thoroughly.

Questions for authors

How does ICBQ perform with a smaller chunk size (K=2), using the same GPTQ bit-width compression as in Table 2? Does it show a similar improvement as ternary DBF under this setting?

6 comments

Thank you for recognizing the improvements in ultra-low-bit quantization across model families and the compatibility with existing PTQ methods.

Critical Weakness 1: Novelty relative to GPTQ and QEP. We agree that progressive quantization and error compensation are established ideas. Our contribution is the temporal scheduling of cross-block refinement relative to ongoing quantization. GPTQ compensates quantization errors within its layer-wise weight updates, while QEP modifies the layer-wise objective to account for accumulated errors. ICBQ retains the same two-block reconstruction objective as our Sequential CBQ baseline, but applies refinement before later blocks are quantized and recomputes their calibration inputs through the corrected prefix. Thus, the distinction concerns how and when error correction is performed, beyond the shared goal of reducing error.

This complementarity is illustrated by Table 2, where we combine ICBQ with GPTQ and improve on the corresponding Sequential CBQ schedule. ICBQ adds a scheduling mechanism while retaining GPTQ as the inner quantizer.

Seam revisiting is an optional addition, not the defining contribution. Table 3 shows that interleaving retains most of its PPL gains without revisiting seams. We will sharpen this distinction in the discussion of novelty.

Critical Weakness 2 and Questions: GPTQ chunk-size ablation. Thank you for suggesting this test. We are prioritizing additional K=2 evaluations under the GPTQ bit-width settings in Table 2, keeping the remaining settings fixed. We will share completed results in a follow-up as they become available during the discussion.

Follow-up on Critical Weakness 2 and Question 1: GPTQ chunk-size ablation.

We now have K=2 results for all seven models under GPTQ W2g128, with the remaining settings unchanged from Table 2. The table compares C4 PPL with the reported K=4 results.

ModelICBQ K=4ICBQ K=2
Mistral-7B14.2014.15
Llama-2-7B13.5413.43
Llama-2-13B10.8210.78
Llama-3.2-3B41.5640.56
Llama-3-8B38.6837.54
Qwen3-8B28.2927.89
Qwen3-14B22.2122.15

K=2 achieves lower C4 PPL than K=4 on all seven models. The favorable K=2 trend is therefore reproduced across every tested model under GPTQ W2g128, confirming that it is not confined to ternary DBF. This directly addresses the requested extension of the chunk-size ablation to another quantizer and bit-width setting. We will include this seven-model GPTQ W2g128 ablation in the revised paper.

To further assess the trend across GPTQ bit widths, W3 evaluations are underway. We will share those results if completed within the discussion period.

Follow-up on Critical Weakness 2 and Question 1: GPTQ W3 chunk-size ablation.

As promised, we now have the K=2 results under GPTQ W3 for all seven models, with all other settings unchanged from Table 2. The table compares C4 PPL with the paper's K=4 results.

ModelICBQ K=4ICBQ K=2
Mistral-7B10.3210.30
Llama-2-7B9.759.69
Llama-2-13B8.188.18
Llama-3.2-3B23.1422.78
Llama-3-8B23.5222.35
Qwen3-8B21.0920.89
Qwen3-14B17.5117.46

K=2 achieves lower C4 PPL than K=4 on six of seven models and matches it on Llama-2-13B. The favorable K=2 trend therefore also holds at W3. The gains are generally smaller than at W2g128, consistent with our earlier observation that cross-block refinement has a smaller effect at W3. We will include both the W2g128 and W3 GPTQ chunk-size ablations in the revised paper.

We hope this addresses your remaining question on chunk size. Please let us know if you have any other concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.