- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies optimization scheduling in low-bit post-training quantization and proposes ICBQ, which inserts a short reconstruction sweep over pairs of adjacent Transformer blocks after each chunk of blocks is quantized and recomputes activations through the current prefix. The method also includes optional seam revisiting at chunk boundaries. Using ternary DBF and GPTQ as inner quantizers, the authors report perplexity and zero-shot accuracy across multiple language models and examine the effects of chunk size, seam revisiting, and prefit through ablation studies. The theoretical analysis derives recurrence-based error upper bounds under assumptions of a local representation-error envelope, per-sample midpoint contraction, and exact activation re-roll.
Critical strengths
- Simple and adaptable scheduling. Earlier correction and activation recomputation improve subsequent quantization inputs while preserving the two-block objective. Experiments with ternary DBF and GPTQ show applicability beyond a single weight representation.
- Fairly comprehensive evaluation. Multi-model results, seam/chunk-size/prefit ablations, and wall-clock timings reveal both benefits and limitations. These results could support a useful empirical contribution if the concerns below are addressed.
Critical weaknesses
- The current full theoretical bound does not support the emphasized geometric benefit of seam revisiting. Let . Substituting Eqs. (22) and (24) into Eq. (23) gives , with canceling completely. With shared constants fixed, this displayed bound is independent of and exceeds . I acknowledge the tighter bounds discussed in Remark B.6 and Proposition B.15. However, with the explicit residual in Eq. (24), the main corollary yields no informative geometric gain, and the paper does not use the tighter residual to establish nontrivial conditions for an improvement in the full bound. The aggregate loss ratios in Appendix B.11 do not provide the per-sample midpoint certificate required by Assumption B.2 and therefore do not close this gap.
- The empirical gains are insufficiently contextualized in terms of computational cost and compression ratio. Full ICBQ uses more pair updates and activation re-rolls, taking approximately 21% longer on average, yet no sequential baseline with the same total optimization budget is provided. The dramatic improvement on Qwen3-8B also depends strongly on prefit (Table 9). Moreover, DBF's two ternary factors and scaling parameters mean that "1.58-bit" does not directly translate into effective bits per original weight. The paper does not report , actual storage requirements, or comparisons at matched compression ratios. These gaps make it difficult to assess the method's practical value.
Questions for authors
- Is the above simplification of Eqs. (22)–(24) correct? Using Proposition B.15 or another full upper bound, could you provide explicit, satisfiable conditions for , together with a numerical example, and explain how the main corollary's geometric gain holds under these conditions?
- Could you report per-sample midpoint errors, local representation envelopes, and Lipschitz diagnostics for the actual CBQ calls, rather than only aggregate MSE ratios at the two-block output? With seam revisiting enabled, how are the relevant assumptions verified when the first boundary update uses an FP16 neighboring block?
- Under the same total optimization-step or wall-clock budget, how does ICBQ compare with a baseline that allocates more budget to sequential refinement? Please clarify the exact prefit objective and implementation in light of Table 9, and report the per-layer values and actual model storage size for ternary DBF.
4 comments
As the discussion period ends in 2 days, we would like to kindly follow up and ask whether our rebuttal has addressed your concerns. Our response includes clarifications, new results, and planned changes to the paper. We would be grateful for your feedback and happy to address any remaining questions.
Best, Authors
Thanks for adding the two-pass baseline. The rebuttal does not report the two-pass runtime, so performance at a matched wall-clock budget remains unclear. My main concern is Corollary 4.2. With the residual from Eq. (24), the seam count cancels from the full bound in Eq. (23), so this bound does not establish the claimed geometric improvement. I’m keeping my rating at 2.
Thank you for your reply. We provide two brief clarifications for completeness.
Optimization budget. The two-pass runs were performed on different hardware from the original runs, with otherwise identical settings, so their wall-clock times are not directly comparable. As a hardware-independent measure, the table below reports total optimizer updates (CBQ refinement and prefit).
| Model | ICBQ K=4 | Seq. CBQ, 2 passes | 2-pass / ICBQ |
|---|---|---|---|
| Mistral-7B | 196,110 | 318,990 | 1.63× |
| Llama-2-7B | 196,110 | 318,990 | 1.63× |
| Llama-2-13B | 247,710 | 401,310 | 1.62× |
| Llama-3.2-3B | 85,830 | 139,590 | 1.63× |
| Llama-3-8B | 196,110 | 318,990 | 1.63× |
| Qwen3-8B | 221,910 | 360,150 | 1.62× |
| Qwen3-14B | 247,710 | 401,310 | 1.62× |
The two-pass baseline performs about 1.6× more optimizer updates than ICBQ. Update counts also track runtime well in our setting, making them a reasonable proxy for cost: relative to one-pass Sequential CBQ, ICBQ performed about 1.23× more CBQ updates, and its measured wall-clock overhead was 1.21× on average (Table 5).
Corollary 4.2. As stated in our response, we agree that the full bound with the residual from Eq. (24) does not establish a geometric improvement. In the camera-ready version, we will restate Corollary 4.2 so that it claims only what the full bound supports, and present the seam-count dependence at the recurrence level where it holds (Lemma B.13, Proposition B.15). This concerns the analysis of the optional seam revisit. The main claims about interleaving rest on the empirical comparisons and ablations and are unaffected.
We hope these clarifications, together with the planned revisions, are helpful for your final assessment.
Thank you for recognizing the scheduling contribution and empirical evaluation, and for carefully examining the theory.
Weakness 1 / Question 1: Theoretical bound. Your simplification is correct. Substituting Eqs. (22) and (24) into Eq. (23) removes the seam-count dependence from the displayed full bound. It therefore does not establish the claimed geometric improvement over the sequential bound. We will correct the interpretation of Corollary 4.2 and the associated claims. Proposition B.15 requires a separate comparison retaining all residual terms.
The seam analysis is conditional. Our main contribution is correcting the prefix and recomputing calibration inputs before later blocks are quantized. Table 3 supports this independently of seam revisiting: Mistral-7B improves from 31.10 PPL with Sequential CBQ to 16.95 without revisiting, versus 16.80 with revisiting.
Weakness 1 / Question 2: Midpoint contraction and FP16 neighbor. Aggregate two-block output-MSE ratios do not certify the per-sample midpoint contraction in Assumption B.2. The uniform midpoint certificate, local representation envelopes, and Lipschitz diagnostics have not been established for the reported runs. We do not claim that the aggregate ratios validate the theorem.
The first seam visit updates the quantized left block through a fixed FP16 right block. The second updates quantization parameters in both blocks after the right block is quantized. The appendix's description of joint updates during the first visit needs correction. Quantizing the right block does not itself change the midpoint, which depends on the left block and its input, but this does not establish contraction at either visit.
Weakness 2 / Question 3: Optimization budget. We evaluated a second sequential CBQ pass on all seven DBF models, keeping epochs per pair and calibration samples unchanged. Two full passes refine every adjacent pair twice, whereas ICBQ K=4 revisits only boundary pairs. The sequential baseline therefore receives a larger CBQ optimizer-step budget, with the same prefit budget.
C4 PPL (lower is better), DBF with prefit and C4 calibration. One-pass Sequential CBQ and ICBQ values are from the submitted paper.
ICBQ achieves lower PPL on all seven models despite the larger sequential refinement budget, supporting the value of interleaving beyond repeated sequential refinement.
Weakness 2 / Question 3: Prefit. Prefit adapts the floating-point student block before DBF, with the teacher frozen, using
Lpre(ϕ)=N1∑n=1NMSE(fbϕ(Xb(n)),fb⋆(Xb⋆,(n))).
Student and teacher inputs come from their respective prefixes, which Eq. (5) should distinguish. Prefit uses up to 50 AdamW steps at learning rate 10−4 in float32.
Table 9 shows that prefit improves ICBQ across all seven models, while its effect on Sequential CBQ is less consistent. ICBQ remains effective in a setting where the sequential pipeline becomes unstable. On Qwen3-8B, with prefit enabled under both schedules, Sequential CBQ yields 5752.90 PPL while ICBQ achieves 29.90. Our new two-pass sequential result reaches 40216.12 PPL, exceeding the published one-pass reference despite the larger refinement-step budget. Additional sequential refinement therefore does not resolve this failure. Without prefit, ICBQ also improves on Sequential CBQ, from 47.28 to 41.59 PPL. The interaction with prefit highlights the practical value of the scheduling change, with benefits observed both with and without prefit.
Weakness 2 / Question 3: Effective precision and storage. The ternary factor alphabet does not imply 1.58 effective bits per original weight. Accounting must include both factors, their intermediate dimensions, scales, encoding, and unquantized weights. We will clarify the labeling. Our scheduling comparisons keep DBF factorization settings fixed but do not establish superiority across quantizers at matched compression ratios. Verified per-layer ranks and serialized-storage figures remain unavailable, so this accounting is unresolved.