- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes BRACE, a backpropagation-free mixed-precision post-training quantization (PTQ) framework for large language models (LLMs), aiming to address the limitations of conventional local error metrics in capturing subsequent compensation requirements under ultra-low-bit quantization. The authors argue that the significance of quantization error depends not only on its local magnitude but also on the corrective response it induces at different stages of the quantization process. Based on this observation, BRACE introduces three key components: (1) Curvature-Conditioned Residual Transport (CCRT), which performs mixed-precision bit allocation by jointly considering Hessian-conditioned quantization errors and the correction burden transported to the immediate successor group; (2) Block-Anchored Ridge Compensation (BARC), which estimates weight corrections through ridge regression based on activation discrepancies between reference and quantized paths, followed by GPTQ quantization around the corrected weight center; and (3) Residual-Feedback Grid Refinement (RFGR), which uses error reductions across adjacent bit-widths to guide element-wise weighted calibration of quantization scales and zero-points. The authors implement BRACE on a GPTQ backbone and evaluate its 2-bit and 3-bit weight-only quantization performance across six models from the Llama2, Llama3, Qwen3, and OLMo3 families.
Critical strengths
Strength 1: The paper presents a comprehensive PTQ pipeline encompassing bit allocation, error compensation, and quantizer calibration, guided by the corrective responses induced by quantization errors. All three components are mathematically well-defined and can be integrated into the GPTQ framework without requiring backpropagation. Experimental results demonstrate that BRACE achieves substantial accuracy improvements across multiple models under ultra-low-bit quantization settings, highlighting its practical value in preserving model quality at low precision.
Strength 2: The paper evaluates six LLMs of different sizes and architectures, comparing BRACE against several representative methods, including GPTQ, OmniQuant, QuIP#, SliM-LLM, and TurboBoA. The authors conduct ablation studies on all three core components and further replace CCRT, BARC, and RFGR with related mechanisms from existing methods in the appendix. They also independently evaluate the successor transport term in CCRT and the feedback-guided refitting mechanism in RFGR. These experiments strengthen the empirical evidence for the effectiveness of the proposed method and help clarify the individual contributions of its components.
Critical weaknesses
Weakness 1: I have concerns regarding the methodological novelty of BRACE. Existing methods such as SliM-LLM [1] have already investigated group-wise mixed-precision bit allocation and salience-weighted quantizer calibration, while TurboBoA [2] has introduced propagated-error compensation and adaptive grid refinement. Qronos [3] has also systematically studied cross-layer quantization error propagation and compensation. Although BRACE proposes new optimization formulations for these related problems, its three components largely represent further combinations and refinements of existing mixed-precision allocation, analytical error compensation, and weighted quantizer calibration techniques. While the authors provide component replacement experiments and ablation studies demonstrating empirical improvements, the paper does not sufficiently clarify what fundamentally distinct technical contributions these modifications offer beyond existing approaches. Furthermore, the three components still follow a sequential pipeline in which the bit map is determined and frozen before error compensation and quantizer fitting. The proposed unified error-guided principle does not lead to a joint optimization objective or corresponding guarantees that explicitly couple all three stages.
Weakness 2: The paper defines its 2-bit and 3-bit settings using exact average quantization precision. However, this budget primarily reflects the nominal bit-width of quantized weights and does not necessarily correspond to the actual storage cost during deployment. Group-wise mixed-precision quantization requires additional bit maps, group offsets, and other quantization metadata. Although BRACE demonstrates clear improvements in model quality, the main experimental tables do not systematically report the effective bits per weight of all methods, including scales, zero-points, metadata, and packing overhead. Furthermore, comprehensive comparisons under strictly matched effective storage budgets are missing. In the 3-bit setting, some perplexity improvements over the strongest baselines are relatively small, making differences in actual storage requirements and inference overhead potentially important when evaluating these gains. Without a unified accuracy–memory–latency comparison, the current experiments do not sufficiently establish whether BRACE achieves a superior accuracy–efficiency trade-off under equivalent practical deployment constraints.
Weakness 3: One of the main motivations of BRACE is that conventional local quantization error metrics fail to capture the effects of error propagation into subsequent computation stages. However, the experimental evaluation primarily focuses on WikiText2/C4 perplexity, multiple-choice commonsense reasoning tasks, and MMLU. Although six language models are evaluated, detailed downstream accuracy comparisons are provided only for Llama2-13B and Qwen3-14B, without challenging mathematical reasoning or code-generation benchmarks. Preserving perplexity and multiple-choice accuracy under ultra-low-bit quantization does not necessarily imply that a model can reliably perform long-horizon, multi-step autoregressive generation. In particular, given the paper's emphasis on quantization error propagation and compensation, the absence of these evaluations makes it difficult to determine whether the proposed mechanisms effectively mitigate error accumulation during complex generation processes. Moreover, the experiments are limited to dense LLMs of at most 14B parameters, leaving the applicability of BRACE to larger models and more challenging reasoning scenarios insufficiently explored. These limitations weaken the empirical support for the practical deployment benefits of BRACE under ultra-low-bit quantization.
[1] Wei Huang et al. "SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models." Proceedings of the 42nd International Conference on Machine Learning (ICML), PMLR 267:25672–25692, 2025. https://proceedings.mlr.press/v267/huang25aa.html
[2] Junhan Kim et al. "TurboBoA: Faster and Exact Attention-aware Quantization without Backpropagation." International Conference on Learning Representations (ICLR), 2026. https://arxiv.org/abs/2602.04929
[3] Shihao Zhang, Haoyu Zhang, Ian Colbert, and Rayan Saab. "Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantization." International Conference on Learning Representations (ICLR), 2026. https://arxiv.org/abs/2505.11695
Questions for authors
Q1: Could the authors clarify the distinct technical contributions of CCRT, BARC, and RFGR compared with SliM-LLM, TurboBoA, and Qronos? Beyond the empirical gains shown in Appendix C, what fundamentally distinguishes BARC's error compensation and RFGR's quantizer refinement from existing approaches? Does the proposed unified framework offer any joint optimization objective or theoretical guarantees beyond combining these components?
Q2: Could the authors report the effective BPW or actual packed model sizes, including scales, zero-points, and mixed-precision metadata? Appendix D shows that BRACE requires 2.2 GiB versus GPTQ's 2.1 GiB and achieves lower inference throughput, despite both being labeled 2-bit. Would BRACE retain its advantages under strictly matched storage budgets and comparable deployment constraints?
Q3: Could the authors extend the 2-bit evaluation to more challenging benchmarks, such as AIME 2024/2025, MATH-500, GPQA Diamond, MBPP+, and LiveCodeBench? In particular, do CCRT and BARC independently improve performance on multi-step reasoning and code generation, where quantization errors may accumulate during autoregressive decoding?
Q4: CCRT considers only the current group's residual and its correction burden on the immediate successor, while ignoring longer-range propagation and interactions among bit-width assignments. Could the authors justify this approximation and clarify how well the resulting surrogate objective reflects actual reconstruction or downstream errors during sequential quantization?
Q5: Appendix D reports inference results using a custom mixed-precision CUDA kernel integrated into vLLM. Could the authors clarify how the kernel handles mixed-bit packing, bit offsets, metadata, and runtime dequantization? Given the different kernel implementations used by BRACE and GPTQ, could the authors also release the anonymous kernel implementation, vLLM integration, and benchmarking scripts to enable independent reproduction of the reported inference results?
Q6: WaterSIC [4] is a recent mixed-precision weight-only PTQ method with publicly available code. At 2 bits on Llama3-8B, WaterSIC without fine-tuning reports a WikiText2 perplexity of 8.93, compared with BRACE's 15.12. Although their calibration protocols and bit-rate accounting differ, could the authors provide a comparison under matched calibration, evaluation, and effective storage budgets, covering both 2-bit and 3-bit settings?
[4] Egor Lifar, Semyon Savkin, Or Ordentlich, and Yury Polyanskiy. "WaterSIC: Information-Theoretically (Near) Optimal Linear Layer Quantization." ICML 2026. https://arxiv.org/abs/2603.04956
3 comments
Continue
Q4. Justification of the one-step approximation
The successor group in CCRT refers to the next weight group in quantization order, not the next Transformer layer or generated token.
The one-step approximation captures both the current group's reconstruction error and its correction burden on the immediately following unquantized group. This enables efficient candidate scoring through shared curvature factors and batched evaluation while accounting for local error propagation.
Extending the lookahead horizon requires additional downstream correction terms. Accurately modeling interactions among bit-width assignments would further require recomputing subsequent residuals and quantization grids for alternative assignments, substantially increasing computational and memory costs and preventing efficient construction of the additive cost table.
Importantly, this approximation applies only to allocation scoring. During final GPTQ quantization, error corrections are still propagated to all remaining columns. BRACE also incorporates limited layer-budget recoupling to account for layer-level interactions.
We do not claim that one-step lookahead is theoretically optimal. Nevertheless, our ablation results demonstrate that incorporating the successor's correction burden improves allocation over local-error-based alternatives while maintaining manageable computational cost.
Q5. Mixed-precision CUDA implementation and reproducibility
The inference results in Appendix D were obtained using our implemented mixed-precision CUDA kernel integrated into vLLM. The measured throughput is already close to GPTQ, demonstrating efficient execution of groupwise mixed-precision quantization without additional inference stages.
Our kernel stores quantized weights in a compact, groupwise bit-packed layout. Each group has metadata recording its bit width and packed storage offset, allowing direct weight addressing despite nonuniform precision.
During inference, the kernel reads group descriptors, performs bit-width-specific unpacking, and applies the corresponding scales and zero-points. Bit extraction, affine dequantization, and matrix multiplication are fused, avoiding materialization of full-precision weights. Metadata and quantization parameters are reused within computation tiles to reduce redundant memory accesses.
The remaining throughput gap relative to GPTQ mainly arises from additional metadata handling, nonuniform bit extraction, and FP16 zero-point accesses. Nevertheless, Appendix D demonstrates throughput close to uniform-precision GPTQ while retaining BRACE's accuracy advantages. CCRT, BARC, and RFGR operate entirely offline and introduce no additional inference operations.
Upon acceptance, we will release the complete BRACE quantization implementation, the CUDA kernel used in Appendix D, its vLLM integration, and the evaluation and benchmarking scripts with corresponding configurations to support independent reproduction.
Q6. Comparison with WaterSIC
WaterSIC's published Llama3-8B comparison uses the full WikiText-2 training split for calibration, whereas BRACE uses 128 C4 sequences. We therefore reran WaterSIC without fine-tuning using BRACE's C4 calibration data (128 × 2048 tokens, seed 0) and evaluator at WaterSIC's target rate of 2. On Llama2-7B, WaterSIC obtains WikiText-2/C4 perplexities of 8.92/10.29, compared with BRACE's 8.67/9.92. On Llama3-8B, WaterSIC obtains 14.45/16.89, compared with BRACE's 15.12/16.21, with WaterSIC performing better on WikiText-2 and BRACE on C4. On Qwen3-8B, WaterSIC obtains 11.50/14.83, compared with BRACE's 11.34/14.49.
These comparisons match calibration and evaluation but not deployment cost. WaterSIC's entropy-based rate, rescaler and side-information overhead differ from BRACE's exact logical bit budget. Its repeated requantization, calibration evaluations, and iterative rescaler fitting also incur additional quantization cost. Nominal 2-bit perplexity therefore does not establish an equal-storage or equal-compute comparison. We will expand the baseline comparisons to include KronQ and SliM-LLM+, with evaluation protocols and bit-rate accounting specified.
Thank you for the detailed response and additional experiments. While the authors clarify the implementation-level differences, the three components still largely combine established techniques: CCRT combines SliM-LLM-style mixed-precision bit allocation with successor-aware error propagation related to Qronos and TurboBoA; BARC adapts propagated-error compensation through paired reference/quantized paths and ridge regression; and RFGR combines salience-weighted quantizer calibration, as explored in SliM-LLM, with adaptive grid refinement related to TurboBoA. Although these modifications provide some empirical improvements, they remain incremental refinements of existing techniques rather than fundamentally new methodological contributions. Therefore, my concerns regarding the limited novelty remain, and I maintain my rating.
Thank you for the detailed review. We further clarify the technical contributions, deployment costs, and additional experimental results.
Q1. Distinct technical contributions and optimization guarantees
BRACE is motivated by a unified view of error-induced response, meaning the correction required by a quantization residual rather than the model output. In sequential GPTQ, equal local rounding errors can induce different corrections to unquantized weights depending on curvature and processing order. BRACE therefore guides quantization decisions by their correction demands, not local error magnitude alone.
CCRT differs from the salience-based allocation in SliM-LLM. It evaluates each candidate bit width using its actual quantization residual, normalized by the inverse curvature factor, together with the correction burden imposed on the successor weight group. Dynamic programming minimizes this candidate-dependent cost under an exact logical bit budget, accounting for how precision choices affect subsequent quantization.
BARC differs from TurboBoA in its compensation target and correction procedure. TurboBoA incorporates propagated error compensation into attention-aware joint output-channel quantization. BARC uses paired reference and quantized paths sharing the same Transformer-block input to isolate the discrepancy introduced within the block. Covariance-scaled ridge regression estimates a stabilized weight correction, which is discretized under the fixed bit map using a ridge-adjusted curvature metric.
RFGR differs from both conventional weighted calibration and Qronos. Qronos focuses on discrete weight selection and error diffusion during sequential quantization, whereas RFGR refines the quantization grid after bit allocation. It derives elementwise fitting weights from curvature-weighted error reductions between adjacent-bit first-pass grids. These weights capture the marginal benefit of added precision and guide refinement of the assigned-bit grid. Unlike salience weighting in SliM-LLM or coordinate-descent refinement in TurboBoA, RFGR uses precision sensitivity derived from actual quantization outcomes, with an objective distinct from Qronos's rounding and error diffusion.
The shared response principle also connects the stages computationally. CCRT determines the bit map, BARC produces corrected weight centers and a discretization metric, and RFGR refits the grid using the resulting working weights and curvature information. These are dependent decisions within one sequential quantization process, rather than independent heuristics. Such a unified optimization perspective can be realized through linked stage-specific objectives without requiring a single monolithic loss. Dynamic programming solves the finite CCRT allocation surrogate exactly, while BARC provides an analytical regularized correction.
Q2. Effective storage and deployment efficiency
The reported 2-bit and 3-bit settings refer to average logical weight precision. We additionally account for scales, zero-points, metadata, and packing overhead. For Llama2-7B, the effective storage requirements under a losslessly compacted mixed-precision layout are shown below.
Both methods store only one final set of scales and zero-points, with FP16 scales occupying 96.5 MiB each. The main difference comes from zero-point representation. BRACE uses FP16 zero-points occupying 96.5 MiB, whereas GPTQ uses packed 2-bit integer zero-points occupying 12.0625 MiB.
BRACE therefore incurs approximately 4% additional total storage, primarily due to quantization parameter representation rather than a higher logical weight precision budget. Together with the throughput measurements in Appendix D, these results provide a concrete characterization of the accuracy, storage, and inference efficiency trade-off.
Q3. Additional reasoning benchmark results
We conducted additional zero-shot experiments on all 198 questions of GPQA Diamond using Llama2-7B.
Under the same evaluation protocol, BRACE achieves 22.7% accuracy versus 18.7% for QuIP# without fine-tuning, a difference of 4.0 percentage points on this question set. We view this as a supplementary benchmark comparison rather than conclusive evidence of improved multi-step reasoning or long-form generation. We plan to extend the evaluation to AIME, MATH-500, MBPP+, and LiveCodeBench in future work.