acceptodds
Under review as a conference paper at ICLR 2027

BRACE: Error-Guided Mixed-Precision LLM Quantization Without Backpropagation

Abstract

Post-training quantization (PTQ) reduces the deployment costs of large language models (LLMs). Uniform-precision quantization remains a widely used approach, but it often incurs substantial quality degradation at ultra-low bit-widths. In contrast, mixed precision quantization offers a better accuracy–budget trade-off by selectively assigning precision. However, at ultra-low precision, a quantization error cannot be judged by its local magnitude alone. Its significance also depends on the stage-specific response it should induce. This observation motivates a common design principle for bit allocation, error compensation, and quantizer calibration: quantization error should not merely be minimized locally, but should guide the response taken at each stage. Our approach builds on this observation and introduces three key components: (i) Curvature-Conditioned Residual Transport (CCRT) allocates bits using a joint curvature-conditioned cost that combines the error committed at a group with the one-step correction burden transported to its immediate successor; (ii) Block-Anchored Ridge Compensation (BARC) pairs reference and quantized paths within each block to derive a ridge-stabilized correction for propagated error before applying GPTQ to the corrected center under a metric with a ridge term; and (iii) Residual-Feedback Grid Refinement (RFGR) converts the elementwise errors of curvature-weighted first-pass grids into weights for a second scale and zero-point fit at the assigned bit-width, with adjacent-bit error reductions providing the feedback signal. Built on a GPTQ backbone, BRACE uses only a small calibration set and local layer-wise statistics, without backpropagation or iterative retraining. Extensive experiments on language modeling and zero-shot commonsense reasoning across the Llama2, Llama3, Qwen3, and OLMo3 model families show that BRACE achieves state-of-the-art performance among comparable backpropagation-free PTQ methods under 2- and 3-bit weight-only quantization settings. Code will be released upon acceptance.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, post-training quantization, mixed-precision quantization, low-bit quantization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_g7fUOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes BRACE, a backpropagation-free mixed-precision post-training quantization (PTQ) framework for large language models (LLMs), aiming to address the limitations of conventional local error metrics in capturing subsequent compensation requirements under ultra-low-bit quantization. The authors argue that the significance of quantization error depends not only on its local magnitude but also on the corrective response it induces at different stages of the quantization process. Based on this observation, BRACE introduces three key components: (1) Curvature-Conditioned Residual Transport (CCRT), which performs mixed-precision bit allocation by jointly considering Hessian-conditioned quantization errors and the correction burden transported to the immediate successor group; (2) Block-Anchored Ridge Compensation (BARC), which estimates weight corrections through ridge regression based on activation discrepancies between reference and quantized paths, followed by GPTQ quantization around the corrected weight center; and (3) Residual-Feedback Grid Refinement (RFGR), which uses error reductions across adjacent bit-widths to guide element-wise weighted calibration of quantization scales and zero-points. The authors implement BRACE on a GPTQ backbone and evaluate its 2-bit and 3-bit weight-only quantization performance across six models from the Llama2, Llama3, Qwen3, and OLMo3 families.

Critical strengths

Strength 1: The paper presents a comprehensive PTQ pipeline encompassing bit allocation, error compensation, and quantizer calibration, guided by the corrective responses induced by quantization errors. All three components are mathematically well-defined and can be integrated into the GPTQ framework without requiring backpropagation. Experimental results demonstrate that BRACE achieves substantial accuracy improvements across multiple models under ultra-low-bit quantization settings, highlighting its practical value in preserving model quality at low precision.

Strength 2: The paper evaluates six LLMs of different sizes and architectures, comparing BRACE against several representative methods, including GPTQ, OmniQuant, QuIP#, SliM-LLM, and TurboBoA. The authors conduct ablation studies on all three core components and further replace CCRT, BARC, and RFGR with related mechanisms from existing methods in the appendix. They also independently evaluate the successor transport term in CCRT and the feedback-guided refitting mechanism in RFGR. These experiments strengthen the empirical evidence for the effectiveness of the proposed method and help clarify the individual contributions of its components.

Critical weaknesses

Weakness 1: I have concerns regarding the methodological novelty of BRACE. Existing methods such as SliM-LLM [1] have already investigated group-wise mixed-precision bit allocation and salience-weighted quantizer calibration, while TurboBoA [2] has introduced propagated-error compensation and adaptive grid refinement. Qronos [3] has also systematically studied cross-layer quantization error propagation and compensation. Although BRACE proposes new optimization formulations for these related problems, its three components largely represent further combinations and refinements of existing mixed-precision allocation, analytical error compensation, and weighted quantizer calibration techniques. While the authors provide component replacement experiments and ablation studies demonstrating empirical improvements, the paper does not sufficiently clarify what fundamentally distinct technical contributions these modifications offer beyond existing approaches. Furthermore, the three components still follow a sequential pipeline in which the bit map is determined and frozen before error compensation and quantizer fitting. The proposed unified error-guided principle does not lead to a joint optimization objective or corresponding guarantees that explicitly couple all three stages.

Weakness 2: The paper defines its 2-bit and 3-bit settings using exact average quantization precision. However, this budget primarily reflects the nominal bit-width of quantized weights and does not necessarily correspond to the actual storage cost during deployment. Group-wise mixed-precision quantization requires additional bit maps, group offsets, and other quantization metadata. Although BRACE demonstrates clear improvements in model quality, the main experimental tables do not systematically report the effective bits per weight of all methods, including scales, zero-points, metadata, and packing overhead. Furthermore, comprehensive comparisons under strictly matched effective storage budgets are missing. In the 3-bit setting, some perplexity improvements over the strongest baselines are relatively small, making differences in actual storage requirements and inference overhead potentially important when evaluating these gains. Without a unified accuracy–memory–latency comparison, the current experiments do not sufficiently establish whether BRACE achieves a superior accuracy–efficiency trade-off under equivalent practical deployment constraints.

Weakness 3: One of the main motivations of BRACE is that conventional local quantization error metrics fail to capture the effects of error propagation into subsequent computation stages. However, the experimental evaluation primarily focuses on WikiText2/C4 perplexity, multiple-choice commonsense reasoning tasks, and MMLU. Although six language models are evaluated, detailed downstream accuracy comparisons are provided only for Llama2-13B and Qwen3-14B, without challenging mathematical reasoning or code-generation benchmarks. Preserving perplexity and multiple-choice accuracy under ultra-low-bit quantization does not necessarily imply that a model can reliably perform long-horizon, multi-step autoregressive generation. In particular, given the paper's emphasis on quantization error propagation and compensation, the absence of these evaluations makes it difficult to determine whether the proposed mechanisms effectively mitigate error accumulation during complex generation processes. Moreover, the experiments are limited to dense LLMs of at most 14B parameters, leaving the applicability of BRACE to larger models and more challenging reasoning scenarios insufficiently explored. These limitations weaken the empirical support for the practical deployment benefits of BRACE under ultra-low-bit quantization.

[1] Wei Huang et al. "SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models." Proceedings of the 42nd International Conference on Machine Learning (ICML), PMLR 267:25672–25692, 2025. https://proceedings.mlr.press/v267/huang25aa.html

[2] Junhan Kim et al. "TurboBoA: Faster and Exact Attention-aware Quantization without Backpropagation." International Conference on Learning Representations (ICLR), 2026. https://arxiv.org/abs/2602.04929

[3] Shihao Zhang, Haoyu Zhang, Ian Colbert, and Rayan Saab. "Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantization." International Conference on Learning Representations (ICLR), 2026. https://arxiv.org/abs/2505.11695

Questions for authors

Q1: Could the authors clarify the distinct technical contributions of CCRT, BARC, and RFGR compared with SliM-LLM, TurboBoA, and Qronos? Beyond the empirical gains shown in Appendix C, what fundamentally distinguishes BARC's error compensation and RFGR's quantizer refinement from existing approaches? Does the proposed unified framework offer any joint optimization objective or theoretical guarantees beyond combining these components?

Q2: Could the authors report the effective BPW or actual packed model sizes, including scales, zero-points, and mixed-precision metadata? Appendix D shows that BRACE requires 2.2 GiB versus GPTQ's 2.1 GiB and achieves lower inference throughput, despite both being labeled 2-bit. Would BRACE retain its advantages under strictly matched storage budgets and comparable deployment constraints?

Q3: Could the authors extend the 2-bit evaluation to more challenging benchmarks, such as AIME 2024/2025, MATH-500, GPQA Diamond, MBPP+, and LiveCodeBench? In particular, do CCRT and BARC independently improve performance on multi-step reasoning and code generation, where quantization errors may accumulate during autoregressive decoding?

Q4: CCRT considers only the current group's residual and its correction burden on the immediate successor, while ignoring longer-range propagation and interactions among bit-width assignments. Could the authors justify this approximation and clarify how well the resulting surrogate objective reflects actual reconstruction or downstream errors during sequential quantization?

Q5: Appendix D reports inference results using a custom mixed-precision CUDA kernel integrated into vLLM. Could the authors clarify how the kernel handles mixed-bit packing, bit offsets, metadata, and runtime dequantization? Given the different kernel implementations used by BRACE and GPTQ, could the authors also release the anonymous kernel implementation, vLLM integration, and benchmarking scripts to enable independent reproduction of the reported inference results?

Q6: WaterSIC [4] is a recent mixed-precision weight-only PTQ method with publicly available code. At 2 bits on Llama3-8B, WaterSIC without fine-tuning reports a WikiText2 perplexity of 8.93, compared with BRACE's 15.12. Although their calibration protocols and bit-rate accounting differ, could the authors provide a comparison under matched calibration, evaluation, and effective storage budgets, covering both 2-bit and 3-bit settings?

[4] Egor Lifar, Semyon Savkin, Or Ordentlich, and Yury Polyanskiy. "WaterSIC: Information-Theoretically (Near) Optimal Linear Layer Quantization." ICML 2026. https://arxiv.org/abs/2603.04956

3 comments

Thank you for the detailed review. We further clarify the technical contributions, deployment costs, and additional experimental results.

Q1. Distinct technical contributions and optimization guarantees

BRACE is motivated by a unified view of error-induced response, meaning the correction required by a quantization residual rather than the model output. In sequential GPTQ, equal local rounding errors can induce different corrections to unquantized weights depending on curvature and processing order. BRACE therefore guides quantization decisions by their correction demands, not local error magnitude alone.

CCRT differs from the salience-based allocation in SliM-LLM. It evaluates each candidate bit width using its actual quantization residual, normalized by the inverse curvature factor, together with the correction burden imposed on the successor weight group. Dynamic programming minimizes this candidate-dependent cost under an exact logical bit budget, accounting for how precision choices affect subsequent quantization.

BARC differs from TurboBoA in its compensation target and correction procedure. TurboBoA incorporates propagated error compensation into attention-aware joint output-channel quantization. BARC uses paired reference and quantized paths sharing the same Transformer-block input to isolate the discrepancy introduced within the block. Covariance-scaled ridge regression estimates a stabilized weight correction, which is discretized under the fixed bit map using a ridge-adjusted curvature metric.

RFGR differs from both conventional weighted calibration and Qronos. Qronos focuses on discrete weight selection and error diffusion during sequential quantization, whereas RFGR refines the quantization grid after bit allocation. It derives elementwise fitting weights from curvature-weighted error reductions between adjacent-bit first-pass grids. These weights capture the marginal benefit of added precision and guide refinement of the assigned-bit grid. Unlike salience weighting in SliM-LLM or coordinate-descent refinement in TurboBoA, RFGR uses precision sensitivity derived from actual quantization outcomes, with an objective distinct from Qronos's rounding and error diffusion.

The shared response principle also connects the stages computationally. CCRT determines the bit map, BARC produces corrected weight centers and a discretization metric, and RFGR refits the grid using the resulting working weights and curvature information. These are dependent decisions within one sequential quantization process, rather than independent heuristics. Such a unified optimization perspective can be realized through linked stage-specific objectives without requiring a single monolithic loss. Dynamic programming solves the finite CCRT allocation surrogate exactly, while BARC provides an analytical regularized correction.

Q2. Effective storage and deployment efficiency

The reported 2-bit and 3-bit settings refer to average logical weight precision. We additionally account for scales, zero-points, metadata, and packing overhead. For Llama2-7B, the effective storage requirements under a losslessly compacted mixed-precision layout are shown below.

MethodEffective BPWComplete model size
BRACE2.25842.1915 GiB
GPTQ2.14632.1069 GiB

Both methods store only one final set of scales and zero-points, with FP16 scales occupying 96.5 MiB each. The main difference comes from zero-point representation. BRACE uses FP16 zero-points occupying 96.5 MiB, whereas GPTQ uses packed 2-bit integer zero-points occupying 12.0625 MiB.

BRACE therefore incurs approximately 4% additional total storage, primarily due to quantization parameter representation rather than a higher logical weight precision budget. Together with the throughput measurements in Appendix D, these results provide a concrete characterization of the accuracy, storage, and inference efficiency trade-off.

Q3. Additional reasoning benchmark results

We conducted additional zero-shot experiments on all 198 questions of GPQA Diamond using Llama2-7B.

MethodWeight precisionAccuracy
Original model16-bit27.8%
BRACE2-bit22.7%
QuIP# without fine-tuning2-bit18.7%

Under the same evaluation protocol, BRACE achieves 22.7% accuracy versus 18.7% for QuIP# without fine-tuning, a difference of 4.0 percentage points on this question set. We view this as a supplementary benchmark comparison rather than conclusive evidence of improved multi-step reasoning or long-form generation. We plan to extend the evaluation to AIME, MATH-500, MBPP+, and LiveCodeBench in future work.

Continue

Q4. Justification of the one-step approximation

The successor group in CCRT refers to the next weight group in quantization order, not the next Transformer layer or generated token.

The one-step approximation captures both the current group's reconstruction error and its correction burden on the immediately following unquantized group. This enables efficient candidate scoring through shared curvature factors and batched evaluation while accounting for local error propagation.

Extending the lookahead horizon requires additional downstream correction terms. Accurately modeling interactions among bit-width assignments would further require recomputing subsequent residuals and quantization grids for alternative assignments, substantially increasing computational and memory costs and preventing efficient construction of the additive cost table.

Importantly, this approximation applies only to allocation scoring. During final GPTQ quantization, error corrections are still propagated to all remaining columns. BRACE also incorporates limited layer-budget recoupling to account for layer-level interactions.

We do not claim that one-step lookahead is theoretically optimal. Nevertheless, our ablation results demonstrate that incorporating the successor's correction burden improves allocation over local-error-based alternatives while maintaining manageable computational cost.

Q5. Mixed-precision CUDA implementation and reproducibility

The inference results in Appendix D were obtained using our implemented mixed-precision CUDA kernel integrated into vLLM. The measured throughput is already close to GPTQ, demonstrating efficient execution of groupwise mixed-precision quantization without additional inference stages.

Our kernel stores quantized weights in a compact, groupwise bit-packed layout. Each group has metadata recording its bit width and packed storage offset, allowing direct weight addressing despite nonuniform precision.

During inference, the kernel reads group descriptors, performs bit-width-specific unpacking, and applies the corresponding scales and zero-points. Bit extraction, affine dequantization, and matrix multiplication are fused, avoiding materialization of full-precision weights. Metadata and quantization parameters are reused within computation tiles to reduce redundant memory accesses.

The remaining throughput gap relative to GPTQ mainly arises from additional metadata handling, nonuniform bit extraction, and FP16 zero-point accesses. Nevertheless, Appendix D demonstrates throughput close to uniform-precision GPTQ while retaining BRACE's accuracy advantages. CCRT, BARC, and RFGR operate entirely offline and introduce no additional inference operations.

Upon acceptance, we will release the complete BRACE quantization implementation, the CUDA kernel used in Appendix D, its vLLM integration, and the evaluation and benchmarking scripts with corresponding configurations to support independent reproduction.

Q6. Comparison with WaterSIC

WaterSIC's published Llama3-8B comparison uses the full WikiText-2 training split for calibration, whereas BRACE uses 128 C4 sequences. We therefore reran WaterSIC without fine-tuning using BRACE's C4 calibration data (128 × 2048 tokens, seed 0) and evaluator at WaterSIC's target rate of 2. On Llama2-7B, WaterSIC obtains WikiText-2/C4 perplexities of 8.92/10.29, compared with BRACE's 8.67/9.92. On Llama3-8B, WaterSIC obtains 14.45/16.89, compared with BRACE's 15.12/16.21, with WaterSIC performing better on WikiText-2 and BRACE on C4. On Qwen3-8B, WaterSIC obtains 11.50/14.83, compared with BRACE's 11.34/14.49.

These comparisons match calibration and evaluation but not deployment cost. WaterSIC's entropy-based rate, rescaler and side-information overhead differ from BRACE's exact logical bit budget. Its repeated requantization, calibration evaluations, and iterative rescaler fitting also incur additional quantization cost. Nominal 2-bit perplexity therefore does not establish an equal-storage or equal-compute comparison. We will expand the baseline comparisons to include KronQ and SliM-LLM+, with evaluation protocols and bit-rate accounting specified.

Thank you for the detailed response and additional experiments. While the authors clarify the implementation-level differences, the three components still largely combine established techniques: CCRT combines SliM-LLM-style mixed-precision bit allocation with successor-aware error propagation related to Qronos and TurboBoA; BARC adapts propagated-error compensation through paired reference/quantized paths and ridge regression; and RFGR combines salience-weighted quantizer calibration, as explored in SliM-LLM, with adaptive grid refinement related to TurboBoA. Although these modifications provide some empirical improvements, they remain incremental refinements of existing techniques rather than fundamentally new methodological contributions. Therefore, my concerns regarding the limited novelty remain, and I maintain my rating.

Reviewer_1GJ7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BRACE, a backpropagation-free mixed-precision weight quantization method built on GPTQ. BRACE consists of three components: CCRT allocates group-wise bit-widths by considering both the current quantization error and its correction cost on the next group; BARC compensates for errors propagated from earlier quantized operations using ridge regression; and RFGR refines the quantization grid using error changes across adjacent bit-widths. The resulting mixed-precision models are evaluated under average 2-bit and 3-bit settings.

Critical strengths

1.Each component addresses a relevant issue in ultra-low-bit GPTQ. CCRT incorporates sequential error propagation into group-wise bit allocation, BARC provides a closed-form, ridge-regularized correction for errors inherited from earlier quantized operations, and RFGR explores adjacent-bit error changes as a signal for quantizer refinement. Together, these components form a technically plausible approach to improving ultra-low-bit GPTQ.

2.Strong empirical performance and practical implementation. BRACE reports substantial and consistent improvements across multiple model families, bit-widths, and evaluation tasks. The paper also includes detailed ablations and runtime measurements, and demonstrates actual mixed-precision inference with a custom kernel rather than relying only on simulated quantization.

Critical weaknesses

1.Lack of clarity. The paper is difficult to follow because several central concepts are introduced without sufficient explanation. For example, the meaning of “response” in “stage-specific response” is not clearly specified. The Introduction repeatedly uses the term “group” without explaining that it refers to consecutive weight columns sharing the same quantization parameters and bit-width. Moreover, since all the proposed components are built around GPTQ, a concise GPTQ preliminary is necessary to make the proposed formulations understandable and self-contained.

2.Lack of a unified motivation. Although CCRT, BARC, and RFGR are individually reasonable, they address different problems using separate objectives. The paper does not clearly explain why these three components should be combined, nor derive them from a common formulation. Consequently, BRACE appears more like a collection of incremental GPTQ modifications than a framework driven by a unified insight.

3.Concerns about the reported results. The gains reported by BRACE are surprisingly large and consistent. My main concern is that the proposed motivation largely targets limitations of the local sequential GPTQ pipeline. In particular, BARC addresses propagated errors that are already visible to OmniQuant’s block-wise reconstruction objective. RFGR introduces a different adjacent-bit sensitivity heuristic for quantizer fitting, but it remains unclear why this local weighted-fitting rule should substantially outperform quantization parameters optimized directly for block-output reconstruction. It is therefore difficult to understand why these local backpropagation-free modifications to GPTQ outperform block-wise optimization by such large margins.

This gap is even harder to explain when compared with the SliM-LLM+ results reported in [1]. SliM-LLM+ applies SliM-LLM's group-wise mixed-precision allocation to OmniQuant's gradient-based block-wise optimization, yet it provides only modest improvements over OmniQuant. BRACE, in contrast, reports much larger gains from CCRT, BARC, and RFGR. The paper does not provide a convincing explanation for why these local analytical modifications should be substantially more effective than mixed-precision block-wise optimization.

The following table compares the reported 2-bit WikiText2 perplexities with group size 128:  

SourceMethodLlama2-7B ↓\downarrowLlama2-13B ↓\downarrow
OmniQuant paperOmniQuant11.068.26
SliM-LLM paperOmniQuant11.068.26
BRACE paperOmniQuant16.9510.34
SliM-LLM paperSliM-LLM16.019.41
BRACE paperSliM-LLM56.5613.08
SliM-LLM paperSliM-LLM+10.877.59
BRACE paperBRACE8.677.12

[1] Wei Huang, Haotong Qin, Yangdong Liu, Yawei Li, Qinshuo Liu, Xianglong Liu, Luca Benini, Michele Magno, Shiming Zhang, Xiaojuan Qi: SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models. ICML 2025

Questions for authors

Could the authors provide a direct comparison between BRACE and SliM-LLM+ under the same experimental settings, and explain why BRACE achieves substantially larger gains than mixed-precision block-wise optimization?

2 comments

We thank the reviewer for the detailed comments.

W1. 'Response' denotes error-induced correction; 'group' is the bit-allocation unit

By "stage-specific response," we mean the adjustment induced by a quantization residual at a particular stage, not the model output. In sequential GPTQ, (wj−qj)/Ujj(w_j-q_j)/U_{jj} propagates through Uj,j+1:U_{j,j+1:} to unquantized weights. Thus, equal rounding errors can require different downstream corrections depending on curvature and processing order. BRACE uses these correction demands for bit allocation (CCRT), weight-center correction (BARC), and grid fitting (RFGR).

A group comprises consecutive input columns in GPTQ order (128 here), sharing one bit-width; each row-group has its own scale and zero-point.

W2. CCRT, BARC, and RFGR share an error-correction principle

Our unifying principle is to choose quantization actions based on the correction an error calls for and the correction available at each stage, rather than its local magnitude alone. CCRT allocates bits using the curvature-conditioned group residual and the correction burden propagated to its successor; BARC uses within-block reference/quantized activation mismatch to correct the weight center, then discretizes under a ridge-adjusted GPTQ metric; RFGR refits the assigned-bit grid using curvature-weighted error reductions across adjacent-bit first-pass grids.

The components are also computationally linked: CCRT fixes the bit map; BARC determines the corrected center and discretization metric; RFGR fits the grid using the resulting working weights and curvature factor. These are connected decisions in one sequential quantization process, not independent GPTQ heuristics.

W3. Explaining the gains over block-wise optimization

Calibration choice explains part of the baseline discrepancy: our main results use C4, whereas the cited published OmniQuant and SliM-LLM values use WikiText2 calibration. We reran 2-bit WikiText2 calibration (128 sequences, 2048 tokens, group size 128, seed 0). The table reports WikiText2 test PPL for Llama2-7B / Llama2-13B.

MethodPublishedOur C4 calibrationOur WikiText2 calibration
OmniQuant11.06 / 8.2616.95 / 10.3411.73 / 8.24
SliM-LLM16.01 / 9.4156.56 / 13.0818.15 / 9.04
BRACE—8.67 / 7.127.23 / 6.21

WikiText2 calibration brings our reruns closer to the published results; both 13B results are comparable to or better than those reported. We followed the reported full-method configurations; remaining differences may reflect unspecified implementation details or sampled calibration text. All main-table comparisons use a common C4 protocol.

The distinction from block-wise optimization is not whether propagated errors are considered: OmniQuant and BRACE both account for within-block activation mismatch. OmniQuant minimizes block-output reconstruction via learnable quantization parameters. BARC instead explicitly estimates a continuous weight-center correction from paired within-block activations, then discretizes using a ridge-adjusted GPTQ metric. RFGR refits the assigned-bit grid on sequentially updated weights using the resulting curvature and adjacent-bit feedback. The methods therefore differ in the corrections they implement and in how they construct the final discrete solution; block-output reconstruction need not yield the same corrected center or curvature-weighted grid. SliM-LLM+ already combines mixed-precision allocation with gradient-based block optimization; Q1 directly evaluates this alternative rather than attributing BRACE's gains to mixed precision alone.

Q1. Direct comparison with SliM-LLM+

We compare SliM-LLM+ with BRACE under C4 calibration, matched nominal bit budget, group size 128, 128 calibration sequences, and PPL evaluation. SliM-LLM+ uses learnable clipping, augmented reconstruction loss, and 50 optimization epochs. We also include C4-calibrated SliM-LLM results from Table 1 of our manuscript. Entries are WikiText2 / C4 PPL.

BitsModelSliM-LLMSliM-LLM+BRACE
2Llama2-7B56.56 / 18.8521.93 / 14.498.67 / 9.92
2Llama2-13B13.08 / 13.3110.29 / 11.067.12 / 8.71
3Llama2-7B6.45 / 7.856.21 / 7.725.93 / 7.42
3Llama2-13B5.43 / 7.025.36 / 6.955.22 / 6.80

BRACE achieves lower WikiText2 and C4 PPL than SliM-LLM+ on both models at 2 and 3 bits. We will include SliM-LLM+ as an additional baseline in the revised manuscript, with its configuration details.

Thank you for the detailed rebuttal and additional experiments. The response clarifies implementation details and strengthen the empirical evidence. But my main concern remains. While I better understand the differences between BRACE and the related methods, the framework still feels like a combination of several incremental modifications rather than a method driven by a unified motivation. It also makes the reported performance gains less intuitive to me. Therefore, I will keep my current rating.

Reviewer_L3hEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BRACE, group-wise mixed-precision quantization framework for LLMs. Based on GPTQ, it proposes Curvature-Conditioned Residual Transport, Block-Anchored Ridge Compensation, and Residual-Feedback Grid Refinement to improve bit-allocation. Experiments on different model families demonstrate improvement on 2bit and 3bit settings.

Critical strengths
  1. The motivation is clear and easy to follow
  2. The ablation study is detailed
  3. Efficiency analysis is provided, covering both algorithm efficiency and deployment efficiency
Critical weaknesses
  1. Lacking of comparison with up-to-date mixed-precision methods. Although BRACE is a mixed-precision method, but in tab1&2, it compares exclusively against uniform-precision methods. The only mixed-preicision baseline is Slim-LLM. Some methods have already pushed the bit-allocation to group-wise level, such as MixLLM[1]. Comparison with recent work, KronQ [2], is also missing.

  2. The formulation of BARC is quite similar to QEP, while the added ridge parameter is a standard regularization heuristic.

  3. The experiments were only conducted on relatively small LLMs. Experiments on larger models would strengthen the generality of the proposed method.

[1] MixLLM: LLM Quantization with Global Mixed-precision between Output-features and Highly-efficient System Design [2] KronQ: LLM Quantization via Kronecker-Factored Hessian

Questions for authors

please see the weakness above. also:

  1. Is the proposed method compatible with other quantization framework such as gptaq?
2 comments

We thank the reviewer for the constructive comments. We provide additional comparisons and clarify the technical differences below.

W1. Comparisons with recent methods

We evaluated KronQ using the same calibration and evaluation protocol (128 C4 sequences, 2048 tokens, group size 128). Results at 2-bit average weight precision are:

ModelKronQ Wiki / C4BRACE Wiki / C4
Llama2-7B9.39 / 10.548.67 / 9.92
Llama2-13B7.83 / 9.197.12 / 8.71
Llama3-8B15.57 / 17.2315.12 / 16.21
Qwen3-8B11.58 / 14.9011.34 / 14.49
Qwen3-14B9.51 / 12.809.40 / 12.66

BRACE achieves lower PPL in all 10 comparisons. For Llama2-7B, BRACE completes quantization in 48 minutes on a single A100. In our KronQ run, the weight-quantization stage also takes less than one hour, but computing its gradient covariance HGH_G takes 8 hours and 52 minutes. Thus, HGH_G precomputation dominates KronQ's total runtime.

We additionally tested KronQ's inter-layer mixed precision. On Llama2-7B, KronQ obtains 8.31 / 9.62 PPL (Wiki / C4) at 2.223 average bits, versus 8.67 / 9.92 for BRACE at 2.000 bits. These results use different bit budgets.

MixLLM's released implementation uses 4/8-bit mixed weights with 8-bit activations, unlike our 2/3-bit weight-only setting. We will clarify this distinction in the revision.

W2. Relationship between BARC and QEP

Both methods address activation mismatch, but BARC differs in three respects:

  • Reference construction: QEP propagates a full-precision reference across blocks; BARC anchors both paths at the same block input, isolating within-block discrepancies.
  • Ridge-stabilized compensation: BARC uses a curvature-scaled ridge (λ_c = γ λ_max(H_q)) to limit unstable correction along poorly conditioned directions.
  • Ridge-aware discretization: BARC further incorporates ridge into the GPTQ discretization metric (H_d = H_q + λ_c I), stabilizing quantization around the corrected center. QEP's damping is applied to its correction solver.

We replaced BARC with QEP's correction under the same fixed bit maps and RFGR settings, using QEP's released default parameters.

ModelQEP Wiki / C4BARC Wiki / C4
Llama2-7B8.90 / 10.208.67 / 9.92
Llama3-8B16.40 / 18.4815.12 / 16.21
Qwen3-8B15.42 / 17.3711.34 / 14.49

BARC yields lower PPL in all six comparisons.

W3. Larger models

We agree that evaluating larger models would strengthen the generality of BRACE. Given limited computational resources and the tight rebuttal timeline, we selected Qwen3-32B instead of Llama2-70B. We are currently comparing BRACE with baselines under the same quantization protocol and plan to post the results in a follow-up within the next few days during the discussion period.

Q1. Compatibility with GPTAQ

Yes. We implemented BRACE's components on GPTAQ's asymmetric quantization backend. To avoid double compensation, we adjust GPTAQ's reference after applying BARC, preserving the original full-precision target. Results on Llama2-7B (2-bit, C4 calibration, group size 128) are:

ConfigurationWikiC4
GPTAQ, uniform37.6925.52
+ CCRT30.2720.03
+ BARC, uniform13.6713.71
+ RFGR, uniform9.8711.73
+ CCRT + BARC + RFGR8.359.61

For this compatibility test, the CCRT bit map is generated in the GPTQ-based workflow and then transferred to GPTAQ as a fixed map. Each component improves GPTAQ on both datasets, while the combination achieves the best results (37.69 / 25.52 to 8.35 / 9.61). This demonstrates transferability beyond the original GPTQ backend.

Additional Results on Qwen3-32B

In response to the reviewer's suggestion that experiments on larger models would strengthen the generality of our method, we evaluated BRACE and the baselines on Qwen3-32B at both 2-bit and 3-bit precision. The results are shown below. Lower perplexity is better.

Method2-bit WikiText-22-bit C43-bit WikiText-23-bit C4
RTN1.04e53.40e424.3030.87
GPTQ36.3226.758.7511.69
OmniQuant14.3517.428.6311.75
QuIP#10.2413.248.2411.39
SliM-LLM16.4619.028.8711.88
TurboBoA11.9314.918.2311.40
KronQ10.3613.138.2711.35
BRACE10.0712.968.0511.11

The FP16 Qwen3-32B checkpoint has WikiText-2/C4 perplexities of 7.61/10.72. Unlike the Qwen3-8B-Base and Qwen3-14B-Base checkpoints used in the paper, Qwen3-32B has no corresponding Base version, so its absolute perplexities are not directly comparable with those Base-model results. Under the same Qwen3-32B setting, BRACE achieves the lowest WikiText-2 and C4 perplexities among the evaluated methods at both 2-bit and 3-bit precision. We will include these larger-model results in the revised paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.