acceptodds
Under review as a conference paper at ICLR 2027

BEYOND LOCAL RECONSTRUCTION: GEOMETRY- PRESERVING OUTPUT ALIGNMENT FOR BINARY LLM QUANTIZATION

28 views

Abstract

Recent advances in post-training quantization for large language models (LLMs) have demonstrated that low-bit quantization methods can maintain most of the original model performance. Despite such successes, 1-bit quantization remains particularly challenging. A common strategy in 1-bit quantization is to determine binary weights by matching full-precision parameters, following a weight-driven criterion. However, this objective is not directly aligned with the quantized model's objective, which is to preserve the model's output behavior under the impact of quantization. A natural alternative is to adopt output-driven criteria that minimize discrepancies in model outputs using calibration data. Surprisingly, naive output-driven approaches often perform even worse in the 1-bit regime. In this paper, we show that this failure arises from two fundamental issues: error accumulation across layers and, more critically, anisotropic distortion of the representation space. Based on this observation, we introduce a geometry-preserving output-alignment framework for 1-bit LLM quantization. First, we formulate output alignment as a constrained optimization problem that improves reconstruction while explicitly controlling changes in token similarity, preventing reconstruction gains from being achieved at the expense of representation geometry. Second, we propose a geometry-aware salient-mask selection strategy that identifies quantization configurations according to their ability to preserve token relations, rather than relying solely on reconstruction error. These components enable effective output-driven optimization while retaining the structural information lost by naive alignment. Extensive experiments across multiple LLM families and downstream tasks demonstrate consistent improvements over existing 1-bit PTQ methods.

Primary area:
foundation or frontier models, including LLMs
Keywords:
binary quantization, post training quantization, large language model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_beKWOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies why matching layer outputs can underperform weight reconstruction in binary LLM quantization. It combines reconstruction against full-precision outputs with a constraint on token affinity and a geometry-based rule for allocating additional binary terms to salient columns. Conditional analytical updates are accepted only when reconstruction does not worsen and the affinity score does not decrease. Experiments on OPT, LLaMA, and Qwen compare the method with existing binary quantizers, with additional ablations and measurements of calibration and inference costs.

Critical strengths

The empirical improvements are substantial and consistent. Table 1 improves all 18 perplexity comparisons against ARB-RC. Table 2 improves all 24 downstream task comparisons, with average gains of 2.55, 2.77, and 5.06 percentage points on the three LLaMA models. Table 3 extends the results to several Qwen architectures. This is convincing evidence that the method offers a useful improvement over the underlying binary quantizer.

The paper identifies a meaningful failure of unconstrained output reconstruction and offers an effective remedy. In Table 4, using the full-precision target alone worsens LLaMA-2-7B C4 perplexity from 20.56 to 29.12. Adding the affinity constraint reduces it to 18.94, before introducing the new mask. The exact acceptance check also makes the algorithm more reliable than relying on the diagonal approximation alone: it checks the actual reconstruction loss and affinity score after each proposed update.

Critical weaknesses

The geometry claim goes beyond what the constraint guarantees, and its empirical validation needs a clearer definition. Section 3 motivates preserving cosine similarities between tokens, but Equation (9) uses the unnormalized affinity (S_{\mathrm{Reg}}=\tfrac12|Y_{\mathrm{FP}}^\top\widehat Y|_F^2). This score can increase without improving normalized token relationships. The paper acknowledges that it is a surrogate, but the claim in Section 4.2 that accepted updates cannot sacrifice relational structure is too strong. Appendix A could support an empirical interpretation, yet Figure 2 labels values around (10^4) to (10^6) as MSE. Those cannot be an elementwise mean squared difference between cosine similarities, which is at most 4. The exact metric and normalization are therefore important to evaluating the central explanation.

The complete binary representation is not sufficiently specified. Section 4.7 assigns two binary terms to salient columns, whereas Equation (4) and Algorithms 1 and 2 describe a single term. It is unclear how refinement and the affinity check operate on the complete two-term output. Mask selection also changes the number of additional terms across groups, but Table 4 does not report the resulting effective bit widths, and Table 3 omits bit widths altogether. These omissions leave uncertainty about both the implemented objective and whether the mask comparisons use comparable binary capacity.

Questions for authors

What exactly is plotted in Appendix A, Figure 2? Please give the formula, including normalization and reduction over tokens and samples. If the figure uses the normalized token similarities defined in Section 3, explaining its scale would resolve much of my concern about the empirical geometry evidence.

Please provide the complete weight parameterization for salient columns and explain whether the acceptance test evaluates their combined output. What are the effective average bit widths for the mask variants in Table 4 and the Qwen models in Table 3? These details determine whether the reported results support improvement at comparable capacity and whether the stated updates describe the implemented quantizer.

Reviewer_WPKGOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper explores the question of why 1-bit quantization e.g. ARB-X performs worse than weight driven methods such as ARB-RC. They further claim that this stems from two fundamental issues, namely error accumulation across different layers as well as the representation space having anisotropic distortion. Following these observations, they attempt to solve this problem by first formulating output alignment as a constrained optimization problem with the goal of improving reconstruction and then they follow this by a geometry-aware salient-mask selection strategy that can identify specific quantization configurations by their ability to preserve the relation between tokens. They test these on multiple LLMs and tasks.

Critical strengths
  • The results show consistent gains. For example, in Table 1 and Table 2 their technique outperforms all of the other techniques across the board.
  • The paper is transparent and reports negative results as well (e.g. for Llama going from 20.56 to 29.12 in Table 4) and the authors include their proposed pseudocode for reproducibility.
Critical weaknesses
  • The output-driven refinement is only applied to the final fully connected layer of each block with the rest of the layers following ARB-RC.
  • The paper only evaluates perplexity and non-generation based datasets, leaving open the question of this technique's impact on actual realistic generation tasks. This is crucial as the results in Table 2 for example are not very far from random luck (e.g. 60% on WinoGrande which has a random baseline of 50%), questioning how much of the improvement is actually coming from their technique.
  • The technique requires more peak memory for quantization than ARB-X or ARB-RC on models such as Llama-2-7B.
Questions for authors
  • What are the actual average bit levels when measured and how do they compare to ARB-RC? In the cases where the bits are higher, how do the results compare at equal bit levels?
  • Can the authors add ARB-X to Table 2?
  • Can you compare your technique against the baseline on generation based datasets?
  • What happens if we refine all of the layers using this technique?
  • Can the authors extend Table 2 to include std as well? Some of the gains are marginal e.g. for COPA and it would be useful to know if they hold under multiple seeds, same as Table 1 already has.
Reviewer_qpJMOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses error accumulation and representation geometry distortion in 1-bit post-training quantization (PTQ) of large language models (LLMs) by proposing a geometry-preserving output alignment method. The proposed approach combines true-target output reconstruction, geometry-constrained projection based on a local second-order approximation, and geometry-aware salient-mask selection to mitigate quantization-induced performance degradation. Experiments on OPT, LLaMA, and Qwen models demonstrate performance improvements over existing methods. However, the novelty of the proposed approach and the necessity of several technical components remain insufficiently justified.

Critical strengths
  1. The paper addresses a practically relevant problem with clear research objectives.
  2. The overall motivation is intuitive and reasonably well presented.
  3. The proposed method enables parameter refinement through conditional closed-form updates without requiring costly retraining of the original model.
  4. The experiments cover multiple model families, including OPT, LLaMA, and Qwen, and evaluate perplexity, downstream task performance, and component-level ablations.
Critical weaknesses
  1. The core ideas and several technical formulations substantially overlap with prior work, leaving the independent novelty of the proposed method insufficiently established.

  2. The paper does not adequately establish the underlying relationship between the observed representation geometry distortion and downstream performance degradation.

  3. There are discrepancies between some optimization objectives and their theoretical motivations, while the advantages of the proposed components over existing approaches remain unclear.

  4. Although the method achieves performance improvements, the current experiments do not sufficiently identify the primary sources of these gains or systematically validate several key design choices.

  5. The paper provides insufficient analysis of applicability across quantization configurations, algorithmic stability, and the trade-offs between performance improvements and computational overhead.

  6. Other Comments

  7. The novelty of the proposed approach requires further clarification. A major concern with this paper is the substantial overlap between its core ideas and existing research. First, preserving pairwise similarities, distances, or angular relationships between representations has been extensively explored in model compression. For instance, [1] introduces similarity-preserving knowledge distillation by maintaining pairwise similarity structures between teacher and student representations, while [2] further investigates the preservation of relational information through distances and angles. Therefore, using Gram matrices to characterize and preserve geometric relationships between token representations is not, by itself, a sufficiently novel contribution. [3] addresses essentially the same problem as this paper. It also investigates error accumulation and anisotropic representation distortion in output-driven 1-bit quantization and proposes true-target output reconstruction and token-relation-preserving updates. In particular, the true-target reconstruction objective in [3] is identical to Equation (6) of this paper. Moreover, the geometry-related objective in Equation (9) is mathematically equivalent to the corresponding formulation in [3], up to a constant factor and notational differences, with both relying on the same Gram-matrix-based affinity. In addition, [3] already employs gradient-based masking to filter reconstruction-optimal updates and applies output-driven refinement to the final fully connected layer of each Transformer block. The relationship between the proposed method and [3] therefore requires explicit clarification. Furthermore, [4] also investigates the limitations of output alignment and the importance of preserving internal representation geometry in low-bit LLM compression. Although [4] focuses on quantization-aware distillation rather than 1-bit PTQ, its central motivation is closely related to that of this paper. In my view, the potentially distinctive contributions of this work lie primarily in the local second-order constrained projection and geometry-aware salient-mask selection, rather than the general principle of geometry preservation. I encourage the authors to provide direct comparisons with these closely related studies, clearly identify the technical problems that remain unresolved by prior approaches, and quantitatively demonstrate the incremental benefits of the newly introduced components.

  8. Section 3 does not sufficiently establish a causal relationship between geometry distortion and performance degradation. Section 3 motivates the proposed approach through Figures 1(a)–(c). Although these experiments provide useful observations, the evidence is insufficient to fully support the claimed relationship between geometry distortion and downstream performance degradation. First, Figure 1(a) shows that MSE-based optimization favors high-magnitude channels, while Figure 1(b) demonstrates that output-driven quantization can introduce greater token geometry distortion. However, the causal connection between these observations remains unclear. In particular, it is not established whether the magnitude-dependent optimization bias directly leads to geometry distortion or whether other factors, such as binary quantization constraints and parameter interactions, are responsible. Second, Figure 1(c) examines geometry distortion and perplexity under perturbations with identical Frobenius norms. While the results suggest a correlation, controlling the overall reconstruction error does not isolate the effect of geometry distortion. Different perturbation directions may also alter errors in important feature channels or directions to which subsequent layers are particularly sensitive. Therefore, the observed performance degradation cannot be attributed solely to geometry distortion. Furthermore, the relatively large perturbation magnitude of 0.5 raises questions about whether these artificial perturbations are representative of actual 1-bit quantization errors. I suggest that the authors conduct additional controlled experiments using realistic quantization errors and investigate whether geometry distortion remains strongly associated with downstream degradation after accounting for other factors, such as feature importance and downstream sensitivity. This would provide stronger evidence for the proposed motivation.

  9. The geometry constraint in Section 4 is not fully consistent with the theoretical motivation in Section 3. Section 3 characterizes token geometry using a normalized Gram matrix, whose entries correspond to pairwise cosine similarities between token representations. Accordingly, the proposed motivation primarily concerns preserving representation directions and relational structures rather than absolute feature magnitudes. However, Equation (9) in Section 4 introduces an unnormalized Gram-matrix affinity to maintain a tractable quadratic optimization structure. This surrogate is not equivalent to the normalized token geometry measure introduced earlier. Consider a simple case: [ \hat{Y}=cY,\qquad c>0. ] In this setting, all pairwise cosine similarities remain unchanged. Nevertheless, the proposed geometry affinity satisfies \frac{c^2}{2}|YY^\top|F^2. ] Therefore, the affinity changes with the global scaling factor even though the relative geometry of token representations remains identical. Consequently, the non-decreasing constraint on (S{\mathrm{Reg}}) in Equation (10) does not strictly guarantee preservation of the normalized token geometry emphasized in Section 3. Although the appendix provides visual evidence supporting the empirical effectiveness of this surrogate, its theoretical justification remains insufficient. The authors should more clearly distinguish unnormalized Gram affinity from cosine-similarity-based geometry and provide a quantitative analysis of their relationship. Furthermore, comparisons with alternative geometry-preserving objectives, such as normalized Gram-matrix distance, cosine similarity loss, or Centered Kernel Alignment would help establish the advantages of the proposed formulation.

  10. The theoretical contribution and practical necessity of the local second-order projection are insufficiently demonstrated. Sections 4.3–4.5 introduce a geometry-preserving projection based on a local second-order approximation. However, Proposition 1 only guarantees non-decreasing geometry affinity under the diagonal Hessian approximation, a property that largely follows from the mask definition in Equation (15), rather than providing a stronger guarantee for the original constrained optimization problem. Moreover, the coordinate-wise constraints may be overly conservative, as individually infeasible updates could potentially become feasible when combined. More importantly, prior work [3] has already employed first-order gradient information to select parameter updates, making the additional benefits of second-order information unclear. The results in Appendix Table 8 further raise concerns. On OPT-6.7B, changing (\beta) from 0 to 1 improves C4 perplexity only from 16.24 to 16.18, while slightly worsening WikiText2 perplexity from 14.56 to 14.59. These marginal differences call into question the practical contribution of the second-order term. I suggest that the authors provide more direct comparisons between first-order and second-order masking, including their effects on geometry preservation, convergence behavior, and downstream performance, to better justify the necessity of the proposed second-order approximation.

  11. The fairness and statistical reliability of the experimental evaluation require further investigation. First, the proposed method allocates different representational capacities to different weight columns. Although the paper reports average bit widths of approximately 1.08–1.11 bits, it remains unclear whether scaling parameters, mask metadata, and other storage overheads are consistently accounted for across competing methods. Second, while some OPT and LLaMA results are averaged over three random seeds, the Qwen experiments and several ablation studies lack comparable statistical analysis. Given the use of random projections and mask selection, additional evaluations are needed to establish the stability of the reported improvements, particularly considering the large variance observed in some LLaMA-2-7B results on PTB. Furthermore, Appendix Table 7 shows non-monotonic performance changes with increasing calibration set sizes, but provides limited explanation. I suggest that the authors clarify the effective storage budgets, report statistical variations for key experiments, and further evaluate robustness to calibration data, quantization configurations, and random initialization.

  12. The applicability and generalizability of the proposed method require further discussion. Section 4.8 and Appendix E.2 indicate that the proposed output-driven refinement is applied only to the final fully connected layer of each Transformer block, while the remaining linear layers continue to use ARB-RC. This design may help avoid excessive modifications to intermediate representations and reduce computational overhead. However, the paper does not provide sufficient experimental evidence to justify this particular layer-selection strategy. In particular, the central motivation of this work emphasizes the importance of preserving token geometry for subsequent Transformer computations. Nevertheless, attention projection layers and other intermediate linear layers may also substantially influence representation geometry. If geometry preservation is indeed a critical mechanism, its effectiveness could vary considerably depending on which layers are refined. It would therefore be valuable to investigate alternative layer-selection strategies, including applying the proposed refinement only to attention projection layers, only to MLP layers, or to all linear layers, in addition to the current configuration. Such experiments would help justify the implementation choices and clarify the applicability and limitations of the proposed geometry-preserving mechanism. References [1] F. Tung and G. Mori, "Similarity-Preserving Knowledge Distillation," Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1365–1374, 2019. [2] W. Park, D. Kim, Y. Lu, and M. Cho, "Relational Knowledge Distillation," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3967–3976, 2019. [3] D. A. Hoang et al., "Rethinking Output Alignment for 1-bit Post-Training Quantization of Large Language Models," arXiv preprint arXiv:2512.21651, 2025. [4] F. Tu et al., "Beyond Output Matching: Preserving Internal Geometry in NVFP4 LLM Distillation," arXiv preprint arXiv:2606.05682, 2026.

Questions for authors

See above

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.