acceptodds
Under review as a conference paper at ICLR 2027

Shared or Independent? A Controlled Study of Transform Search in LLM Quantization

Abstract

Low-bit quantization reduces model storage but introduces approximation error. Rotations can reduce this error by changing the coordinates in which weights are quantized. In attention layers, query, key, and value projections can share an input rotation or use independent rotations. Although independent rotations offer greater flexibility, we show that temporarily sharing Q and K during fitting improves the final independent models on OPT and Llama under matched controls. We study this effect in QKV-only ternary post-training quantization. Our construction copies shared rotations into independent variables without changing the quantized model at the transition, then continues fitting independently. This enables a comparison with independent fitting from the outset while holding the final transform family fixed. We evaluate OPT-1.3B, Llama-3.2-3B, Qwen3-4B-Base, and Ouro-1.4B with matched initializations and restart rules. Temporary sharing lowers negative log-likelihood (NLL) on C4 and WikiText-2 in 19 of 24 model–initialization–corpus comparisons, with 18 improvements remaining significant after correction for multiple comparisons. Additional fitting-budget controls retain the gains on OPT and Llama. Across HellaSwag, ARC-Challenge, and PIQA, normalized accuracy improves in 27 of 36 comparisons, including ten significant improvements after correction. On Llama-3.2-3B, HellaSwag accuracy increases by 1.20 and 0.89 percentage points at the two random initializations. Benefits depend on initialization, sharing duration, and evaluation corpus. Lower reconstruction error can still accompany higher NLL even after accounting for model inputs. Interventions on the fitted models further identify the contributions of Q, K, and layer depth to these performance differences. These results establish temporary sharing as a useful fitting strategy in the studied setting and show why reconstruction error, language-model loss, and downstream accuracy must be distinguished when evaluating quantized models.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language model quantization, post-training quantization, ternary weight quantization, rotation-based quantization, QKV projections, transform sharing, optimization path dependence, reconstruction error

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_ZW69OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies temporary QK sharing during transform fitting for QKV-only ternary post-training quantization. The shared-prefix procedure (S) copies a jointly fitted rotation into independent Q and K variables and continues fitting. The primary control (C) fits independently throughout, with matched initialization and restart. Both procedures finish in the same transform family.

The authors report lower installed reconstruction error and improvements in NLL and downstream accuracy, particularly on OPT and Llama. Further experiments examine disagreements between reconstruction metrics and model performance through frozen-weight interventions. Quality evaluation uses densely reconstructed BF16 weights; the study does not demonstrate a low-bit execution advantage.

Critical strengths

The model-preserving split is a useful control: it separates the fitting procedure from the expressivity of the final model family. The supporting projection, copying and fixed-code update arguments appear sound.

The reported fitting improvement is not confined to identity initialization. Table 12 gives lower installed reconstruction error for S than for both C and M in every main configuration. On OPT and Llama, the main random-start comparisons also favor S in NLL, and the additional starts reinforce the finding. The cost controls and downstream results, including Llama’s random-start HellaSwag improvements, strengthen this empirical observation.

The authors report unfavorable cases and explicitly limit the interpretation of their evaluation intervals and endpoint interventions. The branch and depth interventions provide useful evidence about which differences between the fitted models affect loss.

Critical weaknesses
  1. The reported gains do not establish a practical quantization advance.

The paper reports improvements in NLL and downstream accuracy, particularly on OPT and Llama. However, these improvements are evaluated using reconstructed BF16 effective weights, and do not establish an improvement in the accuracy–compression or accuracy–inference-cost trade-off of a useful quantizer.

The distinction matters because Equation (1) uses arbitrary dense, branch-specific input rotations. Storing the reconstructed weights in BF16 does not compress the affected QKV matrices. Alternatively, retaining the ternary coefficients requires accounting for the learned rotations and their execution costs. For three square QKV projections, storing three dense rotations at the original weight precision already requires as many values as the original QKV weights, before accounting for the ternary coefficients and scales. More economical representations may be possible, but their effectiveness is not demonstrated here.

This is not simply a missing deployment benchmark. Reducing the transform overhead through a compact parameterization can change the fitting problem itself. For example, FlatQuant uses Kronecker-factored transforms to reduce transformation overhead, but restricting the available transforms changes the search space in which the sharing benefit is obtained. The improvement demonstrated with unrestricted rotations therefore cannot be assumed to survive in an efficient transform family.

Appendix E provides some evidence of usefulness within the studied setting. Depth-wise selection among fitted S/C endpoints produces a small confirmed Llama NLL gain over the selected unmixed parent, although the Qwen result is unresolved and the depth combinations do not consistently outperform whole-model schedule selection. The separate TWLA replacement and transform-transfer experiments do not establish an improvement over that pipeline. These results leave the practical relevance of the fitting improvement unresolved.

The finding may still be valuable as an empirical study of optimization, independently of any immediate deployment benefit. Its significance on that basis is considered in the following weakness.

  1. The study establishes a fitting improvement, but develops limited new understanding of transform search.

The main empirical finding deserves credit: temporary QK sharing lowers installed reconstruction error relative to the tested independent controls and improves NLL consistently on OPT and Llama across the reported starts. My concern is not that an empirical contribution requires a new theorem or must succeed on every model, but that the study establishes the effectiveness of this particular prefix without sufficiently characterizing what its joint fitting contributes.

Rotation initialization and preliminary transform optimization are already important parts of methods such as SpinQuant and FPTQuant. Those methods do not establish the present result; the potential advance here is the particular decision to pool Q and K during the prefix and then release the constraint. The restart and budget controls test whether that procedure helps relative to the specified independent routes, but do not distinguish the contribution of joint prefix fitting from a simpler transferred initialization or a more general temporary-coupling effect. Table 29 provides a useful partial distinction: QK and QV prefixes obtain similar reconstruction errors on Llama but substantially different NLL. Establishing what accounts for that difference under the main protocol would give the result a clearer scientific interpretation. Either a QK-specific effect or a more general coupling effect could be valuable.

The additional diagnostics address a related but different question: why better local reconstruction need not yield better model loss. Table 12 shows that projection-output error resolves the identity S–M ranking disagreements on OPT, Llama, and Qwen. The more interesting remaining cases include Ouro, where S improves both weight and input-aware reconstruction metrics but worsens NLL at the main random starts.

The frozen-weight interventions associate Ouro R1's reversal mainly with Q, but the paper does not connect that loss effect to the corresponding branch- and layer-level reconstruction changes. It therefore remains unclear whether the aggregate improvement conceals deterioration in the consequential components or whether the disagreement persists within those components. The amplitude-matched direction experiment demonstrates direction and context sensitivity for S–M interventions, but does not resolve this question for the primary S/C pair.

Consequently, the diagnostics characterize differences between fitted endpoints more clearly than they explain what makes temporary QK sharing beneficial or why its reconstruction improvements sometimes fail to translate into better NLL. The paper establishes a specific empirical improvement more convincingly than a broader, transferable lesson about fitting or evaluating quantization transforms. An analysis clarifying the role of joint prefix fitting, or an intervention-supported explanation connecting the primary S/C errors to their loss outcomes, could materially strengthen that contribution. Without such development, I find the scientific advance limited despite the care of the controlled comparison.

  1. The referenced numerical archive is unavailable

The reproducibility statement refers to a supplementary archive containing frozen source tables, plotting scripts, historical records and an audit of endpoint-selection NLLs and paired intervals, with numerical audit materials provided separately. These materials were not available. Please make the promised materials accessible.

This matters because the central result depends on a finite numerical procedure, not only the exact Procrustes formula. The implementation includes approximate proposals, FP64 rechecks, ternary-code selection, stopping decisions, selection of a saved state, and subsequent repair and casting. The manuscript specifies numerical safeguards and distinguishes fitting-state selection from installed reconstruction error. The remaining verification question is whether the main executions follow those rules and whether the selected states correspond to the evaluated tensors.

The restart and fitting-cost controls address their stated alternatives, but do not independently verify the common update implementation. Conversely, legitimate budget exhaustion, declared patience-based stopping, a harmless nonunique rotation, or an expected casting difference would not constitute an implementation error.

Please provide the complete numerical archive referenced in the manuscript, not only selected aggregate tables, together with the available code and states needed to trace the main comparisons. The audit should check the actual final proposals, the prescribed FP64 rechecks, acceptance thresholds and termination decisions against the declared rules, and separately trace the best selected states through repair, reconstruction and installation.

  1. No substantive theoretical contribution. The paper provides no theoretical result about the relative outcomes of finite shared and independent fitting. Its formal arguments concern feasible-set inclusion, exact ternary projection, standard Procrustes optimization, and algebraic decompositions of endpoint losses. None establishes a new property of the optimization process responsible for the reported improvement.

Overall assessment

I recommend Reject (1), primarily on contribution. The paper reports a potentially novel and supported warm-start effect, together with useful endpoint diagnostics and a small confirmed Llama endpoint-selection gain. However, it does not establish an improvement at a useful quantization cost, the theoretical work is not significant, and the connection between the fitting result, the diagnostic findings and transferable guidance remains limited. Even if the numerical record confirms all reported gains, I do not find the demonstrated combination sufficient for acceptance.

Providing the referenced numerical archive and a satisfactory audit would resolve the verification issue. For the contribution assessment, a focused result showing useful quality gains within a representation with fully accounted storage or inference cost would materially strengthen the practical case. Please also supply an archive for any new results made in rebuttal relevant to my review.

Questions for authors

1. Does the QK-versus-QV distinction in Table 29 persist under the main protocol?

A representative comparison at random starts, retaining the projector, sharing duration, restart rules and fitting budget, would help distinguish a QK-specific benefit from a more general benefit of temporary coupling. Please clarify which interpretation the existing evidence supports.

2. Can the available starts and error measurements be used to test an explanation of the S/C outcomes?

Please present the main random starts and the t=12 additional starts together, identifying any material protocol differences. For representative positive and negative S/C pairs, please relate per-branch/per-layer reconstruction-error changes to the loss interventions. In particular, does Ouro R1's aggregate improvement conceal worse Q reconstruction, and does the same pattern explain a start where S improves NLL? This request can begin with already fitted models rather than a large new seed sweep.

3. Can the referenced archive and a focused endpoint-invariance audit be supplied?

The supplementary archive described in the manuscript was not available in the submission materials I could access. Please provide it, preferably as one anonymous downloadable ZIP, with the fitting implementation, exact configurations, relevant solver states, endpoint tensors and evaluation outputs needed to verify the audited comparisons.

For the narrow K projections, please check whether the nonunique Procrustes completion affects the actual selected effective weights after re-projection, repair and casting. Rank deficiency alone does not establish a problem; cancellation at the installed endpoints would resolve this question. A controlled-completion rerun is needed only if consequential dependence remains unresolved. Reproducing the original endpoint scores alone would not establish completion invariance.

Please also supply an archive for any new claims you make in your rebuttal to me.

Reviewer_RKGzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies the role of rotation in ternary post-training quantization of the query, key and value matrices in a transformer architecture. Both rotation and ternary post-training quantization have been previously established as useful recipes in quantization literature. Instead, the goal of this paper is to study the effect of shared optimization vs independent optimization for the query, key and value branches. The following are the most important variants which are useful to keep in mind while reading the paper.

  • Route S: Here query, key matrices share rotation for 12 rounds and then they are decoupled. The value matrix maintains an independent rotation for these 12 rounds. Thereafter all 3 matrices are optimized independently.
  • Route C: Fits all the branches independently. The results largely show that S improves over C in NLL and downstream accuracy. Though the difference, based on my reading, is rather small and is also not consistent across models, particularly for Ouro and Qwen. Ouro is also not as popular as some of the other models so clarifying how it differs from other models would help a reader. There is also the question of discrepancy between choosing a scheme based on NLL, downstream task accuracy or reconstruction error. The paper also does not provide a coherent conclusion that would be useful from a practitioner's standpoint.
Critical strengths
  • S1: The paper does a principled comparison of various approaches. The variants covered are carefully designed to pinpoint the effect from a causal perspective. The paper also does a good job in carefully reporting competing diagnostics and reversals, ties, multistart control and other controls one would normally think of. The results are unfortunately mixed with no clear takeaways for practitioners.
Critical weaknesses
  • One primary weakness with the paper is the lack of a concrete takeaway for a practitioner. The paper does not establish which scheme should be chosen over the other in a practical deployment quantization pipeline. The disagreement between reconstruction error, NLL and task accuracy also complicates matters.

  • The presentation is generally alright though it can be improved. Better names could have been chosen for the route names, etc. A lot needs to be remembered to prevent going back and forth. One thing that could improve the paper is to think how this can be avoided. I do not have a good suggestion for this. The paper dives right into the optimization details without providing a simple account of overall story. Furthermore, several important qualifications and comparison with Full Precision results have been relegated to the appendix.

  • The practical deployment question is not adequately answered. The paper only explores the ternary quantization for Q, K and V matrices. There are other components such as MLP layers, activation quantization and KL cache quantization which are also used in conjunction with weight quantization. How should a practitioner think about these interactions hasn't been discussed.

  • The authors primarily focus on metrics degradation as the primary vehicle in in their study. Both ternary quantization and rotation as established techniques and the main contribution here is studying the effect of shared weight quantization vs independent weight quantization. This makes the overall scope narrow. The comparison with full-precision results is often relegated to appendix. The paper also doesn't study the other benefits of quantization in conjunction e.g. inference speedups, storage savings, etc. The overall read is that SS is marginally better than CC. Viewed from that light, it is a useful fact worth studying but lacking other important takeaways for practitioners lacks overall significance. The overall motivation and framing could be sharpened.

  • As a followup to my last point. What if the experiments were repeated at different scales and not just ternary quantization. That could be fruitful direction worth exploring to improve the paper. Another possible angle could be incorporating this in Quantization aware training schemes.

  • The paper evaluates only a few starting rotations, and the winner sometimes changes with the starting point. Its confidence intervals measure evaluation-data noise, not necessarily the variability in the fit. Therefore, the results do not yet show that temporary sharing is reliably better across repeated runs or better than spending the same compute on multiple independent runs.

Questions for authors

I do not have specific questions but would appreciate authors comments about some of the weaknesses that I highlighted above.

Reviewer_JZqqOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies the effect of different fitting strategies for transformations that improve weight quantization. Specifically, they focus on the transformation for QKV which all share the same inputs, therefore a natural question is whether to fit the transform for these 3 linear layers jointly or independently. The paper studies this question in-depths for ternary weight quantization. Their analysis shows that while independent transforms are strictly more flexible, it can be beneficial to temporary jointly optimize the transforms. They also show that a lower reconstruction error is not always leading to a lower NLL, this could suggest that the temporary sharing might have a regularization effect (reviewer interpretation).

Critical strengths
  • The paper asks an interesting questions with how sharing a transforms can influence the optimization trajectory and final results. I haven’t seen this extensively explored in literature before.
  • The experimental evaluation is done very carefully and accounting for many different factors. The authors isolate well the effect they study.
  • The paper places itself well in perspective of prior literature (although then misses later the opportunity to align their analysis better with it).
Critical weaknesses
  • The paper has a very narrow scope in many dimensions. It only studies whether during optimization of the transform it is better to keep Q/K/V transforms shared or not, they do not study other places in the LLM that could share transforms like up project and gate project. Theoretically even other layers could share the same transformation. The paper does also not consider activation aware weight quantization, both for fitting the transformation and also for the weight reconstruction error, which are both commonly done in most PTQ literature. Also they only study one specific quantization setting (ternary per-channel). Overall I have strong concerns that the findings are very specific for their chosen setup and are not more broadly applicable for the wider PTQ literature.
  • It is unclear how their results translate to transform-learning based approaches common in literature. They use an optimization procedure based on alternating ternary projection and Orthogonal Procrustes updates, which to my knowledge in this form is not common in PTQ literature (nor they cite prior art) and also don’t compare it with prior art (e.g. they cite SpinQuant and FPTQuant in their related for on fitting which both use SGD). For a pure analysis paper, it is essential that they align with prior literature and if they have to deviate for it clearly motivate the why and carefully compare to literature.
  • The paper is linguistically well written but has an odd read that leaves several questions open when reading. It basically misses building a clear storyline for the reader to follow. A couple of points:
    • In the introduction it jumps immediately to a very detailed technical part in the first sentence (that QKV could share a transform), but it is unclear what transform, what they care about or investigate etc.
    • They never really motivate their research question and it only becomes implicitly clear when reading longer.
    • They spend only a couple of lines on their fitting procedure (sec 3.3), which is the most crucial part of the paper and might influence significantly the results.
Questions for authors
  • How would sharing transforms affect SGD based transform-optimizations?
  • How would sharing interact with weight rounding/correction approaches?
1 comment

After reading over the other reviews and given the authors did not provide any rebuttal, I will maintain my initial reject assessment. I would encourage the authors to resubmit a major revision to another venue or consider a workshop given the narrow scope.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.