- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies temporary QK sharing during transform fitting for QKV-only ternary post-training quantization. The shared-prefix procedure (S) copies a jointly fitted rotation into independent Q and K variables and continues fitting. The primary control (C) fits independently throughout, with matched initialization and restart. Both procedures finish in the same transform family.
The authors report lower installed reconstruction error and improvements in NLL and downstream accuracy, particularly on OPT and Llama. Further experiments examine disagreements between reconstruction metrics and model performance through frozen-weight interventions. Quality evaluation uses densely reconstructed BF16 weights; the study does not demonstrate a low-bit execution advantage.
Critical strengths
The model-preserving split is a useful control: it separates the fitting procedure from the expressivity of the final model family. The supporting projection, copying and fixed-code update arguments appear sound.
The reported fitting improvement is not confined to identity initialization. Table 12 gives lower installed reconstruction error for S than for both C and M in every main configuration. On OPT and Llama, the main random-start comparisons also favor S in NLL, and the additional starts reinforce the finding. The cost controls and downstream results, including Llama’s random-start HellaSwag improvements, strengthen this empirical observation.
The authors report unfavorable cases and explicitly limit the interpretation of their evaluation intervals and endpoint interventions. The branch and depth interventions provide useful evidence about which differences between the fitted models affect loss.
Critical weaknesses
- The reported gains do not establish a practical quantization advance.
The paper reports improvements in NLL and downstream accuracy, particularly on OPT and Llama. However, these improvements are evaluated using reconstructed BF16 effective weights, and do not establish an improvement in the accuracy–compression or accuracy–inference-cost trade-off of a useful quantizer.
The distinction matters because Equation (1) uses arbitrary dense, branch-specific input rotations. Storing the reconstructed weights in BF16 does not compress the affected QKV matrices. Alternatively, retaining the ternary coefficients requires accounting for the learned rotations and their execution costs. For three square QKV projections, storing three dense rotations at the original weight precision already requires as many values as the original QKV weights, before accounting for the ternary coefficients and scales. More economical representations may be possible, but their effectiveness is not demonstrated here.
This is not simply a missing deployment benchmark. Reducing the transform overhead through a compact parameterization can change the fitting problem itself. For example, FlatQuant uses Kronecker-factored transforms to reduce transformation overhead, but restricting the available transforms changes the search space in which the sharing benefit is obtained. The improvement demonstrated with unrestricted rotations therefore cannot be assumed to survive in an efficient transform family.
Appendix E provides some evidence of usefulness within the studied setting. Depth-wise selection among fitted S/C endpoints produces a small confirmed Llama NLL gain over the selected unmixed parent, although the Qwen result is unresolved and the depth combinations do not consistently outperform whole-model schedule selection. The separate TWLA replacement and transform-transfer experiments do not establish an improvement over that pipeline. These results leave the practical relevance of the fitting improvement unresolved.
The finding may still be valuable as an empirical study of optimization, independently of any immediate deployment benefit. Its significance on that basis is considered in the following weakness.
- The study establishes a fitting improvement, but develops limited new understanding of transform search.
The main empirical finding deserves credit: temporary QK sharing lowers installed reconstruction error relative to the tested independent controls and improves NLL consistently on OPT and Llama across the reported starts. My concern is not that an empirical contribution requires a new theorem or must succeed on every model, but that the study establishes the effectiveness of this particular prefix without sufficiently characterizing what its joint fitting contributes.
Rotation initialization and preliminary transform optimization are already important parts of methods such as SpinQuant and FPTQuant. Those methods do not establish the present result; the potential advance here is the particular decision to pool Q and K during the prefix and then release the constraint. The restart and budget controls test whether that procedure helps relative to the specified independent routes, but do not distinguish the contribution of joint prefix fitting from a simpler transferred initialization or a more general temporary-coupling effect. Table 29 provides a useful partial distinction: QK and QV prefixes obtain similar reconstruction errors on Llama but substantially different NLL. Establishing what accounts for that difference under the main protocol would give the result a clearer scientific interpretation. Either a QK-specific effect or a more general coupling effect could be valuable.
The additional diagnostics address a related but different question: why better local reconstruction need not yield better model loss. Table 12 shows that projection-output error resolves the identity S–M ranking disagreements on OPT, Llama, and Qwen. The more interesting remaining cases include Ouro, where S improves both weight and input-aware reconstruction metrics but worsens NLL at the main random starts.
The frozen-weight interventions associate Ouro R1's reversal mainly with Q, but the paper does not connect that loss effect to the corresponding branch- and layer-level reconstruction changes. It therefore remains unclear whether the aggregate improvement conceals deterioration in the consequential components or whether the disagreement persists within those components. The amplitude-matched direction experiment demonstrates direction and context sensitivity for S–M interventions, but does not resolve this question for the primary S/C pair.
Consequently, the diagnostics characterize differences between fitted endpoints more clearly than they explain what makes temporary QK sharing beneficial or why its reconstruction improvements sometimes fail to translate into better NLL. The paper establishes a specific empirical improvement more convincingly than a broader, transferable lesson about fitting or evaluating quantization transforms. An analysis clarifying the role of joint prefix fitting, or an intervention-supported explanation connecting the primary S/C errors to their loss outcomes, could materially strengthen that contribution. Without such development, I find the scientific advance limited despite the care of the controlled comparison.
- The referenced numerical archive is unavailable
The reproducibility statement refers to a supplementary archive containing frozen source tables, plotting scripts, historical records and an audit of endpoint-selection NLLs and paired intervals, with numerical audit materials provided separately. These materials were not available. Please make the promised materials accessible.
This matters because the central result depends on a finite numerical procedure, not only the exact Procrustes formula. The implementation includes approximate proposals, FP64 rechecks, ternary-code selection, stopping decisions, selection of a saved state, and subsequent repair and casting. The manuscript specifies numerical safeguards and distinguishes fitting-state selection from installed reconstruction error. The remaining verification question is whether the main executions follow those rules and whether the selected states correspond to the evaluated tensors.
The restart and fitting-cost controls address their stated alternatives, but do not independently verify the common update implementation. Conversely, legitimate budget exhaustion, declared patience-based stopping, a harmless nonunique rotation, or an expected casting difference would not constitute an implementation error.
Please provide the complete numerical archive referenced in the manuscript, not only selected aggregate tables, together with the available code and states needed to trace the main comparisons. The audit should check the actual final proposals, the prescribed FP64 rechecks, acceptance thresholds and termination decisions against the declared rules, and separately trace the best selected states through repair, reconstruction and installation.
- No substantive theoretical contribution. The paper provides no theoretical result about the relative outcomes of finite shared and independent fitting. Its formal arguments concern feasible-set inclusion, exact ternary projection, standard Procrustes optimization, and algebraic decompositions of endpoint losses. None establishes a new property of the optimization process responsible for the reported improvement.
Overall assessment
I recommend Reject (1), primarily on contribution. The paper reports a potentially novel and supported warm-start effect, together with useful endpoint diagnostics and a small confirmed Llama endpoint-selection gain. However, it does not establish an improvement at a useful quantization cost, the theoretical work is not significant, and the connection between the fitting result, the diagnostic findings and transferable guidance remains limited. Even if the numerical record confirms all reported gains, I do not find the demonstrated combination sufficient for acceptance.
Providing the referenced numerical archive and a satisfactory audit would resolve the verification issue. For the contribution assessment, a focused result showing useful quality gains within a representation with fully accounted storage or inference cost would materially strengthen the practical case. Please also supply an archive for any new results made in rebuttal relevant to my review.
Questions for authors
1. Does the QK-versus-QV distinction in Table 29 persist under the main protocol?
A representative comparison at random starts, retaining the projector, sharing duration, restart rules and fitting budget, would help distinguish a QK-specific benefit from a more general benefit of temporary coupling. Please clarify which interpretation the existing evidence supports.
2. Can the available starts and error measurements be used to test an explanation of the S/C outcomes?
Please present the main random starts and the t=12 additional starts together, identifying any material protocol differences. For representative positive and negative S/C pairs, please relate per-branch/per-layer reconstruction-error changes to the loss interventions. In particular, does Ouro R1's aggregate improvement conceal worse Q reconstruction, and does the same pattern explain a start where S improves NLL? This request can begin with already fitted models rather than a large new seed sweep.
3. Can the referenced archive and a focused endpoint-invariance audit be supplied?
The supplementary archive described in the manuscript was not available in the submission materials I could access. Please provide it, preferably as one anonymous downloadable ZIP, with the fitting implementation, exact configurations, relevant solver states, endpoint tensors and evaluation outputs needed to verify the audited comparisons.
For the narrow K projections, please check whether the nonunique Procrustes completion affects the actual selected effective weights after re-projection, repair and casting. Rank deficiency alone does not establish a problem; cancellation at the installed endpoints would resolve this question. A controlled-completion rerun is needed only if consequential dependence remains unresolved. Reproducing the original endpoint scores alone would not establish completion invariance.
Please also supply an archive for any new claims you make in your rebuttal to me.
After reading over the other reviews and given the authors did not provide any rebuttal, I will maintain my initial reject assessment. I would encourage the authors to resubmit a major revision to another venue or consider a workshop given the narrow scope.