- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
TORQUE allocates an expected bit budget jointly across high-precision retention before rotation, high-precision retention after rotation, and quantization of the remaining values. It retains the largest input coordinates, normalizes and rotates the residual without reducing its dimension, and uses offline codebooks fitted to truncated Gaussian distributions. Under Haar rotation and exact retention, the paper proves that top-k input retention minimizes an NMSE upper bound for each k. A Gaussian surrogate and a largest-feasible-k reduction make parameter selection efficient. Experiments wrap EDEN, RaBitQ and TurboQuant, evaluate captured Qwen2.5-0.5B activations and KV caches, and measure reconstruction and retrieval on GloVe and GIST. Additional results cover HIGGS under the Gaussian model and encoding/decoding time. The main contribution is the joint allocation and its optimization structure; the individual retention mechanisms have prior precedents.
Critical strengths
- The joint optimization has a useful and simple structure. Retaining input outliers reduces residual energy while consuming bits that could improve the inlier quantizer. The paper accounts for retained values and indices, and shows that, for fixed k and the stated assumptions, top-k minimizes the residual-energy bound.
Holding the rotation dimension fixed also permits efficient evaluation of the largest feasible k for each codebook candidate (Section 2.3 and Appendix A). This turns a potentially combinatorial design into a practical per-vector choice. The paper appropriately distinguishes the rigorous bound from its Gaussian approximation.
- The empirical evidence supports a real reconstruction benefit across several quantizer families. The retrieval evaluation uses the complete million-scale GloVe and GIST databases, and the synthetic pre/post/joint comparison helps explain which input distributions favor each stage (Figures 3, 5, 7 and 8).
Reporting optimization-inclusive timings on multiple platforms is useful for assessing cost. The paper also explicitly acknowledges that lower NMSE does not consistently improve GIST Recall@1.
Critical weaknesses
- The practical equal-storage comparisons and unbiased variants cannot yet be adequately verified.
Equations (4)-(5) use a direction-independent rotation model, while the implementations use randomized Hadamard transforms and the KV heads have only 64 coordinates. The exact transform composition and the probability used for feasibility are not specified sufficiently to check the expected budget. A single RHT does not generally have Haar marginals, so the model's feasibility does not automatically imply the implemented expected-rate constraint. Moreover, Section 2 says metadata belongs in the total budget, but the application experiments do not give per-method, per-chunk bit ledgers or measured complete encoded rates; Appendix D explicitly uses a different convention with an additional 160 bits per vector for both methods. The manuscript also does not define the truncated-inlier reconstruction and correction formulas for the named unbiased baselines. These omissions affect the central claim of improved error at equal storage, particularly at small dimensions. They do not establish that the experiments exceed their budgets or that the implementations are biased, but prevent checking those possibilities.
- The experiments do not sufficiently establish the added application value of joint retention. Post-rotation retention and truncated-distribution quantization already appear in QUIC-FL, as the paper acknowledges, but the matched pre-only/post-only/joint accuracy ablation is limited to synthetic data and EDEN (Figure 5b).
The real-workload figures do not isolate the incremental gain from the new joint allocation. The LLM experiments measure captured-tensor NMSE for one 0.5B model, without inference using the quantized activations/caches or a downstream quality metric. For KV caches, Appendix C.1 additionally changes both the context count and the retention policy, so it cannot isolate the stage contribution. This matters because GIST demonstrates that NMSE and task quality can diverge, and joint retention, including parameter optimization, increases CUDA encoding time by 48.4-99.7% in the tested setting, with larger overheads in Appendix D. A matched application ablation and an inference-quality result would make the contribution's practical significance much more convincing.
Questions for authors
-
Budget and rotation clarification: For each application figure, what exactly does the horizontal-axis budget include? Please specify the random transform, the retention probability used by the optimizer, and the bits for norms/corrections, counts, codebook identifiers, indices and padding. Can you report the measured mean complete encoded bits per coordinate at the target budgets, particularly for d = 64, 128 and 256, together with modeled versus observed retention rates? An expected budget is appropriate to the formulation. The question is whether that expectation is met by the actual implementation.
-
Quantizer clarification: What are the exact reconstruction/debiasing formulas for EDEN, RaBitQ and TurboQuant after truncation, including fractional-bit allocation and the treatment of retained values? Does the TurboQuant comparison retain the original residual-QJL construction or use a different correction? Please give the candidate sets, codebook fitting procedure, accuracy-experiment precision and error-bar definitions, and clarify whether measured NMSE includes all stored-value and scale-rounding errors. These details determine whether the reported comparisons implement the intended baselines.
-
Evidence needed to resolve the significance concern: Do matched pre-only/post-only/joint results on the same real inputs and complete storage budgets show a material advantage for joint allocation? Is there a result measuring inference quality with the quantized KV cache or activations actually used by the model? Please also explain the relationship of the post-only variant to QUIC-FL. These are targeted checks of the submitted application claims, rather than requests for a new application domain.