HYBRIDQUANT: CALIBRATION-FREE ENTROPYCODED E8 LATTICE QUANTIZATION OF THE KV CACHE

Compressing the key–value (KV) cache is central to long-context inference: for the compact and distilled models deployed where memory is scarce, the cache, not the weights, sets the limit. Established quantizers force a trade-off: calibration- free methods use fixed-rate scalar codes that leave a large gap to the rate–distortion limit, while entropy-coded compressors close the gap but fit their models to cal- ibration data. An entropy code needs no calibration if the vectors it sees are Gaussian, isotropic and nearly independent across coordinates. HybridQuant achieves this with three rounds of the randomized fast Walsh–Hadamard trans- form, 3d log2 d operations per vector instead of the d^2 of a dense random rotation, then rounds to the E8 lattice and codes with one static rANS code built for a Gaus- sian source; the step follows from the target rate alone, with every side bit counted. We prove how much rotation is needed: one round leaves a spike far from Gaus- sian (Kolmogorov distance 0.34), two bring every input close but leave a floor of order d −1/2 , and three lower it to order 1/d up to a logarithm (0.035 versus 0.0097 at d = 128, computed exactly). A dithered variant has input-independent errors, unbiased attention logits and an attention concentration bound that does not grow with context length, and an exact, calibration-free centering removes the collapse that the projection biases of Qwen2 models cause in rotation-based quantizers. On Qwen2.5-7B, Mistral-7B-v0.3 and Llama-3.1-8B the rate is within 0.003 bits of its target, and at 2.125 bits TurboQuant-MSE has 8.8%, 13.3% and 31.3% higher perplexity than HybridQuant, and a rotated entropy-coded scalar quantizer 2.3%, 3.0% and 6.5% higher, all significant under a paired bootstrap.