ESPTQ: Post-Training Ternarization of Weights and KV Cache for Eliminating Per-Element Multiplications

Reading the weights and KV cache dominates large language model inference. Ternary quantization to -1, 0, +1 shrinks both and removes per-element multipliers: each weight product becomes a sign flip or skip, so a signed adder tree suffices. Ternary weights alone, however, leave multipliers in attention's $Q{K}^{\top }$ and $PV$, whose operands are both run-time floating-point tensors. We present ESPTQ (element-sparse-plane ternary post-training quantization), which quantizes the weights and KV cache into one ternary storage format, removing per-element multipliers from every matrix product inside the Transformer layers except the rotation of layer inputs. By design, ESPTQ caps the residual multiplication ratio at the weight-side budget of two multiplications per 128-element reduction ($2/128=1.5625\%$). For weights, after a learnable butterfly rotation within each 128-column group, a rate-distortion criterion selects, at a continuously variable second-plane assignment ratio $\rho$, which elements receive a sparse second ternary plane. For the KV cache, static tables replace per-token offsets and the V-side scaling factor, meeting the same budget. We compare methods on effective bits per weight, serializing each in its own storage format with losslessly compressed discrete codes, so scaling factors, rotation angles and other auxiliary metadata count. On LLaMA-2-7B, ESPTQ matches TWLA in perplexity and average zero-shot accuracy at less than half the effective bits, and at comparable bits lowers perplexity by 36% against PT2-LLM. Ternarizing the KV cache as well shrinks it to about one fifth of its FP16 size, at less cost in perplexity and zero-shot accuracy than ternarizing the weights.