acceptodds
Under review as a conference paper at ICLR 2027

Curvature Is a Budget: Coordinates Before Resolution in Second-Order Quantization

Abstract

A weight rotation preserves the full-precision model but changes the Hessian that measures quantization error. Multiple public repositories contain code paths that pair transformed weights with untransformed curvature: we reproduce this error in ModelTC/LightCompress and identify the corresponding metric-collection pattern in spcl/QuaRot's export source. Users can unknowingly trigger this silent failure by omitting one setting in LightCompress: the run completes without a frame warning while Llama-3.1-8B perplexity rises from 6.525 to 7.456, worse than rotated rounding at 7.283. In a controlled four-bit panel, stale dense curvature makes GPTQ worse than round-to-nearest on five of seven checkpoints, while aligned block-32 curvature beats stale dense curvature on all seven. At three bits, the median stale-to-aligned dense-metric perplexity ratio reaches 25.31. We derive the exact output error of a complete GPTQ sweep under a mismatched metric and characterize when block projection commutes with coordinate transport, enabling exact transport within compatible block representations using O(db) stored coefficients. Curvature must be aligned before it is budgeted.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.