acceptodds
Under review as a conference paper at ICLR 2027

CurvProbe: Damage Has Two Factors, and the Checkpoint Sets One

Abstract

Post-training compression first has to decide where: which layers to delete, factorise or quantise. The criterion the field deploys ranks layers by displacement, how far an operation moves a layer's output, and never asks how much that move raises the loss. Expanding the loss where these operations inject their change splits the damage at a layer into that displacement times the loss's curvature along it. Second-order salience already estimates this curvature, one operation at a time. That curvature belongs to the frozen checkpoint, not to the operation, so it can be read once, before any operation is named. CurvProbe reads it as a curvature profile, one number per layer from forward passes, with no backward pass and no Hessian. The factorisation predicts that displacement ranks layers below chance on checkpoints whose curvature is highest where displacement is lowest, as on Llama-3-8B, where the attention sublayer ranked safest to delete costs the most; the profile flags such failures before any damage is measured. One profile, never shown the operation it is judged on, repairs the damage ranking across eighteen checkpoints and eleven operations. On Llama-3-8B its layer sets keep that gain once a quarter of the layers are compressed. After low-rank approximation they hold 65.8% LAMBADA accuracy, against Block Influence's 55.8%. A checkpoint can therefore ship with its profile, and a compressor that arrives later ranks its layers with one forward pass where measuring their damage would take .

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.