acceptodds
Under review as a conference paper at ICLR 2027

Same Behavior, Different Compression: Auditing Coordinate Dependence

Abstract

Can a compression audit change its verdict when reference behavior stays fixed? On 2,669 Pets examples previously unevaluated in this project, we test a transformation that doubles the feature components in the reference nullspace while preserving the fixed reference behavior. At the fixed setting of random projection to 64 dimensions, agreement falls from 91.7% to 81.2%. The change from passing to failing occurs in all 20 fresh projection draws. We also compare maps without reducing dimension and maps that retain the same subspace on ViT/CIFAR-10, DINOv2/CIFAR-10, DINOv2/Pets and BERT/TREC. These maps retain the same available linear score functions but can yield different fitted predictions. When the regularization objectives match, separate fits give the same class predictions as the orthonormal-coordinate baseline in all tested pairs. Some logit and probability differences still exceed the declared tolerances. Classical covariance equivalence and whitening principles give explicit invariance conditions for the complete randomized audit. Using verified LayerNorm support greatly improves conditioning. None of the four settings gains a feasible compressed point at the tested under the ordinary isotropic Gaussian-RP probe. The tests of matching objectives and correcting support use benchmark examples examined earlier. They are not independent confirmation on new data. Compression audits should distinguish reference behavior, retained information and access through a fitted procedure.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.