Compression-Induced Representational Dissonance: Anticipating and Suppressing Prediction Flips

A compressed model can preserve benchmark accuracy while altering individual predictions, leading to potentially severe reliability failures. Across accuracy- preserving quantization, pruning, and low-rank compression of 58 vision models on 9 datasets, we find that models lose only 0.42 percentage points of accuracy on average while changing 9.4% of their predictions. We trace these changes to the interaction between compression error and the constraints it imposes, and to how they affect features of different energies. Compression disproportionately distorts low-variance feature directions, drowning them in noise. Whether these changes alter a prediction depends on the base model’s decision margin, linking differential representational change to decision instability. This relationship generalizes across compression families and model classes, allowing prediction flips to be anticipated before evaluating the compressed model on the target inputs. Across 453 configurations of vision and language models spanning quantization, sparsification, and low-rank, we predict the flip rates with a median relative error of 5.7%. And as we focus more on preserving representation during compression, we substantially restore decision fidelity and reduce flips at matched accuracy by 50 to 93%.