What Survives Is Not Enough: Loss-Aligned Directional Salience Calibration for LLM Compression

Low-rank decomposition facilitates the deployment of large language models (LLMs) by representing dense transformations within compact low-dimensional subspaces. Existing approaches primarily optimize reconstruction objectives to preserve approximation fidelity during compression. However, preserving a direction does not guarantee that its contribution is properly calibrated to the end-to-end language-modeling objective. Towards this end, we propose a novel post-compression framework termed Loss-aligned Directional Salience Calibration (LENS) for LLM compression. The core of LENS is to recalibrate what survives compression through two complementary components: loss-aligned direction identification and state-adaptive salience calibration. In particular, we characterize the first-order loss response to directional rescaling within the retained subspace and aggregate these responses into a response matrix whose eigendirections reveal modes that locally favor amplification or suppression. Subsequently, we estimate direction-specific relative gains using a response-based quadratic surrogate and calibrate their joint magnitude through a shared stage-level scale. The resulting calibration is accepted only when it achieves reliable loss reduction on a held-out validation set and is then folded directly into the existing low-rank factors, preserving the retained subspaces and ranks without introducing additional inference-time parameters or operations. Extensive experiments across multiple LLM backbones and downstream tasks demonstrate the effectiveness of LENS. We further show that LENS can be integrated with different low-rank compressors while improving their performance, supporting its use as a general post-compression calibration framework. The code is available at https://lens1.github.io/.