AdaMem: Distortion-Guided Adaptive Context Compression

Soft context compression improves long-context inference efficiency by replacing raw contexts with compact continuous memory representations. Existing methods, however, either use fixed compression budgets or adapt them according to indirect signals such as relevance, redundancy, or information density, without explicitly modeling the quality degradation induced by different compression ratios. We formulate context compression as a quality-constrained memory allocation problem and propose AdaMem, a distortion-guided framework that adaptively allocates memory budgets to individual context chunks according to their predicted compression sensitivity. Specifically, AdaMem employs a shared multi-ratio compressor to provide multiple compression budgets. Based on prefix representations, it predicts the distortion of each candidate ratio and selects the most aggressive ratio that satisfies a local predicted-distortion threshold. AdaMem further enables ratio selection before memory construction while reusing the prefix KV cache. Experiments on MRQA show that AdaMem consistently improves the quality–compression trade-off over fixed-ratio and adaptive baselines, reducing exported memory representations by 73.18% in-domain and 62.48% out-of-domain relative to fixed \(5\times\) compression, with consistent gains across 1B–8B model scales.