DiffContext: Learning Adaptive Budgets for Risk-Controlled Context Compression

Long contexts expand the information available to language models, but increase the cost of inference. Context compression offers a way to reduce this cost, yet its savings may come at the expense of evidence essential to the downstream task. The amount of context needed to preserve task performance varies across inputs: some contain repeated evidence, whereas others require complementary information from several passages. Applying the same retention budget across inputs can therefore discard necessary evidence in some cases while retaining unnecessary content in others. These differences motivate adaptive budget selection that balances token savings against the risk of task degradation. However, directly evaluating task quality at multiple budgets for each new input would require repeated task-model calls, potentially offsetting the savings. To address the challenge, we introduce Differentiated Context Compression (DiffContext), a task-aware, plug-in budget controller for existing compressors with adjustable retention budgets. Keeping the compressor and task model frozen, DiffContext uses offline task feedback to estimate harm and token cost across budgets and construct input-dependent policies. Because predicted harm alone does not guarantee risk control, independent calibration selects a deployment policy that limits the population frequency of utility drops beyond a specified tolerance relative to full-context inference, with a finite-sample guarantee under i.i.d. calibration and deployment sampling. At deployment, the selected policy chooses a budget from input features, requiring at most one compressor call and one task-model call per input. Across twelve configurations spanning three datasets and five compression backbones, DiffContext increases mean task-model token-cost savings from 3.61% for certified fixed budgets to 12.14% on the evaluated budget grid, under the same 10% limit on harm frequency. These results support adaptive budget control as a reusable way to retain compression savings under an explicit task-risk requirement. The code and data are available at https://anonymous.4open.science/r/DiffContext-v1.