When Is More Context Worth It? Risk-Controlled Expansion for Compressed-Context Inference
Abstract
Context compression reduces the cost of long-context inference but can remove information needed for a particular query. Selective expansion can recover such information, yet uncertainty or context-sufficiency estimates only indicate whether the current answer should be trusted; they do not determine which part of the context would improve it or whether the expected improvement justifies its cost. We propose Risk-Controlled Progressive Expansion (RCPE), a cost-sensitive sequential framework for answering, restoring compressed context, or abstaining. RCPE separately estimates current-answer reliability and the value of candidate restoration actions, learning the latter from task-score changes observed after controlled context additions. After each restoration, it regenerates the answer and reassesses the resulting state. RCPE calibrates complete policies on independent data using simultaneous confidence bounds, controlling absolute task loss, compression-induced loss, and minimum coverage under i.i.d. sampling. Across LongBench and ZeroSCROLLS, RCPE recovers quality lost under fixed compression and improves the quality–cost trade-off over relevance-only restoration, while processing fewer context tokens and achieving lower end-to-end latency than full-context inference.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.