acceptodds
Withdrawn from ICLR 2027

When Is More Context Worth It? Risk-Controlled Expansion for Compressed-Context Inference

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Context compression reduces the cost of long-context inference but can remove information needed for a particular query. Selective expansion can recover such information, yet uncertainty or context-sufficiency estimates only indicate whether the current answer should be trusted; they do not determine which part of the context would improve it or whether the expected improvement justifies its cost. We propose Risk-Controlled Progressive Expansion (RCPE), a cost-sensitive sequential framework for answering, restoring compressed context, or abstaining. RCPE separately estimates current-answer reliability and the value of candidate restoration actions, learning the latter from task-score changes observed after controlled context additions. After each restoration, it regenerates the answer and reassesses the resulting state. RCPE calibrates complete policies on independent data using simultaneous confidence bounds, controlling absolute task loss, compression-induced loss, and minimum coverage under i.i.d. sampling. Across LongBench and ZeroSCROLLS, RCPE recovers quality lost under fixed compression and improves the quality–cost trade-off over relevance-only restoration, while processing fewer context tokens and achieving lower end-to-end latency than full-context inference.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
context compression, long-context inference, adaptive context expansion, risk control, selective prediction

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Lt1tOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes RCPE, a sequential controller for compressed-context inference. Starting from a compressed representation, it decides whether to answer, restore one or more context chunks, or abstain. It trains reliability and restoration-utility heads from counterfactual restorations, then selects a policy using simultaneous bounds on compression risk, absolute loss, and coverage.

Critical strengths

The policy-level formulation is well motivated. In particular, the paper recognizes that a useful restoration policy must account for repeated generation, context access, and the possibility that a restoration is harmful, rather than treating context expansion as free retrieval. The failure-stage analysis and the explicit cost accounting are also useful.

Critical weaknesses

The empirical evidence does not support the paper’s broad practical claims. In the main table, RCPE is usually below LongLLMLingua, which is the strongest aggregate baseline in both backbone settings; the reported gains are mainly over weaker base compressors. The risk-control claim is demonstrated only for one LCLM/Qwen configuration on LongBench. The other compressor/backbone combinations and ZeroSCROLLS are used for quality comparisons but do not receive the same certification analysis, so the central guarantee is not shown to transfer.

The controller is trained with many counterfactual generations whose labels are task-score changes measured against full-context behavior. This is an expensive and highly protocol-dependent source of supervision. The paper does not make clear how sensitive the result is to the trajectory orders, candidate chunking, and policy-family choices, nor does it report the total offline generation budget. In addition, the table’s “matched” operating point combines different methods with different policies and coverage behavior, making the quality–cost comparison harder to interpret than the presentation suggests. These are central experimental and validity issues rather than small presentation problems.

The related-work coverage is also incomplete. The paper should position RCPE against the following directly relevant works:

  • COMI: Coarse-to-Fine Context Compression via Marginal Information Gain — ICLR 2026
  • GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignment — ACL 2026
  • RECOMP: Improving Retrieval-Augmented Language Models with Compression and Selective Augmentation — EMNLP 2023
  • In-context Autoencoder for Context Compression in a Large Language Model — arXiv, 2023
  • Long-Context LLMs Meet RAG: Overcoming Challenges for Long Context Understanding — ACL 2024

These omissions make it harder to assess the novelty of the proposed controller relative to recent compression and retrieval-compression settings.

Questions for authors

Please report risk and coverage certification for every backbone–compressor pair and for ZeroSCROLLS. What is the total number of offline generations used to construct the controller labels, and how were trajectory order, chunk size, and the candidate policy family fixed before evaluation? Please also provide a comparison at a clearly identical coverage and end-to-end cost, including the strongest baseline rather than only the underlying compressor.

Reviewer_dtPLOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Risk-Controlled Progressive Expansion (RCPE), a framework for deciding when restoring compressed context justifies its computational cost. RCPE separately estimates current-answer reliability and the expected task-score improvement from restoring each candidate chunk, learning restoration values from controlled context additions. Its controller can answer, restore selected chunks and regenerate, or abstain. Independent calibration selects a low-cost policy from a fixed family using simultaneous bounds on absolute task loss, compression-induced loss, and answer coverage, with guarantees under i.i.d. sampling.

Validation covers LongBench and ZeroSCROLLS, two model backbones, and both latent and textual compressors. Experiments include risk-control audits, matched-budget comparisons against relevance-based restoration, component ablations, end-to-end cost measurements, and failure analysis. RCPE consistently improves its underlying compressors and selects more beneficial restorations than BM25. The reported Qwen configurations reduce end-to-end latency by approximately 20–24% relative to full-context inference. Ablations identify restoration-outcome supervision as the largest contributor to quality gains. Overall, the results support separating answer confidence from restoration value, although LongLLMLingua retains the highest aggregate task scores.

Critical strengths

The separation of answer reliability from restoration value is well motivated. An unreliable answer does not establish which additional context would help or whether its benefit justifies the cost. RCPE addresses these decisions separately and trains restoration values from observed task-score changes, including harmful and ineffective additions. This gives the controller a concrete objective aligned with useful information acquisition rather than relevance alone.

Critical weaknesses

W1. The incremental contribution over existing learned expansion and utility-based selection is insufficiently established. RCPE’s distinguishing contribution lies in state-conditioned marginal gains and calibrated cost-sensitive control. However, its main restoration-selection experiment compares against random selection and BM25, without a comparably trained selector operating under the same calibration and budget. These comparisons do not isolate the benefit of the proposed formulation from stronger supervision and neural ranking, leaving the central methodological contribution under-supported.

W2. Material inconsistencies undermine the central quality–cost evaluation. Section 5.5 describes Figure 4 as using Qwen configurations, yet it includes AutoCompressor, which Section 5.1 pairs with Llama-2-7B. Figure 4 also plots AutoCompressor and RCPE(AutoCompressor) at approximately 38.0 and 38.9 aggregate points, whereas Table 1 reports 34.8 and 37.1. The paper does not explain whether these differences reflect distinct backbones, subsets, or operating points. Consequently, the principal efficiency comparison cannot be reliably interpreted. Because improved quality–cost trade-offs are the main practical contribution, these discrepancies materially weaken the evidence supporting acceptance.

Questions for authors

Q1 Which backbones, evaluation subsets, and operating points are used in Figure 4? Section 5.5 describes a Qwen comparison, but includes AutoCompressor, which is paired with Llama elsewhere. How should its plotted scores be reconciled with Table 1?

Q2 How are abstentions incorporated into Table 1’s official task scores, and what is each method’s realized coverage? Are improvements over full-context inference preserved when coverage is matched or abstentions are scored consistently across all queries?

Q3 What are the offline costs of counterfactual trajectory generation, controller training, and policy calibration? Which components must be rebuilt when changing the backbone, compressor, or task distribution, and over approximately how many queries do the reported online savings amortize these costs?

Reviewer_a4HaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper asks when it is worth restoring original text after context compression. RCPE trains two reliability heads: one estimates whether the current answer has kept full-context quality, the other estimates its absolute quality. A third head is trained on the actual score change from adding one chunk back and regenerating. The whole answer / restore / abstain policy is then calibrated with Learn-then-Test to control risk and coverage. On LongBench and ZeroSCROLLS, RCPE adds about 2–3 AVG points over four base compressors, restores more useful chunks than BM25 ordering, and is about 20% faster than full-context inference.

Critical strengths

The point that relevance is not the same as restoration value (Fig. 1) is a useful framing. Training on the real score change instead of retrieval scores makes a visible difference: harmful restorations fall from 18% to 8% compared with BM25 (Fig. 3). I also appreciate that calibration covers the full multi-step policy and that every generation round counts toward cost, which is more honest than most adaptive-retrieval evaluations.

Critical weaknesses
  • I think algorithm itself is too complex which need careful ablation to simplify it. LongLLMLingua is a single-pass compressor with no restoration loop, and it beats every RCPE variant on AVG (45.6 vs. 44.3 on Qwen3-4B, 40.1 vs. 38.8 on Llama-2-7B). RCPE, by contrast, needs a separate encoder, three heads, per-chunk counterfactual rollouts, and retraining for every backbone, compressor, and compression ratio. BM25 is the only restoration baseline, so it is hard to tell whether this complexity is needed at all, or whether a strong reranker plus a confidence gate would get most of the benefit.

  • Not compatible with prefix/KV caching. Restored chunks go back into their original positions (App. I). That invalidates the KV cache for everything after the insertion point and shifts the positions of the tokens that follow, so each round re-prefills most of the context, and different queries end up with different prefixes. The main practical appeal of query-agnostic compression is that it can be cached and reused, and this design loses that after the first restoration. Latency is also measured at batch size 1 with no caching (App. H.2). In a real serving stack, where many queries share one document's KV cache, the reported 20% speedup over full context may not hold.

Questions for authors

Could you report (a) RCPE on top of LongLLMLingua, and (b) latency and throughput with prefix caching enabled (e.g., vLLM or SGLang, several queries per document)? For (b), please include a variant that appends restored chunks before the query instead of inserting them in place.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.