acceptodds
Under review as a conference paper at ICLR 2027

Keep the Chain Alive: Bridge-Survival Compression for Multi-Hop Question Answering

Abstract

Multi-hop question answering reads long retrieved contexts, so compression governs both the cost and the correctness of that reading. Current compressors, however, drop the bridge entities that connect one hop to the next, making the answer entity in the context unreachable. We propose a dual-paradigm theory of compression that organizes the scattered field into one design space. The quality paradigm attributes every compressor's losses to selection, generation, and arrangement, and identifies bridge survival as the fatal channel for multi-hop questions, one that obeys a product law and collapses exponentially with depth. The overhead paradigm prices compressor, reader, auxiliary, and amortized training, and yields a deployment criterion under which single-call compression is a necessity. We instantiate the theory as ChainPack, a single model trained to keep bridges alive through survival-filtered self-training. Across three full-scale multi-hop benchmarks, one call cuts 90% of the tokens and improves accuracy over the uncompressed context (+3.8 points, full-scale paired), stays within 4 points of the oracle bound on 3-4 hop chains, and on a stronger base model even surpasses this selection ceiling. The theory locates the common failure of existing compressors in bridge survival rather than selection, and the remaining headroom in readers rather than in better compression.

Primary area:
foundation or frontier models, including LLMs
Keywords:
context compression, multi-hop question answering, retrieval-augmented generation, bridge survival, self-training, large language models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_kXwMOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies context compression for multi-hop question answering. It argues that existing compressors often remove intermediate bridge entities even when the final answer evidence remains in the compressed context. The paper introduces a “dual-paradigm theory” consisting of a quality paradigm, which decomposes compression into selection, generation, and arrangement, and an overhead paradigm, which accounts for compressor, reader, auxiliary, and training costs. Based on this framework, the authors propose ChainPack, a two-stage fine-tuning procedure that first trains a model to produce hop-aligned summaries and then performs survival-filtered self-training by retaining generated summaries that contain all annotated bridge entities.

Critical strengths
  1. The paper addresses a practically relevant failure mode of context compression for multi-hop QA.
  2. The paper reports token consumptions of the methods.
Critical weaknesses
  1. The paper is unnecessarily difficult to follow. The introduction makes broad claims about extractive, abstractive, and iterative methods before providing definitions or experimental support. It introduces the “dual-paradigm theory,” “product law,” and several failure mechanisms in a dense paragraph, while their precise meanings only become clear in Section 3. The claims for weakness of the existing methods also lack for sufficient supports.

  2. The theoretical contribution is weak. Most results follow directly from strong assumptions or definitions. Indeed, the theories seem very weird together with their proofs. I doubt the validity and the necessity of such theories.

  3. The technical novelty is limited. ChainPack is essentially chain-formatted LoRA fine-tuning followed by rejection-sampling self-training: generate several candidates, retain those containing all gold bridge strings and passing an embedding-similarity threshold, and fine-tune again. This is a reasonable recipe, but it introduces neither a new architecture nor a new optimization objective.

  4. The expermental evaluation is rather weak. The main comparison is largely restricted to the LLMLingua family. Direct comparisons with relevant trained, abstractive, or context-aware compressors are missing. Finally, all three evaluation datasets are Wikipedia-style multi-hop QA benchmarks, two of which also supply training data. There is little evidence that the approach generalizes to other RAG settings, tasks without bridge annotations, or even single-hop and out-of-domain QA.

Questions for authors

See the weaknesses.

Reviewer_ymQSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies context compression for multi-hop question answering, focusing on the loss of bridge entities that link successive reasoning steps. It proposes a quality paradigm that decomposes compression into selection, generation, and arrangement, and an overhead paradigm that prices deployment costs. The proposed method, ChainPack, produces a chain-structured summary, with one line per hop, in a single compression call. Training first applies supervised fine-tuning on annotated evidence chains, then performs survival-filtered self-training that retains generated summaries preserving all gold bridge entities and sufficiently similar to reference summaries. Across three full-scale multi-hop QA benchmarks, the paper reports an approximately 90% reduction in reader input tokens and a pooled F1 improvement of 3.8 points over uncompressed contexts under the same reader, with gains concentrated on 2WikiMultiHopQA and MuSiQue.

Critical strengths
  1. Table 1 reports substantial F1 gains over the uncompressed context on 2WikiMultiHopQA and MuSiQue, corresponding to approximately +3.8 points when pooled across all evaluated questions. This is the strongest consideration in favor of acceptance.

  2. The paper identifies a consequential evidence-preservation problem: retaining an answer-bearing sentence does not necessarily preserve the intermediate evidence required for multi-hop reasoning.

Critical weaknesses
  1. Theorem B.2 only gives a no‑loss bound under the restrictive assumption that all signal is preserved. It cannot serve as a general upper bound for lossy abstractive compressors that rewrite content. Nevertheless, the paper uses the oracle‑selection context as a universal ceiling to argue for a saturated compression frontier and to dismiss alternative compression designs. This extrapolation is not theoretically justified.

  2. Key causal claims for bridge‑survival filtering lack complete matched‑budget ablation evidence. Critical control variants including SFT‑only, unfiltered self‑training, bridge‑only filtering and semantic‑only filtering are missing. Direct head‑to‑head comparisons against important abstractive baselines (BRIEF, RECOMP, EXIT, CompAct) under identical reader and realized token budget are absent. Additionally, Figure 3 robustness plots show several‑point score fluctuations which appear inconsistent with textual claims that performance changes are “at most 1.1 points” and “no cliff”.

Questions for authors
  1. How do you reconcile the textual robustness claims (“at most 1.1 points”, “no cliff”) with the several‑point F1 variation visible in Figure 3? Please provide exact numeric scores, evaluation sizes and full protocols for every curve in Figure 3; clarify which curves come from full‑scale benchmark runs and which are from small stratified probe subsets.

  2. Do you have matched‑budget experimental results for SFT‑only, unfiltered self‑training, bridge‑only filtering, and semantic‑only filtering ablation variants? These controls are required to isolate whether empirical gains stem specifically from bridge‑survival filtering, rather than from generic additional supervised fine‑training.

  3. In Theorem B.6 your proof defines expected gain a⋅r⋅Ga\cdot r\cdot G and expected cost c⋅rc\cdot r; factor rr cancels, giving threshold a≥c/Ga\ge c/G instead of your reported a≥c/(rG)a\ge c/(rG). Please provide corrected derivation for Theorem B.6 and evaluate if its anti‑post‑hoc‑repair conclusion still holds. What auxiliary assumptions justify that single‑call compression is necessary and oracle‑selection is a general upper bound for arbitrary lossy abstractive compression?

Reviewer_tyfwOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The authors aim to tackle the problem of context compression for multi-hop question answering, where existing compressors often drop the bridge entities that link one reasoning hop to the next. To address this, the authors present a theoretical framework that characterizes where compressors lose accuracy and what they may cost at deployment, and then propose a compressor that outputs hop-aligned evidence chains, trained with supervised fine-tuning followed by self-training on outputs filtered for bridge-entity survival. The authors perform experiments on multiple multi-hop QA benchmarks, showing that the proposed method outperforms extractive compression baselines and even the uncompressed context.

Critical strengths
  • The problem of bridge entities being dropped during compression seems well-motivated and practically important.

  • The proposed approach seems effective, outperforming the (compared) baselines and the uncompressed context with a single compression call.

Critical weaknesses
  • In the era of long-context language models, I am not convinced that context compression is necessary for multi-hop question answering, as the number of tokens used to handle the retrieved documents seems way too small compared to the context length of recent language models. Also, considering the cost of the context compression as well as the retrieval and question answering, it might be much efficient to not performing context compression (or simply using the smaller models).

  • This paper is difficult to follow. For example, it is not clear what the dual-paradigm theory is and how it is related to the proposed method for context compression. In addition, many terms (especially in the Introduction section) are not clearly defined, making the core idea difficult to understand. Furthermore, the conceptual and method figures are very complex and not easy to follow. And, it would also be great to include the motivational example in an earlier part of the paper.

  • The baselines included in the paper (which appear around 2023-2024) might be weak, as I am pretty sure that there are recent context compression approaches, for example, BRIEF-Pro, CORE-RAG, and EXIT [1-3].

  • The analysis part of this paper could be improved; for example, as the authors suggest the quality paradigm, which decomposes the compression method into selection, generation, and arrangement, deeper analysis of each part of this paradigm would be great.

[1] BRIEF-Pro: Universal Context Compression with Short-to-Long Synthesis for Fast and Accurate Multi-Hop Reasoning

[2] Less Is More: Elevating RAG via Performance-Driven Context Compression

[3] EXIT: Context-Aware Extractive Compression for Enhancing Retrieval-Augmented Generation

Questions for authors

See above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.