- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
SAGE uses prefill attention from a frozen selector model to construct query-dependent relevance heatmaps, subtracts attention from a contrast query, and selects coherent spans under a token budget before generation. The paper evaluates several selector sizes, budgets, document types, and retrieval or prompt-compression baselines.
Critical strengths
The proposed pipeline is simple to inspect and does not require training a compressor. The budget sweep, raw-versus-differential attention comparison, selector-size study, and table-row experiment provide useful empirical probes. Reducing context while retaining performance close to full-context generation is an important practical goal.
Critical weaknesses
The main comparisons do not provide sufficiently controlled evidence for the headline claims. The selector, generator, and evaluator are different models, including a proprietary Gemini generator and GPT-4o-mini as the correctness judge. The paper does not report confidence intervals or repeated-run variability for the main tables, so it is difficult to know whether the small QASPER gain over CPC (0.4 points) is meaningful. The FINCH comparison is explicitly made with a different generator and different budget definitions, yet it is still used in the surrounding argument about superiority.
The contrast-query setting also changes the available information: the farthest-query variant requires another question from the same document, which is not generally available at inference time. The fixed contrast prompt is not justified by a sensitivity study. Finally, the “top-4” leaderboard statement is not independently documented in the paper, and the evaluation relies heavily on LLM judging rather than task-native metrics or human verification. These are validity problems in the central evaluation, not merely missing implementation details.
The related-work coverage is incomplete. The paper should position SAGE against the following directly relevant works:
- COMI: Coarse-to-Fine Context Compression via Marginal Information Gain — ICLR 2026
- GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignment — ACL 2026
- Adapting Language Models to Compress Contexts — EMNLP 2023
- Selective Context: Efficient Contextualization for Large Language Models — EMNLP 2023
- Long-Context LLMs Meet RAG: Overcoming Challenges for Long Context Understanding — ACL 2024
This would clarify how differential-attention span selection differs from latent compression, sentence-level compression, and long-context retrieval-compression approaches.
Questions for authors
Please provide task-native evaluation and statistical uncertainty for every main result, and repeat the comparisons with one open generator and exactly matched retained tokens. How sensitive are the results to the wording of the fixed contrast query and to the availability of a same-document contrast question? Please document the leaderboard protocol and the exact source of the claimed ranking.