acceptodds
Under review as a conference paper at ICLR 2027

SABR: Structure-Aware Budgeted Retention under Fixed Token Budgets

Abstract

Selecting relevant fragments is not sufficient when a context budget separates evi- dence needed together. We introduce Structure-Aware Budgeted Retention (SABR), a training-free selector combining frozen semantic scores, local coordination, and exact generator-token packing. We compare SABR with a strong frozen reranker at identical per-question context and prompt lengths, separating selection quality from full-context quality and online cost. On a previously inspected reserve of 1,384 questions across four tasks and three generators, SABR improves SQuAD F1 at a 70% context cap by 3.18, 4.19, and 4.23 points; the latter two paired 95% intervals exclude zero. The four-task macro difference is +0.92 [−0.08, 1.91]. A separate 896-question cohort yields a positive SQuAD estimate but does not confirm an ag- gregate gain. A matched-token sentence-unit pilot increases answer-string presence on SQuAD and mean intact-support fraction on HotpotQA by 10.94 and 13.80 percentage points. Timing studies show that exact implementation optimizations reduce selection overhead, but do not establish an overall speedup over full context. These findings identify a concrete regime in which coordinated retention improves fixed-budget selection, while distinguishing retained evidence, answer quality, and computational efficiency.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, context selection, prompt compression, question answering, token budgets, structure-aware retrieval

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_4rFtOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

SABR is a training-free selector that combines frozen semantic scores with local coordination, redundancy and proximity features, and exact generator-token packing. The paper compares it with a reranker at matched realized context and prompt lengths, and reports separate analyses of answer quality, evidence retention, and runtime.

Critical strengths

The paper is careful about a real confound in context-selection experiments: retaining more tokens can make a method look better. The exact-token implementation and the separation of primary, confirmation, diagnostic, and timing cohorts are clearly described. The authors also report negative or inconclusive findings instead of presenting every diagnostic as a success.

Critical weaknesses

The central claim is not supported at the level needed for acceptance. The strongest positive result is a localized SQuAD improvement on a primary reserve that was inspected during method development. The four-task macro effect is uncertain on that reserve and slightly negative on the independent cohort; the independent confirmation also uses only one generator. Thus the paper does not establish that SABR improves context selection beyond the chosen SQuAD setting.

The method contains several hand-designed decisions—eight-word fragments, mixing coefficients, route rules, neighborhood bonuses, and the 70% operating point—that were developed using public data. The independent cohort reduces but does not remove this development dependence. The component interventions change multiple parts of the policy or use a separate matched-prefix adapter, so they do not identify which claimed structural mechanism is responsible. The runtime experiments are small and show that SABR is not faster than full context in the reported settings. Taken together, the work is a careful analysis of a promising heuristic, but its general methodological conclusion is too strong for the evidence provided.

The related-work coverage is incomplete. The paper should also discuss the following directly relevant works:

  • COMI: Coarse-to-Fine Context Compression via Marginal Information Gain — ICLR 2026
  • GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignment — ACL 2026
  • In-context Autoencoder for Context Compression in a Large Language Model — arXiv, 2023
  • Learning to Compress Prompts with Gist Tokens — NeurIPS 2023
  • Adapting Language Models to Compress Contexts — EMNLP 2023

Without these references, the relationship between SABR’s token-selection heuristic and prior learned or latent context-compression methods is not sufficiently clear.

Questions for authors

Can you evaluate one frozen configuration on a genuinely untouched, question-disjoint test set with more than one generator? How were the fragment size, coefficients, route rules, and operating point selected relative to the inspected reserve? Please report whether the same conclusions hold under paragraph- or sentence-level segmentation and under a fully matched end-to-end cost budget.

1 comment

Thank you for pointing out these issues. Your questions helped us distinguish choices made during development from evidence on held-out questions.

For your first question about untouched questions and multiple generators, we completed a frozen evaluation. We fixed one configuration before running Qwen2.5-7B and Mistral-7B on 512 SQuAD questions from 177 articles. The questions come from the public training split, with the official validation split and training articles in recovered development/evaluation records excluded. We found zero overlap in question IDs, normalized questions and paragraph hashes with those records. Checkpoint pretraining exposure is unknown. At the 70% cap all arms matched realized context and prompt tokens. SABR minus reranker answer F1 was +3.15 [0.46, 5.88] for Qwen and +1.62 [−0.70, 3.99] for Mistral. Article-cluster intervals were [0.37, 5.96] and [−0.90, 4.10]. The Qwen gain survives both analyses. The primary four-task macro was +0.92 [−0.08, 1.91] and the independent 896-question macro was −0.34 [−2.40, 1.76]. Neither supports an overall QA gain.

For your second question about design choices, we now state the development history explicitly. Eight-word fragments, routing rules and bonuses were hand-designed. The inspected reserve informed the mixing coefficients. We tested 50% and 70% and highlighted 70% after seeing positive SQuAD estimates for all three generators. In the frozen 512-question test, disabling only the four adaptive terms changed F1 little (SABR minus control +0.20/−0.09, Qwen/Mistral). A stronger control held semantic mixing and packing fixed and disabled grouping, smoothing, bridging, proximity, redundancy and the boundary prior together. On its 709-question sample, full SABR minus this control was +2.86 [0.56, 5.11] for Qwen and +1.24 [−0.75, 3.28] for Mistral. This supports a joint contribution for Qwen on these questions.

For your third question about segmentation and end-to-end cost, we tested sentence selection and matched end-to-end time separately. SABR minus reranker was +3.40/+1.19 on the coordination sample and +1.55/+0.53 on the 19-arm component sample. Each has 709 questions, sharing all 28 articles and 186 question IDs, and share no article with the 512-question test. We report them separately. On the component sample, sentence selection minus the fragment policy was +0.23 [−1.73, 2.26] and +1.02 [−0.84, 2.84]. Neither interval resolves a sentence advantage. We plan to compare paragraph and fragment selection on HotpotQA at equal token budgets. No alternative mixing weight, radius or density threshold cleared its within-family Holm correction. Appendix I includes the complete contrasts.

We plan to compare paragraph and fragment selection on HotpotQA at equal token budgets and will post the results of that comparison here.

Matched end-to-end time. We calibrated the reranker’s cap using time alone on 96 separate questions, then locked 100% for Qwen and 80% for Mistral against SABR at 70%. On all 709 component-study questions, mean reranker/SABR request-time ratios were 1.016 and 0.996, both within the prespecified 5% tolerance. Timing included online helper scoring, packing and natural answer generation. SABR minus reranker was −4.37 F1 for Qwen and −0.81 for Mistral. The two-model mean was −2.59 [−4.49, −0.81] by paired article bootstrap. Thus the token-matched advantage does not carry over to this time budget.

Thank you also for the related work you suggested. Section 2.1 discusses all five suggested works. COMI [1] directly shares the relevance–redundancy motivation. GMSA [2] merges tokens into soft representations. Compact representations are also learned by ICAE [3], Gist Tokens [4] and AutoCompressors [5]. SABR retains source text for a frozen generator. These methods were not benchmarked head-to-head here.

[1] Tang et al. COMI: Coarse-to-Fine Context Compression via Marginal Information Gain. ICLR 2026. [2] Tang et al. GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignment. ACL 2026. [3] Ge et al. In-context Autoencoder for Context Compression in a Large Language Model. ICLR 2024. [4] Mu et al. Learning to Compress Prompts with Gist Tokens. NeurIPS 2023. [5] Chevalier et al. Adapting Language Models to Compress Contexts. EMNLP 2023.

We welcome questions during the discussion. We hope these revisions and clarifications will be helpful in your later assessment of the paper.

Reviewer_eyJ5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

SABR is a training-free context selector for fixed token budgets. It mixes frozen bi-encoder and cross-encoder scores, adds deterministic coordination bonuses (shared entities, shared rare terms, positional proximity) during greedy packing, and enforces the budget with the generator's own tokenizer. Its central claim is that selection requires deciding which fragments to keep together, not merely ranking them.

Compared against a strong frozen reranker under exactly matched context and prompt token counts (1,384 questions, 4 tasks, 3 generators, 50%/70% budgets), SABR yields a repeatable SQuAD gain at the 70% cap (+3.18 / +4.19 / +4.23 F1). The four-task macro difference is +0.92 [−0.08, 1.91]; a held-out 896-question queue gives macro −0.34 and does not confirm an overall gain; no end-to-end speedup is demonstrated (1.12× slower than full context).

Critical strengths
  1. The equal-length control is exemplary. Both arms are verified to have identical actual context token counts and identical prompt token counts per question. This closes the most common loophole in compression papers ("you just kept more tokens") and is the single most reusable idea in the paper.
  2. Budget enforcement is done properly. Realized retention is 49.82–49.99% and 69.66–69.98% at the two caps, validated with the generator's tokenizer rather than a nominal ratio. The authors explicitly note that "both arms use 70%" is not sufficient for a length-matched comparison, correct and worth repeating in the literature.
  3. Low reproduction cost. Two frozen open models plus a small set of hand-tuned constants; no training, no reference answers, no supporting-fact labels.
Critical weaknesses
  1. The central concept is never causally established. SABR and the reranker arm differ in two ways simultaneously: semantic mixing (bi-encoder + cross-encoder vs. cross-encoder alone), and the presence of coordination. The observed +3–4 F1 could simply be "two retrieval models beat one." The matched-prefix diagnostic that should disentangle this changes the reserve, the scorer, and the strategy itself, and the authors correctly refuse to treat it as causal.
  2. Selective emphasis across cells. Of 12 task × generator cells at the 70% cap, only the three SQuAD cells are positive; the other nine straddle zero (Mistral HotpotQA is −2.47).
  3. Weak external validity on input length. Full-context SQuAD averages only 182 tokens and HotpotQA 2,411, with a 4,096 cap. Discussing "context budget compression" at 182 tokens has little bearing on modern 128K-window models. The genuinely long regime is probed with 38 examples only, and in that group the two arms are not length-matched.
  4. No efficiency gain. The method is 1.12× slower than full context (1.07× on long inputs). Efficiency is the primary motivation for compression, and the paper does not deliver it. The comparison against its own reranker is not a complexity claim, as the authors acknowledge.
  5. Limited practical relevance. The evaluated regime barely needs compression: full-context SQuAD averages 182 tokens and HotpotQA 2,411, far below the 128K+ windows of current models, so a 50–70% budget is not a constraint practitioners actually face. Where compression does pay — API cost or true window overflow — the paper demonstrates no benefit: it is slower than full context (W5), and its 38 longest inputs (up to 6,986 tokens) are not length-matched. Weighing a macro gain of +0.92 (interval crossing zero) against the online cost of two encoders plus iterative re-packing, I do not see a deployment setting where this is currently the right choice.
Questions for authors
  1. What remains of the SQuAD gain if the full pipeline is re-run, frozen, on an untouched test set?
  2. Why is supporting-fact F1 not reported for HotpotQA?
2 comments

Thank you for asking how much coordination contributes to SABR’s performance. To measure the contribution of coordination directly, we held the semantic scores fixed and tested SABR with and without its coordination terms.

For your first question about rerunning a frozen pipeline on untouched questions, we completed a new two-generator evaluation. In that evaluation, we froze the pipeline and evaluated both generators on 512 SQuAD questions from 177 public training articles outside our recovered evaluation history. All three arms matched context and prompt lengths. SABR minus reranker was +3.15 answer F1 [0.46, 5.88] for Qwen2.5-7B and +1.62 [−0.70, 3.99] for Mistral-7B. The Qwen gain persists under article resampling, while Mistral remains uncertain. Public training data leave checkpoint exposure unknown. The primary four-task macro (+0.92) and the independent macro (−0.34) both have intervals spanning zero. We retain all 12 primary cells, including Mistral HotpotQA at −2.47 F1.

Does coordination add anything beyond the second encoder? We held the semantic score vectors, initial lexical/entity/title signals, routing, materializer and token counts fixed, then jointly disabled seven coordination terms, including grouping, smoothing and the fixed boundary prior. On 709 questions, SABR minus that control was +2.86 [0.56, 5.11] for Qwen and +1.24 [−0.75, 3.28] for Mistral. The article-bootstrap intervals were [0.10, 5.66] and [−0.97, 3.52]. The Qwen gain exceeded the Mistral gain by 1.61 F1, with an exploratory paired article-bootstrap 95% interval of −1.29 to +4.74. Removing just adaptive bridging, proximity and redundancy gave no reproducible benefit from enabling those terms.

A second 709-question list shares the same 28 articles and 186 question IDs with the coordination list. On this list, removing semantic scoring from SABR cost 6.50 and 2.67 F1 points (within-family Holm-adjusted p = 0.0009 and 0.0387). SABR minus reranker was +1.55/+0.53 on this list, versus +3.40/+1.19 on the coordination list. Neither list shares articles with the 512-question set. These results support semantic scoring and, for Qwen on one sample, coordination as a group. Individual rule effects remain unresolved. For your second question about HotpotQA supporting-fact F1, we changed what the generators were asked to return. Our original prompts requested only answers. Retention of a gold sentence in the input cannot tell us whether the generator identified it as supporting evidence. We asked for answers and sentence IDs on 512 HotpotQA distractor-development questions. The two compressed arms matched realized token counts, with malformed outputs receiving zero. On official supporting-fact F1, SABR minus reranker was +1.42 [−1.13, 4.08] for Qwen and −0.24 [−2.27, 1.78] for Mistral. Joint F1 changed by +0.72 [−1.60, 3.03] and +0.07 [−1.60, 1.74]. The support-F1 advantage remains uncertain. Historical overlap is unverified for this HotpotQA sample (Appendix I.3). On these same questions, with frozen scores and a 70% body-word budget, sentence selection retained 98.03% of gold supporting sentences intact on average, versus 96.81% for paragraph selection (paragraph minus sentence, −1.22 points, paired 95% interval [−2.21, −0.29]). We plan to generate answers and supporting facts for both selection policies (paragraphs and fragments) at equal token counts and score both outputs using official supporting-fact and joint F1.

We also agree with your broader concern about efficiency on longer inputs. When would compression pay? The 182- and 2,411-token means describe the small sentence pilot. A separate sweep with 72 contexts and six bins up to 6,144 tokens found no SABR speedup over raw full context. On the 709-question component sample, a time-matched reranker achieved higher mean F1, with SABR minus reranker at −2.59 [−4.49, −0.81] across the two models, with mean times within 5% for each. These results establish neither an API-cost benefit nor a 128K-context deployment case. We revised the abstract and conclusion to specify the settings in which SABR helped.

We welcome questions during the discussion and will post our results here. If these revisions address your concerns, please consider revisiting your rating.

Thank you for the thorough and candid rebuttal. The coordination ablation (W1) is well done and does distinguish coordination from a pure "two retrievers beat one" effect. However, my assessment is unchanged. The gain remains generator-dependent — Mistral's intervals still around zero. More decisively, the efficiency concern is confirmed rather than resolved and no speedup was found even at 6,144 tokens. Since efficiency is the primary motivation for compression, and the evaluated regime barely needs it, I do not see a deployment setting where SABR beats a standard reranker. My score is unchanged.

Reviewer_yFLZOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

AI-generated low quality work.

Critical strengths

None

Critical weaknesses

AI-generated low quality work.

Questions for authors

None.

2 comments

Thank you for raising your concerns. Your review prompted us to clarify the role of AI in this work and our responsibility for the research decisions. The AI-use statement describes assistance with literature retrieval, manuscript drafting and revision, experimental code, and statistical checks of the saved outputs. The core research ideas, selector design, evaluation methodology, and conclusions are our own, and we take full responsibility for them.

To address your concern about the quality and rigor of the research, we have conducted additional experiments, strengthened the evaluation controls, and clarified the scope and limitations of our findings. Two results are particularly informative. On 512 SQuAD questions from 177 articles excluded from the recorded development and evaluation sets, SABR gained 3.15 answer-F1 points over a reranker for Qwen2.5-7B at identical context and prompt lengths (95% interval [0.46, 5.88]). For Mistral-7B the gain was 1.62 [−0.70, 3.99]. The primary and independent four-task macro intervals both include zero.

When the comparison instead matched mean end-to-end time, the reranker could retain more text. On 709 SQuAD questions its average F1 was 2.59 points higher across the two generators (95% article-bootstrap interval [0.81, 4.49]). Appendix I gives the question-list overlaps, the fixed-semantic coordination control, official HotpotQA support scores, and the calibration and evaluation details for this timing comparison. These additions narrowed what we claim for the method.

We are conducting further experiments to examine the robustness and limitations of the method and will share the results here. We welcome further questions during the discussion.

Again, the full paper is low-quality AI-generated; the rebuttal is 100% AI generated. I keep my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.