acceptodds
Under review as a conference paper at ICLR 2027

Remember What Matters: Relevance-Banded Compression for Long-Horizon LLM Agents

Abstract

While long-horizon large language model (LLM) agents show immense promise in complex environments, long-horizon execution inevitably leads to a continuously growing interaction history, often triggering context explosion and performance degradation. To mitigate this issue, existing context compression methods typically compress historical context using uniform, blanket strategies. However, such indiscriminate operations fail to recognize the dynamic value of intermediate steps, risking over-compression of fine-grained, task-critical execution details. To address this limitation, we propose Relevance-Banded (ReBand) Compression, a training-free framework that uses semantic relevance between historical chunks and the agent's current runtime state to guide differentiated compression. Specifically, ReBand assigns unsummarized chunks to low-, medium-, and high-relevance bands, applying aggressive compression, concise factual summarization, and structured key-value retention, respectively. A single compressor call applies this guidance to update the rolling summary, while the latest interaction remains verbatim. Across three long-horizon task benchmarks, ReBand consistently outperforms ACON-UTCO, improving primary task metrics by 3.16–4.8 points while reducing peak context by up to 42.4%. These results demonstrate that runtime relevance provides a simple, transferable signal for allocating compression fidelity across context chunks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
LLM Agents, Context Compression, Long-Horizon Agents, Context Management

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Rr87OpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ReBand, a training-free method for compressing context in long-horizon LLM agents. It assigns different levels of compression to historical information, preserving important details while shortening less relevant content. The results on three benchmarks show that ReBand outperforms the baseline while reducing context usage.

Critical strengths

The framework adopts an intuitive and clear strategy of allocating different levels of information retention according to estimated relevance. This provides a transparent design principle for balancing memory compression against information preservation. Additionally, it does not require additional model training which makes it a practical approach for improving memory management in long-horizon LLM agents without incurring the cost of training a separate model or fine-tuning the underlying LLM.

Critical weaknesses

ReBand relies on semantic similarity to the current query as a proxy for the future utility of historical information. However, information with low current relevance could be essential in later tasks or changing objectives. The current analysis does not fully establish how reliably this signal predicts future information reuse across different settings. A more quantitative analysis of predictive power, false-negative rates for task-critical chunks, and robustness to alternative embedding models or thresholds would strengthen the justification.

The novelty of the framework’s core components is not entirely clear. In particular, relevance estimation, adaptive compression based on relevance, and the combination of different compression strategies appear conceptually related to existing memory management and summarization approaches. The paper would benefit from a clearer discussion of how ReBand fundamentally differs from prior methods and what conceptual advance its relevance-guided routing introduces beyond combining existing techniques.

Questions for authors

As mentioned in the weaknesses, the current analysis does not fully establish how reliably semantic relevance predicts the future utility of information. Could you provide further evidence to support this assumption, particularly regarding how often information classified as low-relevance is later needed for a task?

Does your assumption that current relevance reflects future utility also hold beyond the task-oriented settings evaluated in the paper? In long-horizon conversational settings, information that is not immediately relevant may become important as the user's goals or context evolve. Evaluating ReBand on datasets such as MSC or REALTALK could help clarify whether the approach generalizes to these settings.

5 comments

We deeply thank the reviewer for the careful assessment and for recognizing ReBand's intuitive design, training-free nature, and practical value. We address the concerns below with new experiments and quantitative analysis.

1. Predictive power of similarity and later use of low-relevance information (W1; Q1)

W1:ReBand relies on semantic similarity to the current query as a proxy for the future utility of historical information. However, information with low current relevance could be essential in later tasks or changing objectives.

Q1:Could you provide further evidence to support this assumption, particularly regarding how often information classified as low-relevance is later needed for a task?

a. We analyzed real compression points from ReBand runs on AppWorld. Keeping each point's actual band assignment, we check whether at least one concrete value from a pre-compression history chunk reappears in the post-compression executed actions (FVR as defined in Appendix F). We have further expanded Section 4.5 of the revised manuscript with additional quantitative analysis (Table 5 and Figure 4).

Low-band FVRMid-band FVRHigh-band FVR
14.7%24.8%42.5%

FVR rises with the band, so similarity is positively associated with later reuse. Only 14.7% of chunks actually assigned to the low band show value reuse.

b. We do not deny that low-similarity information can occasionally be critical, so the low-band policy keeps the key fields. To verify retention, we measure the critical-information retention rate on real AppWorld trajectories (per fact, counting only facts preserved solely via the summary, requiring complete and correct retention):

ModelBase promptingACON-UTCOReBand
Qwen3.6-27B71.6%72.4%76.3%
GPT-4.141.6%52.4%69.8%

ReBand retains the most critical information on both models. We will add this to the appendix.

2. Core contribution and differences from prior methods (W2)

W2: The novelty of the framework’s core components is not entirely clear. In particular, relevance estimation, adaptive compression based on relevance, and the combination of different compression strategies appear conceptually related to existing memory management and summarization approaches. The paper would benefit from a clearer discussion of how ReBand fundamentally differs from prior methods and what conceptual advance its relevance-guided routing introduces beyond combining existing techniques.

Thank you for raising this. We agree that the individual ingredients (relevance estimation, summarization, multiple compression levels) are familiar. Our contribution is the allocation principle that combines them, together with evidence that it matters.

What is different. Uniform compressors such as ACON and LLMLingua apply a single rule to the whole history. Retrieval-style methods such as PACE store each chunk at several levels of detail in advance and use similarity to select a version for the next context. ReBand instead uses relevance to the current state to decide, at compression time, how much detail each chunk keeps within a single rolling summary: low-relevance chunks keep key fields, mid-relevance chunks become concise factual summaries, and high-relevance chunks are kept as structured key-value pairs. The shift is from a uniform rule (or keep/drop choice) to graded, relevance-routed retention granularity, without training.

Why routing matters. Offline, chunks closer to the agent's current state were more likely to have their values reused later (FVR rises from the low to the high band; see our response to W1/Q1). To test whether the gain comes from routing itself rather than from having multiple compression levels, Shuffle-Band randomly assigns chunks to bands at each compression event, keeping the band sizes and per-band policies unchanged.

MethodAppWorld ACC (%)
Shuffle-Band55.3
ReBand60.7

ReBand is 5.4 points higher (Table 4), suggesting that matching retention detail to relevance contributes beyond the use of multiple compression levels alone.

3. Robustness to thresholds and embedding models (W1)

Thank you for this important weakness on robustness.

W1: A more quantitative analysis of predictive power, false-negative rates for task-critical chunks, and robustness to alternative embedding models or thresholds would strengthen the justification.

Thresholds. We compare three threshold pairs under the setup of the main table (Table 1): same 50-task subset of the test set, same LLM (GPT-4.1), same embedding model. Cutoffs are similarity quantiles, with the corresponding values shown in parentheses.

Low-band cutoffHigh-band cutoffACC (%)
Q20 (0.54)Q50 (0.67)60.0
Q30 (0.58)Q60 (0.71)58.0
Q40 (0.62)Q70 (0.75)58.0

ReBand is fairly insensitive to the thresholds: the two alternative settings both reach 58.0%, within 2 points of our default Q20/Q50. We will add this experiment to the appendix.

Embedding models. We are running the same pipeline with two alternative embedding models. We will report the results in a follow-up comment if they are completed within the rebuttal period, and otherwise in the revised paper.

4. Applicability to long-horizon conversations (Q2)

Thank you for raising the question about extending this to other benchmarks.

Q2: Does your assumption that current relevance reflects future utility also hold beyond the task-oriented settings evaluated in the paper? In long-horizon conversational settings, information that is not immediately relevant may become important as the user's goals or context evolve. Evaluating ReBand on datasets such as MSC or REALTALK could help clarify whether the approach generalizes to these settings.

We completely agree with this extended suggestion. Following your suggestion, we compared ReBand (run on 5 random seeds) with ACON-UTCO on REALTALK Memory Probing across three models, using the official lexical F1:

ModelACON-UTCOReBandΔF1 (pp)
GPT-4.115.0616.58 ± 0.54+1.52
GPT-5.4 mini16.3616.77 ± 0.45+0.42
Qwen3.6-27B13.3114.29 ± 0.49+0.97

ReBand scores higher on all three models, with comparable mean peak QA prompt length (within 2%) and compression calls (46–50 per method). We will add the full results to the appendix.

We sincerely appreciate your careful assessment and constructive feedback. We have provided additional experimental evidence and clarifications addressing the concerns raised in your review, including analyses of future-value reuse, critical-information retention, robustness, and generalization.

We would be grateful to hear whether these results have addressed your concerns and would be happy to clarify any remaining questions.

Reviewer_U96jOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents ReBand to compress context for long-horizon LLM agents. Rather than applying a single uniform compression rule across the entire interaction history, ReBand uses an adaptive, fine-grained method. It evaluates the semantic similarity between historical interaction chunks and the agent's current state, assigning tailored compression policies based on relevance: low-relevance chunks are aggressively compressed, medium-relevance chunks are reduced to concise factual summaries, and high-relevance chunks are preserved as structured key-value pairs. Benchmark evaluations on AppWorld, OfficeBench, and Multi-objective QA show the effectiveness of ReBand relative to alternative compression methods, e.g. ACON and LLMLingua.

Critical strengths
  1. Adaptive fine-grained compression strategy. I particularly appreciate the idea of adaptively applying different compression strategies to historical interactions based on their estimated importance, rather than relying on a uniform compression policy. This provides an intuitive and flexible perspective on context management for long-horizon agents. The proposed semantic-similarity-based allocation is also simple and training-free.

  2. Clear presentation and comprehensive experiments. The paper is well written, with intuitive illustrations and clear explanations of the proposed method. The evaluation covers three relevant benchmarks and includes a rich set of baselines, particularly the strong ACON baseline. The authors also provide useful ablation studies and analyses to investigate the effectiveness of relevance-guided compression, making the empirical results reasonably convincing.

Critical weaknesses

My main concern is the validity and robustness of semantic similarity as a proxy for future information utility, which serves as the central motivation for the proposed method.

  1. Limited analysis of the relationship between semantic similarity and future utility. The authors introduce Future-Value Reuse (FVR) and demonstrate a positive trend across relevance bands. However, the strength of this relationship is not sufficiently quantified. A more systematic analysis of how well semantic similarity predicts future information reuse would help substantiate the key assumption underlying ReBand. Additionally, since FVR measures the recurrence of concrete values rather than their actual contribution to downstream task success, its relationship to task-critical information retention could be better clarified.

  2. Evaluation could be further strengthened. AppWorld and OfficeBench are relevant benchmarks, but including more recent and challenging agentic benchmarks, such as τ²-Bench or AutomationBench, could further demonstrate the effectiveness of ReBand in complex, long-horizon workflows. Additionally, the main experiments rely on GPT-4.1, with only limited cross-model evaluation. Experiments with newer models would help strengthen the generalizability claims.

Questions for authors
  1. Does high semantic similarity necessarily indicate future utility? For example, a highly similar chunk may be redundant with the existing summary, while a dissimilar chunk may contain critical information needed later. I am humble to learn this from the authors.

  2. Could the authors consider evaluating ReBand on newer models (e.g., GPT-5.4-mini) and more challenging benchmarks (e.g., τ²-Bench or AutomationBench)?

3 comments

We sincerely thanks for your insightful and constructive feedback. Below, we address your questions and concerns point-by-point.

1.Relationship between semantic similarity and future utility (W1; Q1)

W1: Limited analysis of the relationship between semantic similarity and future utility.

Q1: Does high semantic similarity necessarily indicate future utility? For example, a highly similar chunk may be redundant with the existing summary, while a dissimilar chunk may contain critical information needed later.

We sincerely appreciate this insightful comment, which motivates us to further clarify the relationship between semantic similarity and future utility. We clarify that similarity in ReBand is a signal for routing each history chunk to a compression policy, not a perfect predictor of future utility.

1. Strength of the relationship. In an offline analysis of successful, uncompressed AppWorld trajectories, future-value reuse rises monotonically with similarity, from 4.0% in the lowest bin to 35.9% in the highest (Fig. 4). Within queries, the mean AUROC is 0.690. At real compression points in ReBand runs, FVR under the three bands actually used is 14.7% / 24.8% / 42.5% (low / mid / high). This second measurement is taken after compression and reflects reuse under the deployed policy, while the offline analysis isolates similarity itself. We regard similarity as an effective but imperfect ranking signal, hence three graded bands rather than a binary keep/drop rule.

2. From reuse to contribution. FVR measures reuse, not contribution to the task. We therefore add a direct measure on real AppWorld trajectories: the retention rate of critical facts, counting only facts preserved solely through the summary, each required to be complete and correct.

ModelBase promptingACON-UTCOReBand
Qwen3.6-27B71.6%72.4%76.3%
GPT-4.141.6%52.4%69.8%

ReBand retains the most critical information on both models, by a particularly large margin on GPT-4.1. We will add this to the appendix.

3. Failure cases in Q1. A high-similarity chunk that is redundant with the existing summary costs extra tokens, not accuracy. A low-similarity chunk can still prove critical: 14.7% of low-band chunks show value reuse in later actions, which is why the low band keeps key fields instead of dropping the chunk. We do not deny the remaining risk of omission.

2. Evaluation on different benchmark. (W2; Q2)

W2: Evaluation could be further strengthened.

Q2: Could the authors consider evaluating ReBand on newer models (e.g. GPT-5.4-mini) and more challenging benchmarks(e.g. τ²-Bench or AutomationBench)?

We sincerely thank the reviewer for the valuable suggestions, especially on further validating ReBand's generalization with more challenging benchmarks and newer models. We have accordingly added the following experiments.

1. New benchmark and model. Due to the limited time of the rebuttal period, we have so far only completed the airline domain of τ²-Bench (50 tasks), evaluated with GPT-5.4-mini.

MethodACC (%) ↑Steps ↓Peak (×10³) ↓Dep. (×10⁶) ↓
Base Prompting34.0011.605.3152.193
ACON-UTCO30.0014.725.2772.360
ReBand36.0013.045.2212.027

ReBand achieves the highest ACC, 2.0 points above Base Prompting and 6.0 points above ACON-UTCO. Interaction histories in this domain are short, and the Peak is almost identical across the three methods (about 5.2–5.3k), so we do not claim an efficiency gain from this result. We regard it instead as preliminary evidence that ReBand does not degrade performance under short histories. We will try to add other domains such as retail and telecom in later versions.

2. Existing coverage. Besides GPT-4.1, the paper already reports AppWorld results with Qwen3.6-27B (Table 6), where ReBand is 4.2 points above ACON-UTCO. In addition, on REALTALK Memory Probing, ReBand achieves a higher F1 than ACON-UTCO on all three models, GPT-4.1, GPT-5.4 mini and Qwen3.6-27B (+0.42 to +1.52 pp; see Section 4 of our response to Reviewer Rr87). We will add these results to the appendix.

Many thanks to the authors for the effective rebuttal. The authors largely address my concerns, and I maintain my already positive score.

I also suggest the authors to put the analysis regarding the relationship between semantic similarity and future utility in earlier sections, along with the observed failure mode. Good luck!

Thank you very much for your constructive feedback and for acknowledging that our responses have addressed your concerns. We will also adjust the placement of the newly added section in accordance with your suggestion. Your feedback has been valuable in improving the clarity and presentation of our work.

Reviewer_PtM8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ReBand, a hierarchical context compression method for long-horizon LLM agents. Based on the semantic relevance between historical information and the current state, the method categorizes chunks into three levels: Low, Medium, and High, and applies different prompts for compression. Experiments on AppWorld, OfficeBench, and Multi-objective QA demonstrate that ReBand improves task performance across multiple benchmarks while reducing peak context consumption.

Critical strengths

1. The paper addresses a practically relevant problem: context accumulation and the loss of critical information in long-horizon agents. It proposes a simple and easy-to-implement hierarchical compression strategy that requires no additional model training, offering good engineering practicality. 2. The experiments cover three benchmarks of different types and include comparisons with multiple existing methods. The paper also provides ablation studies, cross-model evaluations, and case analyses, resulting in relatively comprehensive experimental coverage and reasonably convincing results.

Critical weaknesses

1. ReBand primarily combines embedding similarity-based grouping with differentiated prompt-based compression. Overall, it appears to be a combination of existing techniques, with limited algorithmic novelty. 2. Future-Value Reuse treats the subsequent reappearance of specific values from historical chunks as evidence of information reuse. However, textual repetition does not necessarily indicate actual information value, nor does it rule out the possibility that the agent obtained the same information from other sources. Therefore, this metric has limitations as a proxy for actual information utility. 3. The paper reports results on AppWorld, OfficeBench, and other tasks, but the main experimental tables provide only single-run results, without reporting experiments across multiple random seeds. In particular, it remains unclear whether the reported improvements of 3–4 percentage points are consistent and statistically reliable.

Questions for authors

1. What is the core technical novelty of ReBand compared with existing context compression methods? Could additional comparisons or ablation studies be conducted to verify the necessity of its design choices? 2. Could the authors remove historical chunks with high Future-Value Reuse scores and compare task success rates before and after removal to verify the actual utility of this information? 3. Could the authors report results across multiple random seeds, together with confidence intervals, for the main experiments and the ablation studies in Table 4, and clarify whether the observed performance differences are statistically significant?

3 comments

We thank the reviewer for their thorough review and specific suggestions. We will address each of the three weaknesses and issues in detail.

1. Core novelty (W1; Q1)

W1: ReBand primarily combines embedding similarity-based grouping with differentiated prompt-based compression. Overall, it appears to be a combination of existing techniques, with limited algorithmic novelty.

Q1: What is the core technical novelty of ReBand compared with existing context compression methods? Could additional comparisons or ablation studies be conducted to verify the necessity of its design choices?

We agree that relevance estimation, summarization, and multi-level compression are each familiar. Our contribution is the allocation principle that combines them: each history chunk's retention granularity is set by its relevance to the current state, within a single rolling summary.

Distinction from prior methods. Existing approaches differ in how they manage historical information. ACON and LLMLingua generally apply uniform compression policies across the history, while retrieval-based methods such as PACE pre-store chunks at multiple levels of detail and select the appropriate representation based on similarity. In contrast, ReBand determines the retention granularity of each history chunk dynamically at compression time, according to its relevance to the agent's current state. Specifically, low-relevance chunks retain essential fields, mid-relevance chunks receive concise factual summaries, and high-relevance chunks preserve structured key-value information. This enables differentiated information preservation within a single compression process, without additional training.

Evidence for the necessity of relevance-guided routing. To isolate the contribution of routing, we introduce a Shuffle-Band ablation that randomly assigns chunks to compression bands while keeping the same band sizes and compression policies. On AppWorld, Shuffle-Band achieves 55.3% accuracy, compared with 60.7% for ReBand (+5.4 percentage points) (Table 4). This demonstrates that the improvement is not merely due to combining multiple compression levels, but is associated with assigning retention granularity according to semantic relevance.

2. Whether Future-Value Reuse Reflects Utility (W2; Q2)

W2: Future-Value Reuse treats the subsequent reappearance of specific values from historical chunks as evidence of information reuse. However, textual repetition does not necessarily indicate actual information value, nor does it rule out the possibility that the agent obtained the same information from other sources. Therefore, this metric has limitations as a proxy for actual information utility.

Q2: Could the authors remove historical chunks with high Future-Value Reuse scores and compare task success rates before and after removal to verify the actual utility of this information?

We agree that textual reuse is only an indirect signal of utility. In this paper, Future-Value Reuse (FVR) serves a motivational role: it shows that history chunks closer to the current state are more likely to be reused by later actions (Table 5, Figure 4; further analysis has been added in the new Section 4.5), which motivates our choice of similarity as the routing signal. FVR itself is not a measure of utility, nor is it used as a predictive indicator.

Direct evidence of utility comes from two sources.

First, the retention rate of critical facts. We count only facts that the compressed summary alone must preserve, and require each to be retained completely and correctly:

ModelBase PromptingACON-UTCOReBand
Qwen3.6-27B71.6%72.4%76.3%
GPT-4.141.6%52.4%69.8%

Second, a removal experiment. Within the high-similarity band, we remove the chunks that are reused by later actions and observe the change in task success.

SettingAppWorld Accuracy
Original ReBand60.7%
Removing reused chunks from the high band54.2%

Removing the reused chunks from the high band lowers accuracy by 6.5 percentage points, indicating that the retained high-band chunks contain information that contributes to task success. We will add this study in appendix.

3. Statistical reliability (W3; Q3)

W3: The paper reports results on AppWorld, OfficeBench, and other tasks, but the main experimental tables provide only single-run results, without reporting experiments across multiple random seeds. In particular, it remains unclear whether the reported improvements of 3–4 percentage points are consistent and statistically reliable.

Q3: Could the authors report results across multiple random seeds, together with confidence intervals, for the main experiments and the ablation studies in Table 4, and clarify whether the observed performance differences are statistically significant?

We sincerely appreciate this valuable suggestion. We agree that single-run differences should be interpreted with care. To verify the stability of ReBand, we repeated the AppWorld test-normal evaluation (168 tasks) with five random seeds, keeping all other settings aligned with ACON. ReBand achieves 60.58 ± 1.65% accuracy. The 56.5% reported for ACON-UTCO in its original paper is a single-run result from its original setting.

Thank you again for your constructive feedback. In response to your concerns, we have provided additional evidence on the contribution of relevance-guided routing, the utility of retained information through a removal experiment, and the stability of ReBand across multiple random seeds.

We would greatly appreciate your thoughts on whether these results have addressed your concerns. We remain happy to discuss any outstanding issues.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.