- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes ReBand, a training-free method for compressing context in long-horizon LLM agents. It assigns different levels of compression to historical information, preserving important details while shortening less relevant content. The results on three benchmarks show that ReBand outperforms the baseline while reducing context usage.
Critical strengths
The framework adopts an intuitive and clear strategy of allocating different levels of information retention according to estimated relevance. This provides a transparent design principle for balancing memory compression against information preservation. Additionally, it does not require additional model training which makes it a practical approach for improving memory management in long-horizon LLM agents without incurring the cost of training a separate model or fine-tuning the underlying LLM.
Critical weaknesses
ReBand relies on semantic similarity to the current query as a proxy for the future utility of historical information. However, information with low current relevance could be essential in later tasks or changing objectives. The current analysis does not fully establish how reliably this signal predicts future information reuse across different settings. A more quantitative analysis of predictive power, false-negative rates for task-critical chunks, and robustness to alternative embedding models or thresholds would strengthen the justification.
The novelty of the framework’s core components is not entirely clear. In particular, relevance estimation, adaptive compression based on relevance, and the combination of different compression strategies appear conceptually related to existing memory management and summarization approaches. The paper would benefit from a clearer discussion of how ReBand fundamentally differs from prior methods and what conceptual advance its relevance-guided routing introduces beyond combining existing techniques.
Questions for authors
As mentioned in the weaknesses, the current analysis does not fully establish how reliably semantic relevance predicts the future utility of information. Could you provide further evidence to support this assumption, particularly regarding how often information classified as low-relevance is later needed for a task?
Does your assumption that current relevance reflects future utility also hold beyond the task-oriented settings evaluated in the paper? In long-horizon conversational settings, information that is not immediately relevant may become important as the user's goals or context evolve. Evaluating ReBand on datasets such as MSC or REALTALK could help clarify whether the approach generalizes to these settings.
5 comments
3. Robustness to thresholds and embedding models (W1)
Thank you for this important weakness on robustness.
W1: A more quantitative analysis of predictive power, false-negative rates for task-critical chunks, and robustness to alternative embedding models or thresholds would strengthen the justification.
Thresholds. We compare three threshold pairs under the setup of the main table (Table 1): same 50-task subset of the test set, same LLM (GPT-4.1), same embedding model. Cutoffs are similarity quantiles, with the corresponding values shown in parentheses.
| Low-band cutoff | High-band cutoff | ACC (%) |
|---|---|---|
| Q20 (0.54) | Q50 (0.67) | 60.0 |
| Q30 (0.58) | Q60 (0.71) | 58.0 |
| Q40 (0.62) | Q70 (0.75) | 58.0 |
ReBand is fairly insensitive to the thresholds: the two alternative settings both reach 58.0%, within 2 points of our default Q20/Q50. We will add this experiment to the appendix.
Embedding models. We are running the same pipeline with two alternative embedding models. We will report the results in a follow-up comment if they are completed within the rebuttal period, and otherwise in the revised paper.
4. Applicability to long-horizon conversations (Q2)
Thank you for raising the question about extending this to other benchmarks.
Q2: Does your assumption that current relevance reflects future utility also hold beyond the task-oriented settings evaluated in the paper? In long-horizon conversational settings, information that is not immediately relevant may become important as the user's goals or context evolve. Evaluating ReBand on datasets such as MSC or REALTALK could help clarify whether the approach generalizes to these settings.
We completely agree with this extended suggestion. Following your suggestion, we compared ReBand (run on 5 random seeds) with ACON-UTCO on REALTALK Memory Probing across three models, using the official lexical F1:
| Model | ACON-UTCO | ReBand | ΔF1 (pp) |
|---|---|---|---|
| GPT-4.1 | 15.06 | 16.58 ± 0.54 | +1.52 |
| GPT-5.4 mini | 16.36 | 16.77 ± 0.45 | +0.42 |
| Qwen3.6-27B | 13.31 | 14.29 ± 0.49 | +0.97 |
ReBand scores higher on all three models, with comparable mean peak QA prompt length (within 2%) and compression calls (46–50 per method). We will add the full results to the appendix.
We sincerely appreciate your careful assessment and constructive feedback. We have provided additional experimental evidence and clarifications addressing the concerns raised in your review, including analyses of future-value reuse, critical-information retention, robustness, and generalization.
We would be grateful to hear whether these results have addressed your concerns and would be happy to clarify any remaining questions.
We deeply thank the reviewer for the careful assessment and for recognizing ReBand's intuitive design, training-free nature, and practical value. We address the concerns below with new experiments and quantitative analysis.
1. Predictive power of similarity and later use of low-relevance information (W1; Q1)
a. We analyzed real compression points from ReBand runs on AppWorld. Keeping each point's actual band assignment, we check whether at least one concrete value from a pre-compression history chunk reappears in the post-compression executed actions (FVR as defined in Appendix F). We have further expanded Section 4.5 of the revised manuscript with additional quantitative analysis (Table 5 and Figure 4).
FVR rises with the band, so similarity is positively associated with later reuse. Only 14.7% of chunks actually assigned to the low band show value reuse.
b. We do not deny that low-similarity information can occasionally be critical, so the low-band policy keeps the key fields. To verify retention, we measure the critical-information retention rate on real AppWorld trajectories (per fact, counting only facts preserved solely via the summary, requiring complete and correct retention):
ReBand retains the most critical information on both models. We will add this to the appendix.
2. Core contribution and differences from prior methods (W2)
Thank you for raising this. We agree that the individual ingredients (relevance estimation, summarization, multiple compression levels) are familiar. Our contribution is the allocation principle that combines them, together with evidence that it matters.
What is different. Uniform compressors such as ACON and LLMLingua apply a single rule to the whole history. Retrieval-style methods such as PACE store each chunk at several levels of detail in advance and use similarity to select a version for the next context. ReBand instead uses relevance to the current state to decide, at compression time, how much detail each chunk keeps within a single rolling summary: low-relevance chunks keep key fields, mid-relevance chunks become concise factual summaries, and high-relevance chunks are kept as structured key-value pairs. The shift is from a uniform rule (or keep/drop choice) to graded, relevance-routed retention granularity, without training.
Why routing matters. Offline, chunks closer to the agent's current state were more likely to have their values reused later (FVR rises from the low to the high band; see our response to W1/Q1). To test whether the gain comes from routing itself rather than from having multiple compression levels, Shuffle-Band randomly assigns chunks to bands at each compression event, keeping the band sizes and per-band policies unchanged.
ReBand is 5.4 points higher (Table 4), suggesting that matching retention detail to relevance contributes beyond the use of multiple compression levels alone.