acceptodds
Under review as a conference paper at ICLR 2027

Progressive Hierarchical Context Compaction for LLM Agents

Abstract

Long-horizon LLM agents accumulate increasingly large interaction histories, making context management essential for both inference efficiency and task performance. An important challenge in context compaction is how to maintain and progressively update historical information at different levels of abstraction as an agent executes. We introduce a progressive hierarchical context-compaction framework that maintains an agent's working context as an ordered sequence of blocks at three abstraction levels: trajectory, subtask, and state. At each compaction checkpoint, an LLM compactor examines the current working context and chooses an abstraction level. The selected level determines how much of the recent history is rewritten, producing an updated working context. Repeated compaction progressively produces a hierarchical representation in which different portions of interaction history are maintained at different abstraction levels. The framework further supports a plug-and-play compactor that can be trained independently of the task-performing agent. Experiments on BrowseComp-Plus and NL2Repo-Bench demonstrate substantial improvements in task performance and the context processed per task-model call over existing compaction methods. Training a smaller compactor using decisions generated by a stronger teacher yields further performance improvements.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agents, Context Compaction, Hierarchical Semantic, Plug-and-Play

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_azBDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

HiContext compacts an agent's history into trajectory, subtask, and state blocks. A separate LLM chooses the level every 30K new tokens and rewrites part of the context, while earlier blocks are mostly kept. The task agent is Qwen3.5-27B and the compactor is Qwen3.5-9B. The paper evaluates on BrowseComp-Plus and NL2Repo-Bench, with fixed-level ablations and a teacher fine-tuning experiment.

Critical strengths
  1. The method is simple to deploy. It does not require retraining the task model. The prompts, budget rules, and task-model versus compactor costs are well documented, which makes the system easier to reproduce.

  2. The gains appear in both domains, on BrowseComp-Plus and NL2Repo-Bench. It also beats subtask-only compaction at the same context length on BrowseComp-Plus. This suggests that the gain is not only from using less context.

Critical weaknesses
  1. It is hard to separate hierarchy from prompt engineering. HiContext has detailed domain-specific instructions to preserve document IDs, start.md, paths, and errors. Some baselines lose exactly this information or suffer from tool errors. The fixed-level ablations do not include an equally engineered single-level summarizer or a simple level-selection control

  2. The statistical evidence is thin. There are no repeated-run confidence intervals or significance tests. The NL2Repo ablation uses a small amount of instances, and the fine-tuning result is a single in-domain comparison

Questions for authors
  1. How many independent tasks and runs support each result, and what does “Score” mean on each benchmark? Can you add per-task confidence intervals

  2. Can you compare against a single-level summarizer with the same preservation instructions and a simple random or count-based level-selection rule

  3. How many training examples survive the 0.8 outcome filter, and was that filter ablated? Please also clarify whether any question can appear in both training and evaluation after concatenation

3 comments

We thank you for recognizing HiContext’s design and its improvements on both benchmarks. We would like to address your concerns as follows:

Prompt Engineering and Hierarchical Compaction (W1)

In the original experiments, HiContext and its fixed-level variants used the domain-specific stall footer; the external baselines used their original prompts and received no footer. The fixed-level variants also shared HiContext’s preservation instructions for their assigned level, so this ablation supports the benefits of hierarchical compaction. We retained each external baseline’s original prompt to preserve the operations it was designed to support.

For the context engineering ablation, we rerun the compaction baselines with the same domain-specific footer appended to their compaction outputs and add preservation instructions to their original prompts. We add state-level instructions to ReSum for its consolidated task summary, subtask-level instructions to Context-Folding for its subtask-based folding, and trajectory-level instructions to AgentFold to preserve concrete execution details during its folding at every turn.

Due to time constraints during rebuttal, the additional NL2Repo-Bench experiments labeled "30 tasks" use 30 randomly sampled instances. We will extend these evaluations to all 104 instances in the camera-ready version.

Baseline scores before and after adding preservation instructions and footers:

MethodBrowseComp-Plus Original -> AugmentedNL2Repo-Bench (30 tasks) Original -> Augmented
ReSum28.7 -> 54.720.79 -> 23.41
Context-Folding58.7 -> 55.330.33 -> 31.68
AgentFold48.0 -> 49.38.56 -> 10.70
HiContext65.334.75

HiContext still outperforms augmented baselines by 10.0 percentage points on BrowseComp-Plus and 3.07 percentage points on NL2Repo-Bench. These results support the benefit of hierarchical compaction beyond the engineered prompt.

Single Level and Random Level Summarization (Q2)

The fixed-level ablation in Section 4.3 already provides the single-level summarization comparison you requested. Each uses the same instructions as HiContext for its assigned level and always compacts at that level.
With the rest of HiContext configuration held fixed, random level selection scores 60.7 versus 65.3 for HiContext on BrowseComp-Plus and 31.19 versus 34.75 on NL2Repo-Bench (30 tasks). This comparison shows that the level-selection policy contributes to performance.

Statistical Significance and Full NL2Repo Ablation (W2 + Q1)

BrowseComp-Plus groups 150 test questions into 30 five-question tasks. Score is the mean fraction of questions answered correctly per task. NL2Repo-Bench has 104 repository-generation tasks. Score is the mean fraction of passing test cases per task. Both scores are reported as percentages.

We supplement the evaluation with three independent runs, paired confidence intervals, and an expanded ablation on the full NL2Repo-Bench evaluation set.
Repeated-run results: We report the mean ±\pm standard deviation across three independent runs.

MethodBrowseComp-PlusNL2Repo-Bench
No Compaction45.5±0.645.5 \pm 0.627.31±0.7127.31 \pm 0.71
Sliding Window47.7±0.447.7 \pm 0.427.53±0.8527.53 \pm 0.85
ReSum29.4±0.629.4 \pm 0.618.56±0.9318.56 \pm 0.93
AgentFold50.6±1.950.6 \pm 1.911.33±0.6211.33 \pm 0.62
Context-Folding58.9±0.558.9 \pm 0.528.94±1.9828.94 \pm 1.98
HiContext66.0±1.266.0 \pm 1.233.38±1.3233.38 \pm 1.32

Paired uncertainty analysis: We conduct paired bootstrap analyses and compute 95% confidence intervals using the percentile method to quantify uncertainty in the performance differences between methods.

BenchmarkComparisonScore differencePaired bootstrap 95% CI
BrowseComp-PlusHiContext − Context-Folding+6.6[+2.7, +20.0]
BrowseComp-PlusHiContext − Sliding Window+18.0[+12.7, +32.0]
NL2Repo-BenchHiContext − Sliding Window+5.95[+2.87, +10.99]
NL2Repo-BenchHiContext − Context-Folding+3.94[+0.15, +9.35]

All four confidence intervals exclude zero, providing statistical evidence for positive improvements in these comparisons.

We also expand the NL2Repo-Bench ablation from the original 40-task subset to all 104 tasks.

Full NL2Repo-Bench ablation:

Ablation variantScore
Trajectory-only30.25
Subtask-only25.35
State-only28.11

Trained Compactor Transfer (W2)

We also evaluate on the shopping task of WebArena-Verified Hard, with Qwen3.5-27B as task model and the BrowseComp-Plus-trained compactor.

Transfer to WebArena-Verified Hard shopping tasks:

MethodPass RateCost / Task
Sliding Window41.07%$0.094
Context Folding46.43%$0.122
HiContext (Qwen3.5-9B)48.21%$0.058
HiContext (BC+ SFT Qwen3.5-9B)51.79%$0.071

These results show that HiContext outperforms the baselines on this additional benchmark and that the BrowseComp-Plus-trained compactor improves performance on an unseen benchmark.

Training Filter and Data Split (Q3)

We split original BrowseComp-Plus questions before concatenating them and generated training rollouts only from the training split. Consequently, concatenation cannot cause an original question to appear in both training and evaluation. Across four rollout rounds, the 0.8 outcome filter retained 4,212 compaction examples. This use of task outcomes to filter training data is consistent with AgentTuning (Zeng et al., 2024). We did not ablate the filter or its threshold.

We hope these clarifications address the concerns and would greatly appreciate your reconsideration of the current assessment based on our responses. We also welcome further discussion and are happy to provide additional analysis if helpful.

Dear Reviewer azBD,

Thank you for your time and effort in reviewing this paper. We would appreciate it if you could review our rebuttal and check whether your concerns are addressed. Thanks again for your time and consideration!

Regards from authors.

Reviewer_c9uxOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes HiContext, a framework for managing an LLM agent’s context across three abstraction levels: trajectory, subtask, and state. In particular, a compactor rewrites different parts of the history according to the selected abstraction. Experiments on BrowseComp-Plus and NL2Repo-Bench show improvements over baselines.

Critical strengths

Relevant problem and well-motivated design. Managing context is an important problem for long-horizon agents. The combination of hierarchical representation tiers and deterministic replacement rules is clear and well motivated. Separating the compactor from the task agent also makes the approach modular and allows independent training.

Promising empirical results. HiContext improves over Context-Folding by 11% on BrowseComp-Plus and 13% on NL2Repo-Bench (relative improvements), while also reducing context per call. The ablations are insightful, and the paper also reports inference costs including compactor overhead, although total costs increase on coding tasks.

Critical weaknesses
  1. Results are reported without confidence intervals, which is not critical given the promising improvements across benchmarks, but it makes it harder to judge their robustness.
  2. Some improvement might come from prompt engineering instead of context management. In the implementation, _SWE_STALL_FOOTER is incorporated into the agent’s context after compaction by default for NL2Repo-Bench, while BROWSE_STALL_FOOTER for BrowseComp-Plus seems disabled by default. These instruct the LLM on how to proceed (e.g., to change its approach when stalled and, for coding, satisfy explicit installation, import, file-layout, etc. checks). These instructions go beyond what was in the history and could partly explain the improvements that the paper attributes to context management. However, the coding ablations also include these instructions, so they cannot explain the benefits over the fixed-level variants on their own, although they might affect comparisons with external baselines. In any case, clarifying whether they were used in the reported experiments, and which methods they were applied to, is important for contextualizing the results.
  3. Potential text-code discrepancy in level selection (minor). The text describes selecting the abstraction level based on semantic progress, as determined by the LLM, but the coding implementation seems to enforce a deterministic count-based policy (_rule_granularity and _compact_span in ult_compact.py), overriding the LLM’s choice. This is not necessarily problematic for the hierarchical compaction claims, but it means the coding experiments may not establish the benefit of LLM-driven semantic-level selection as the text suggests. I invite the authors to clarify whether this implementation was used in the reported experiments and to discuss this distinction.
Questions for authors

These questions correspond one-to-one to the weaknesses above.

  1. Can you provide paired uncertainty estimates?
  2. Can you confirm how _SWE_STALL_FOOTER and BROWSE_STALL_FOOTER were used in the experiments, including whether the baselines received equivalent instructions?
  3. Can you clarify exactly how the abstraction-level selection was performed in the experiments?

My current recommendation assumes 2 was either not used or used fairly across all baselines. Resolving 1 and 3 would mostly affect my confidence, but not my recommendation.

2 comments

We thank you for recognizing HiContext’s design, modular structure, and empirical results. We would like to address your concerns as follows:

Prompt Engineering and Hierarchical Compaction (W2/Q2)

In the original experiments, HiContext and its fixed-level variants used the domain-specific stall footer. The external baselines used their original prompts and received no footer. The fixed-level variants also shared HiContext’s preservation instructions for their assigned level, so this ablation supports the benefits of hierarchical compaction. We retained each external baseline’s original prompt to preserve the operations it was designed to support.

For the context engineering ablation, we rerun the compaction baselines with the same domain-specific footer appended to their compaction outputs and add preservation instructions to their original prompts. We add state-level instructions to ReSum for its consolidated task summary, subtask-level instructions to Context-Folding for its subtask-based folding, and trajectory-level instructions to AgentFold to preserve concrete execution details during its folding at every turn.

Due to time constraints during rebuttal, the additional NL2Repo-Bench experiments labeled "30 tasks" use 30 randomly sampled instances. We will extend these evaluations to all 104 instances in the camera-ready version.

Baseline scores before and after adding preservation instructions and footers:

MethodBrowseComp-Plus Original -> AugmentedNL2Repo-Bench (30 tasks) Original -> Augmented
ReSum28.7 -> 54.720.79 -> 23.41
Context-Folding58.7 -> 55.330.33 -> 31.68
AgentFold48.0 -> 49.38.56 -> 10.70
HiContext65.334.75

HiContext still outperforms augmented baselines by 10.0 percentage points on BrowseComp-Plus and 3.07 percentage points on NL2Repo-Bench. These results support the benefit of hierarchical compaction beyond the engineered prompt.

Level Selection (W3/Q3)

The reported NL2Repo-Bench runs asked the LLM to apply count-based rules in its prompt. The supplied code implements those rules directly. We will correct the experimental description and provide the run-matched code and configuration.
Changing only the level-selection instructions to use semantic progress gives 34.34 versus 33.62 for count-guided selection on NL2Repo-Bench. The comparable performance shows that the count-based rules are not necessary to obtain this result.

Statistical Significance (W1/Q1)

We supplement the evaluation with three independent runs, paired confidence intervals, and an expanded ablation on the full NL2Repo-Bench evaluation set.
Repeated-run results: We report the mean ±\pm standard deviation across three independent runs.

MethodBrowseComp-PlusNL2Repo-Bench
No Compaction45.5±0.645.5 \pm 0.627.31±0.7127.31 \pm 0.71
Sliding Window47.7±0.447.7 \pm 0.427.53±0.8527.53 \pm 0.85
ReSum29.4±0.629.4 \pm 0.618.56±0.9318.56 \pm 0.93
AgentFold50.6±1.950.6 \pm 1.911.33±0.6211.33 \pm 0.62
Context-Folding58.9±0.558.9 \pm 0.528.94±1.9828.94 \pm 1.98
HiContext66.0±1.266.0 \pm 1.233.38±1.3233.38 \pm 1.32

Paired uncertainty analysis: We conduct paired bootstrap analyses and compute 95% confidence intervals using the percentile method to quantify uncertainty in the performance differences between methods.

BenchmarkComparisonScore differencePaired bootstrap 95% CI
BrowseComp-PlusHiContext − Context-Folding+6.6[+2.7, +20.0]
BrowseComp-PlusHiContext − Sliding Window+18.0[+12.7, +32.0]
NL2Repo-BenchHiContext − Sliding Window+5.95[+2.87, +10.99]
NL2Repo-BenchHiContext − Context-Folding+3.94[+0.15, +9.35]

All four confidence intervals exclude zero, providing statistical evidence for positive improvements in these comparisons.

We also expand the NL2Repo-Bench ablation from the original 40-task subset to all 104 tasks.

Full NL2Repo-Bench ablation:

Ablation variantScore
Trajectory-only30.25
Subtask-only25.35
State-only28.11

We hope these clarifications address the concerns. We also welcome further discussion and are happy to provide additional analysis if helpful.

Thank you for the clarifications and experiments. My concerns are resolved, and I maintain my score.

Reviewer_w69zOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes HICONTEXT, a hierarchical context-compaction approach for improving long-horizon LLM agents by progressively representing past interactions at three levels of abstraction: trajectory, subtask, and overall task state. Instead of repeatedly summarizing the entire interaction history into a single representation, the method allows older information to be progressively rewritten into higher-level summaries while preserving finer-grained information for more recent interactions. An LLM-based compactor determines the appropriate abstraction level, while deterministic rules determine which portion of the context should be replaced. The approach is evaluated on BrowseComp-Plus and NL2Repo-Bench, where it outperforms the evaluated context-management baselines while substantially reducing the average context length processed per model call. The experiments also show that dynamically selecting among abstraction levels performs better than using any single fixed level, and that the compactor can be further improved through teacher-supervised fine-tuning. Overall, the paper presents a simple and practically motivated approach for balancing information retention and context efficiency in long-running LLM agents.

Critical strengths
  1. The paper addresses an important practical challenge for long-horizon LLM agents: controlling the growth of interaction context while retaining information needed for future reasoning
  2. The empirical results are encouraging, with HICONTEXT outperforming the evaluated context-management baselines on both BrowseComp-Plus and NL2Repo-Bench
Critical weaknesses
  1. Limited evaluation across models and tasks. The main experiments use a single task model and evaluate on only two benchmarks, BrowseComp-Plus and NL2Repo-Bench. This makes it difficult to determine whether the benefits of HICONTEXT generalize across model families, agent architectures, and other long-horizon tasks.
  2. Reliance on domain-specific prompt engineering. The compaction instructions are substantially customized for the two domains, including specific guidance about what information should be preserved. This makes it difficult to disentangle the contribution of hierarchical context compaction from the benefit of carefully engineered, task-specific prompts.
  3. Missing controls for the abstraction-level selection mechanism. Although the paper compares HICONTEXT against fixed trajectory-only, subtask-only, and state-only variants, additional controls such as random or heuristic level selection would help establish how much of the improvement specifically comes from the learned dynamic selection of abstraction levels.
  4. Lack of statistical significance and uncertainty analysis. The empirical results are primarily presented as point estimates without confidence intervals or statistical significance testing. This is particularly relevant for the NL2Repo-Bench ablation, which is conducted on a randomly sampled subset of 40 instances.
  5. Context reduction does not consistently translate into cost reduction. While HICONTEXT substantially reduces average context length per task-model call, the total cost on NL2Repo-Bench is higher than the no-compaction baseline. Thus, the method demonstrates clear context-efficiency benefits, but not consistent end-to-end cost savings.
  6. Compression ratios are not sufficiently characterized. The paper reports average context length per model call, but it does not clearly quantify how much information is compressed at each individual compaction operation or provide separate compression ratios for trajectory, subtask, and state-level compaction. Such analysis would make the efficiency benefits of the hierarchical design easier to understand.
  7. Limited evidence for transfer of the trained compactor. The teacher-supervised compactor experiments demonstrate improved performance on BrowseComp-Plus, but do not establish whether the learned compaction policy transfers to unseen domains, benchmarks, or different task models.
  8. BrowseComp-Plus uses a constructed long-context setting. The BrowseComp-Plus evaluation concatenates five questions into a longer task. While this provides a useful stress test, additional evaluation on naturally long-running tasks would provide stronger evidence that the benefits extend beyond this constructed setting.
Questions for authors

Please refer weakness section above

3 comments

We thank you for recognizing the practical motivation behind our method and its performance gains on both benchmarks. We would like to address your concerns as follows:

Prompt Engineering and Hierarchical Compaction (W2)

In the original experiments, HiContext and its fixed-level variants used the domain-specific stall footer; the external baselines used their original prompts and received no footer. The fixed-level variants also shared HiContext’s preservation instructions for their assigned level, so this ablation supports the benefits of hierarchical compaction. We retained each external baseline’s original prompt to preserve the operations it was designed to support.

For the context engineering ablation, we rerun the compaction baselines with the same domain-specific footer appended to their compaction outputs and add preservation instructions to their original prompts. We add state-level instructions to ReSum for its consolidated task summary, subtask-level instructions to Context-Folding for its subtask-based folding, and trajectory-level instructions to AgentFold to preserve concrete execution details during its folding at every turn.

Due to time constraints during rebuttal, the additional NL2Repo-Bench experiments labeled "30 tasks" use 30 randomly sampled instances. We will extend these evaluations to all 104 instances in the camera-ready version.

Baseline scores before and after adding preservation instructions and footers:

MethodBrowseComp-Plus Original -> AugmentedNL2Repo-Bench (30 tasks) Original -> Augmented
ReSum28.7 -> 54.720.79 -> 23.41
Context-Folding58.7 -> 55.330.33 -> 31.68
AgentFold48.0 -> 49.38.56 -> 10.70
HiContext65.334.75

HiContext still outperforms augmented baselines by 10.0 percentage points on BrowseComp-Plus and 3.07 percentage points on NL2Repo-Bench. These results support the benefit of hierarchical compaction beyond the engineered prompt.

Level Selection Mechanism Control (W3)

With the rest of HiContext configuration held fixed, random level selection scores 60.7 versus 65.3 for HiContext on BrowseComp-Plus and 31.19 versus 34.75 on NL2Repo-Bench (30 tasks). This comparison shows that the level-selection policy contributes to performance.

Statistical Significance and Full NL2Repo Ablation (W4)

We supplement the evaluation with three independent runs, paired confidence intervals, and an expanded ablation on the full NL2Repo-Bench evaluation set.
Repeated-run results: We report the mean ±\pm standard deviation across three independent runs.

MethodBrowseComp-PlusNL2Repo-Bench
No Compaction45.5±0.645.5 \pm 0.627.31±0.7127.31 \pm 0.71
Sliding Window47.7±0.447.7 \pm 0.427.53±0.8527.53 \pm 0.85
ReSum29.4±0.629.4 \pm 0.618.56±0.9318.56 \pm 0.93
AgentFold50.6±1.950.6 \pm 1.911.33±0.6211.33 \pm 0.62
Context-Folding58.9±0.558.9 \pm 0.528.94±1.9828.94 \pm 1.98
HiContext66.0±1.266.0 \pm 1.233.38±1.3233.38 \pm 1.32

Paired uncertainty analysis: We conduct paired bootstrap analyses and compute 95% confidence intervals using the percentile method to quantify uncertainty in the performance differences between methods.

BenchmarkComparisonScore differencePaired bootstrap 95% CI
BrowseComp-PlusHiContext − Context-Folding+6.6[+2.7, +20.0]
BrowseComp-PlusHiContext − Sliding Window+18.0[+12.7, +32.0]
NL2Repo-BenchHiContext − Sliding Window+5.95[+2.87, +10.99]
NL2Repo-BenchHiContext − Context-Folding+3.94[+0.15, +9.35]

All four confidence intervals exclude zero, providing statistical evidence for positive improvements in these comparisons.

We also expand the NL2Repo-Bench ablation from the original 40-task subset to all 104 tasks.

Full NL2Repo-Bench ablation:

Ablation variantScore
Trajectory-only30.25
Subtask-only25.35
State-only28.11

Task Model, Benchmark and Trained Compactor Transfer (W1 + W7)

We repeat the comparison on BrowseComp-Plus with Gemma-4-26B-A4B-it as the task model.

BrowseComp-Plus results with Gemma-4-26B-A4B-it as the task model:

MethodScore
Sliding Window20.0
Context Folding30.7
HiContext (Qwen3.5-9B)32.6
HiContext (BC+ SFT Qwen3.5-9B)42.0

We also evaluate on the shopping task of WebArena-Verified Hard, with Qwen3.5-27B as task model and the BrowseComp-Plus-trained compactor.

Transfer to WebArena-Verified Hard shopping tasks:

MethodPass RateCost / Task
Sliding Window41.07%$0.094
Context Folding46.43%$0.122
HiContext (Qwen3.5-9B)48.21%$0.058
HiContext (BC+ SFT Qwen3.5-9B)51.79%$0.071

These results support transfer of HiContext and its BrowseComp-Plus-trained compactor to another task model and an unseen benchmark.

Cost Reduction (W5)

On NL2Repo-Bench, HiContext reduces context per task-model call and costs more per task than the original No Compaction run. When prompted to continue after stopping below $1.30, No Compaction averages $1.408 per task and scores 29.20, compared with $1.273 and 34.75 for HiContext.

Compression Ratio (W6)

We measure the ratio as replacement tokens divided by rewritten-span tokens. The compression ratios for trajectory, subtask, and state are 1.58%, 1.17%, and 2.10% on BrowseComp-Plus and 1.35%, 0.93%, and 2.16% on NL2Repo-Bench.

Naturally Long-running Tasks (W8)

We would like to clarify that NL2Repo-Bench already evaluates naturally long-running tasks. Each instance is a single repository-generation task, whose interaction horizon arises from implementing, testing, and debugging the repository. The five-question BrowseComp-Plus tasks deliberately extend the horizon, following the multi-objective task setup used in MEM1 (Zhou et al., 2025). On individual BrowseComp-Plus questions, HiContext scores 78.7 versus 74.0 for Sliding Window (Appendix B.2).

We hope these clarifications address the concerns and would greatly appreciate your reconsideration of the current assessment based on our responses. We also welcome further discussion and are happy to provide additional analysis if helpful.

Dear Reviewer w69z,

Thank you for your time and effort in reviewing this paper. We would appreciate it if you could review our rebuttal and check whether your concerns are addressed. Thanks again for your time and consideration!

Regards from authors.

Comments on the paper

1 comment

We thank all reviewers for their thoughtful feedback. We have revised the manuscript to incorporate the additional experiments and clarify the experimental protocol. Below, we summarize the main updates addressing the concerns raised across reviews.

1. Hierarchical compaction provides gains beyond prompt engineering.

We augment external compaction baselines with the same domain-specific stall footer and preservation instructions appropriate to their compaction operations. HiContext remains ahead of the strongest augmented baseline, shown in Appendix B.2 and Table 4. Together with the fixed-level ablations, which share HiContext’s corresponding preservation instructions, these results support the contribution of hierarchical compaction beyond these prompt additions.

2. Additional controls support the hierarchy and level-selection policy.

We expand the fixed-level NL2Repo-Bench ablation to all 104 tasks. Replacing HiContext’s level-selection policy with random selection also lowers performance on both benchmarks, shown in Section 4.3 and Tables 2–3.

3. Repeated runs and paired uncertainty analyses strengthen the empirical evidence.

We add three independent runs on both benchmarks, including the full NL2Repo-Bench evaluation set. HiContext achieves 66.0 ± 1.2 on BrowseComp-Plus and 33.38 ± 1.32 on NL2Repo-Bench, with the highest mean scores among the evaluated methods. Paired bootstrap 95% confidence intervals for its improvements over Context-Folding and Sliding Window exclude zero on both benchmarks, shown in Appendix B.3 and Tables 5–6.

4. New experiments provide evidence of transfer.

On WebArena-Verified Hard shopping tasks, HiContext achieves 48.21% pass rate, increasing to 51.79% with the BrowseComp-Plus-trained compactor, compared with 46.43% for Context-Folding, shown in Section 4.5. With Gemma-4-26B-A4B-it as the task model on BrowseComp-Plus, the trained compactor scores 42.0, compared with 32.6 for the base compactor and 30.7 for Context-Folding, shown in Appendix C.3.

The revision also clarifies evaluation metrics, training data separation, compression ratios. To further examine cost efficiency, we also add a continued-execution control on NL2Repo-Bench.

We sincerely appreciate reviewers' feedback and welcome further discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.