- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes ReFold, a training-free context management technique for long-horizon LLM agents. It compresses the context passed to the model while preserving the original interaction history through Deduplication, STALE-based Folding, Chunked Rendering, and Restoration.
Across five benchmarks, the paper reports substantial reductions in token consumption and KV-cache usage while maintaining comparable aggregate task success rates. Its vLLM-based serving experiments also demonstrate improved prefix-cache reuse through chunked rendering and reduced scheduling delays under high concurrency due to lower KV-cache pressure.
Critical strengths
S1. Practical and Meaningful Serving Improvements
ReFold offers a practical approach to context management that requires neither additional training nor auxiliary LLM calls. Beyond reducing token consumption, the paper demonstrates meaningful system-level benefits in a vLLM-based serving environment. In particular, reducing per-session KV-cache occupancy alleviates memory contention under high concurrency, leading to substantial improvements in scheduling delay, task completion time, and inference cost.
I consider this an important strength because the paper connects context compression to actual serving efficiency rather than relying solely on token reduction as a proxy.
S2. Comprehensive Experimental Evaluation
The evaluation covers five benchmarks, two backbone models, multiple context budgets, and varying concurrency levels. The authors also compare ReFold with LLMLingua-2, SWE-Pruner, and AgentDiet under the same harness in Appendix D.4, strengthening the empirical comparison.
ReFold achieves substantial reductions in token usage and KV-cache footprint while maintaining comparable aggregate task success rates across the main benchmarks. The ablation studies further demonstrate the contributions of Folding, Deduplication, and Chunked Rendering. These results provide credible evidence of the practical value of the proposed system.
Critical weaknesses
W1. Missing Baselines for STALE-Based Folding
ReFold modifies the agent's prompt to generate PROGRESS/STALE metadata and uses the selected indices to fold previous turns (Section 3.2). Appendix E demonstrates that Folding contributes substantially to token and KV-cache savings.
However, the experiments do not separate the effect of the modified prompt on task execution from the effect of context compression. They also do not show whether model-selected STALE indices are more effective than simpler folding policies.
A metadata-only baseline using the same prompt without compression, together with a static or heuristic folding baseline under matched prompting and chunking conditions, would help isolate the contribution of STALE-based Folding.
W2. Limited Evidence for Task-Success Preservation on SWE-bench Pro
In Table 1, the baseline solves only 5/50 (10%) and 6/50 (12%) SWE-bench Pro tasks. With nearly 90% of tasks already failing, these results provide limited evidence about whether ReFold preserves the ability to solve difficult tasks.
Appendix A.4 provides useful evidence of run-to-run variability, including task-level success changes. However, it does not directly analyze which SWE-bench Pro tasks regress when switching from Base to ReFold. The reported aggregate success rates are comparable, but task-level regressions could still be hidden by changes in which tasks are solved.
A larger SWE-bench Pro evaluation or a stronger backbone with a higher baseline success rate would make the task-success preservation claim more convincing.
W3. Technical Reversibility Does Not Guarantee Effective Information Recovery
ReFold preserves the original interaction history, but this does not guarantee that the agent recognizes when folded information is needed or restores it in time. In long-horizon tasks, previously discarded information may become relevant again, and restoration may occur only after incorrect decisions have already been made.
Appendix C.3 demonstrates a successful recovery through command re-execution, but does not establish how reliably recovery occurs across trajectories. Similarly, Figure 4c evaluates technical recoverability rather than actual information use or task preservation.
An analysis of missed or delayed recovery and its impact on information retrieval and task success is needed.
Questions for authors
Q1. Isolating the Contributions of Prompting and Folding
Could the authors report a metadata-only baseline that uses PROGRESS/STALE prompting without compression? Could they also compare ReFold with a simple static or heuristic folding policy using the same prompt and chunked rendering?
Q2. Task-Level Regressions
How many tasks solved by Base fail under ReFold, and vice versa? For tasks solved by Base but not by ReFold, could the authors examine whether the failures involved missing information from folded turns? How do these failures compare with the task-level fluctuations reported in Appendix A.4?
Could the authors also strengthen the SWE-bench Pro evaluation with more tasks or a stronger backbone?
Q3. Reliability of Information Recovery
How frequently do agents actually recover folded information, and how often is necessary information never recovered or recovered only after incorrect decisions?
For tasks solved by Base but not by ReFold, could the authors analyze whether missing information or delayed recovery contributed to the failures? Could they also directly evaluate the agent's ability to retrieve and use information from earlier turns after folding?
11 comments
W1 / Q1. Separating the prompt from folding, and agent-selected STALE vs. simpler policies.
All runs below use our default setting: SWE-bench Verified, Qwen3.6-27B, 50 tasks, full context window, w = 8.
(a) Does the prompt alone contribute to the results?
We find that the prompt alone does not account for the savings. With the prompt but no compression, the KV cache stays at the base level and task success is not hurt. The savings come from the operators.
| Method | Resolve | Steps | Tokens (k) | KV cache (GiB) |
|---|---|---|---|---|
| base | 34 | 50 | 859 | 1.66 |
| prompt only (no compression) | 35 (+1) | 43 (−14%) | 1,029 (+20%) | 1.67 (+1%) |
| ReFold | 35 (+1) | 40 (−20%) | 477 (−44%) | 0.80 (−52%) |
(b) Does the agent choose well what to fold?
In our experiments, it does. The agent writes its PROGRESS note and STALE selection in the same turn, so a simpler policy can take over from the agent in two ways; we tested both, and each performs worse than the agent's own choice.
Replacing both PROGRESS and STALE. With neither the agent's notes nor its selection, the policy is essentially observation masking (mask), the static alternative the reviewer suggests, which Figure 2 evaluates at a 16k budget. Masking rewrites old observations at every step, so it keeps invalidating the prefix cache: the cache hit falls to 23%, and the lost cache alone adds 5.1¢ per task, 31% of its cost. Even at its best setting, it resolves only 17 of 50 tasks, under half of what compact resolves at the same budget (Appendix B.2, Table 8).
Replacing only STALE. The agent must still write PROGRESS at every step, and a rule then overrides its STALE choice. We use the simplest such rule, a random sample of the same size from the same window, to test whether the agent's choice is selective. As the reviewer suggests, the prompt and chunked rendering stay fixed. Compared with the agent's own choice, random selection leads the agent to re-run 55–72% more folded commands and use 32–37% more tokens. Task success is only slightly lower, within run-to-run noise, likely because even when a useful step is folded, its PROGRESS note tells the agent what it accomplished, so the agent can continue from that summary or restore the full turn if it needs a missing detail. This control comes from a separate run, so its base and ReFold rows differ slightly from (a).
| STALE source | Resolve | Steps | Tokens (k) | KV cache (GiB) | Re-runs of folded steps |
|---|---|---|---|---|---|
| base | 36 | 48 | 774 | 1.55 | — |
| agent (ReFold) | 35 (−1) | 40 (−17%) | 421 (−46%) | 0.74 (−52%) | 128 |
| random (seed 0) | 33 (−3) | 50 (+4%) | 576 (−26%) | 0.75 (−52%) | 198 |
| random (seed 1) | 32 (−4) | 43 (−10%) | 557 (−28%) | 0.70 (−55%) | 220 |
We have added both controls to Appendix E.2 (Table 13).
W2 / Q2. Task-level regressions and SWE-bench Pro.
(a) Which tasks flip.
We agree that aggregate success rates can hide task-level regressions. Appendix A.4 reports task-level flips on SWE-bench Verified between two runs of base and of ReFold. For the other four benchmarks of Table 1 (Qwen3.6-27B), the per-task outcomes are:
| Benchmark | Base | ReFold | Lost (base only) | Gained (ReFold only) |
|---|---|---|---|---|
| SWE-bench Pro | 5 | 6 (+1) | 1 | 2 |
| Multi-SWE-bench | 25 | 25 (±0) | 3 | 3 |
| Terminal-Bench 1.0 | 23 | 23 (±0) | 2 | 2 |
| Terminal-Bench 2.1 | 23 | 26 (+3) | 3 | 6 |
| Total | 76 | 80 (+4) | 9 | 13 |
On no benchmark does ReFold lose more tasks than it gains on Qwen3.6-27B, and overall it gains slightly more than it loses. These flips are within the task-level fluctuation of Appendix A.4, where 2–7 tasks flip in each direction between two identical runs of one configuration. We have added this table to Appendix A.4 (Table 6b).
(b) Do the losses come from folded information?
We ran the 9 lost tasks again in the same setting. They fall into three groups:
| On a repeat run | Tasks | # |
|---|---|---|
| ReFold solves it | webclients-0d0267 (Pro), bytes-721 (MSB), sam-cell-seg (TB 2.1) | 3 |
| Base also fails it | Catch2-2849 (MSB), ponyc-1486 (MSB), llm-inference-batching-scheduler (TB 1.0) | 3 |
| Loss reproduces | puzzle-solver (TB 1.0), overfull-hbox (TB 2.1), mailman (TB 2.1) | 3 |
Appendix D.4 walks through the 3 losses that reproduce: in each, the task statement and the files that matter either stay unfolded or are re-read when needed, so we found none of them to come from folded information.
(c) SWE-bench Pro with a stronger backbone.
Following the reviewer's suggestion, we ran Claude Opus 5.5 on the same 50 SWE-bench Pro tasks and have added the result to the main text (Table 1):
| Method | Resolve | Lost / Gained | Steps | Tokens (k) | Cost (¢) |
|---|---|---|---|---|---|
| base | 30 | — | 11 | 230.9 | 28.2 |
| ReFold | 31 (+1) | 3 / 4 | 10 (−9%) | 140.9 (−39.0%) | 27.5 (−2.3%) |
With base solving 30 of the 50 tasks, losses and gains remain balanced (3 lost, 4 gained), so ReFold preserves task success on a stronger backbone as well, while cutting tokens by 39%.
Response to Reviewer HJCf
We thank the reviewer for the careful and constructive review. We ran the suggested controls and analyses, and we reply in three parts below: