- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes OWG, a training-free context management method for long-horizon agents. Instead of keeping the full interaction history, OWG maintains an Outcome-Witness Graph that links task requirements, observations (supporting or contradicting evidence), the current output, and its checks. After each tool call, the graph is updated and used to build a short “control capsule” for the next prompt, suggest the next action, and decide when to stop. The authors evaluate OWG on six benchmarks (three 50-task SWE-bench subsets, PinchBench, ClawBench and TerminalBench-2) with five models. OWG lowers cost in all 30 settings, while success increases in 17, stays the same in 6 and decreases in 7.
Critical strengths
- The cost reduction is consistent across five models and six benchmarks, which is useful for long-horizon agents in practice.
- The idea of organizing context around unfinished requirements and their evidence, instead of the raw history, is intuitive and well motivated.
- The implementation details are well documented, including the controller rules, thresholds, prompt templates and cost accounting (uncached input, cache read and output are counted separately), which makes the method easy to reproduce.
Critical weaknesses
- It is unclear where the gain comes from. The graph is built with hand-crafted rules and many hard-coded thresholds, and OWG also adds behavior control (e.g., Semantic Review, Release Valve) and directly truncates long tool outputs on PinchBench. Removing Semantic Review alone drops the solved tasks from 37 to 25, so the gain may come from these extra rules rather than the graph itself.
- The baselines are weak and the success gains are likely within noise. The main results only compare with the full-history agent, and other context management methods are only tested on 50 SWE tasks with one model. With 50-task subsets, 2 points is just one task, and Pass@1 only improves by 0.4 points in repeated runs.
Questions for authors
- How much of the token saving comes from simple tool output truncation?
- Can you add a baseline that uses the same behavior control (Semantic Review, Release Valve) with full history, to separate the effect of the graph from these rules?
- How are the thresholds and character limits chosen? Are they tuned on the evaluation tasks?