acceptodds
Under review as a conference paper at ICLR 2027

OWG: Structuring Agent Evidence as Outcome-Witness Graphs for Efficient Long-Horizon Task Closure

Abstract

An agent can retain its entire interaction history and still lose track of what remains to be done. Long transcripts place decisive evidence alongside obsolete outputs, repeated observations, and failures unrelated to the requested result. We introduce OWG, a training free method that organizes an agent's context around the evidence for completing its current task. An Outcome-Witness Graph links each requirement to observations that support or contradict it, the output being produced, and the checks performed on that output. After every tool response, OWG updates these relations and uses the same state to select the next prompt, direct the next action, and decide when to return the result. Raw observations remain available for targeted recovery, while completed work and repeated messages leave the active context. Across six benchmarks and five models, OWG lowers cost in all 30 evaluated settings, with task success increasing in 17, remaining unchanged in six, and decreasing in seven. Five runs on fixed PinchBench and SWE Bench subsets reduce tokens by 53.2% and 74.2%, respectively, while Pass@1 increases by 0.4 percentage points on each. These results identify task evidence as a useful basis for reducing the cost of sustained agent execution.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, long-horizon agents, context management, agent memory, task evidence, task closure

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_SffjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes OWG, a training-free context management method for long-horizon agents. Instead of keeping the full interaction history, OWG maintains an Outcome-Witness Graph that links task requirements, observations (supporting or contradicting evidence), the current output, and its checks. After each tool call, the graph is updated and used to build a short “control capsule” for the next prompt, suggest the next action, and decide when to stop. The authors evaluate OWG on six benchmarks (three 50-task SWE-bench subsets, PinchBench, ClawBench and TerminalBench-2) with five models. OWG lowers cost in all 30 settings, while success increases in 17, stays the same in 6 and decreases in 7.

Critical strengths
  1. The cost reduction is consistent across five models and six benchmarks, which is useful for long-horizon agents in practice.
  2. The idea of organizing context around unfinished requirements and their evidence, instead of the raw history, is intuitive and well motivated.
  3. The implementation details are well documented, including the controller rules, thresholds, prompt templates and cost accounting (uncached input, cache read and output are counted separately), which makes the method easy to reproduce.
Critical weaknesses
  1. It is unclear where the gain comes from. The graph is built with hand-crafted rules and many hard-coded thresholds, and OWG also adds behavior control (e.g., Semantic Review, Release Valve) and directly truncates long tool outputs on PinchBench. Removing Semantic Review alone drops the solved tasks from 37 to 25, so the gain may come from these extra rules rather than the graph itself.
  2. The baselines are weak and the success gains are likely within noise. The main results only compare with the full-history agent, and other context management methods are only tested on 50 SWE tasks with one model. With 50-task subsets, 2 points is just one task, and Pass@1 only improves by 0.4 points in repeated runs.
Questions for authors
  1. How much of the token saving comes from simple tool output truncation?
  2. Can you add a baseline that uses the same behavior control (Semantic Review, Release Valve) with full history, to separate the effect of the graph from these rules?
  3. How are the thresholds and character limits chosen? Are they tuned on the evaluation tasks?
Reviewer_3PezOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

OWG is a training-free method that links task requirements, evidence, outputs, and checks in a graph. It uses this graph to select context, guide actions, and decide when to stop. Across six benchmarks and five models, OWG reports lower cost in all 30 settings, with mixed changes in success. Repeted runs show large token savings and similar mean Pass@1.

Critical strengths
  1. Useful design. Connecting evidence to requirements and current outputs helps agents track unfinished work and avoid stale information.
  2. Broad efficiency results. Experiments cover six benchmarks and five models. Repeated runs show 53.2% and 74.2% token savings on two subsets, with similar mean Pass@1.
Critical weaknesses

1)The graph’s contribution is unclear. OWG also adds review, intervention, stopping, and recovery rules. The ablations do not compare against full history or a flat evidence summary with the same controls. Removing the release valve alone increases tokens from 20.60M to 112.83M. 2)Some scores do not establish correct task completion. In Appendix D.5, OWG receives full credit for a current-stock-price task after using simulated data with a disclaimer. The checks verify report content and format, but not price accuracy. This raises concerns about the meaning of some general tool-use results.

Questions for authors

Can you compare OWG with full history or a flat evidence summary using the same execution controls, to isolate the graph’s benifit?

Reviewer_dPdVOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

OWG links task requirements, evidence, current artifacts, and validation results to guide agent context selection, actions, and task completion. Across six benchmarks and five models, it consistently reduces reported monetary cost, with smaller and mixed changes in task success.

Critical strengths
  1. The paper addresses a meaningful problem: retaining interaction history does not ensure that an agent understands what remains unfinished or which evidence still applies.
  2. The efficiency results are promising. Reported costs decrease across all 30 model–benchmark combinations, supported by repeated runs and detailed token accounting.
  3. Component ablations, trajectory examples, and failure analysis provide useful insight. The paper also acknowledges success-rate regressions and premature completion.
Critical weaknesses
  1. The graph’s added value is not sufficiently isolated. Table 26 provides component ablations, but no comparison replaces the graph with a structured task-state summary while retaining the same review, intervention, stopping, and recovery rules. The contribution of graph-based evidence organization therefore remains unclear.

  2. The detailed comparisons cover only one backbone. Although Table 1 evaluates five models, the baseline comparisons and component ablations in B.7–B.8 use only DeepSeek‑V4‑Flash. A recent strong backbone should be tested to establish whether these additional controls still outperform a simpler harness.

  3. Runtime efficiency is not evaluated. Token-based costs, including the hybrid builder, are accounted for. However, end-to-end wall-clock time and graph-update/controller-processing time are not reported. Monetary savings do not establish lower execution latency.

  4. The detailed SWE evaluation uses a narrow sample. The method comparisons and ablations use a convenience sample of 50 tasks, including 37 from Astropy and Django. Repeated runs measure execution variability but do not address sample representativeness.

The problem is meaningful and the efficiency results are promising. If the authors satisfactorily address these concerns, particularly contribution isolation and broader validation, I will raise my rating to 3 (Weak acceptance).

Questions for authors
  1. Does OWG outperform a structured task-state summary under identical control rules, including on a recent strong backbone?
  2. Can the key comparisons be validated on a larger or randomly sampled SWE task set?
  3. What is the end-to-end runtime, and how much time is spent maintaining the graph and executing the controller?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.