acceptodds
Under review as a conference paper at ICLR 2027

Suffice: Inferring Scopes and Interfaces for Reusable Agent Execution with Sufficient Information Flow

Abstract

Long-horizon agents often repeat similar tool calls and recovery steps across tasks, motivating the abstraction of repeated agent execution into reusable callable units (e.g., skills and task workflows) to reduce context and decision costs. Such abstraction therefore appears to require a tradeoff between omitting intermediate information that may alter subsequent decisions and retaining excessive history that erodes its benefits. We present Suffice, which solves this problem by deriving precise conditions on the information that must cross an abstraction boundary to preserve agent decisions. We further prove that passive trajectories generally cannot uniquely identify this information. Suffice uses controlled replay queries to hold candidate information fixed while varying other information and observing whether the agent's decision changes. Based on these queries, it identifies the minimal information required to preserve each decision and uses it to construct callable units. Across 200 synthetic programs, Suffice achieves an F1 score of 0.974 in recovering decision dependencies and exactly recovers 96.0% of abstraction boundaries. On AppWorld, Suffice improves Scenario Goal Completion (SGC) over the existing method ICAL by up to 23.5%, while achieving comparable performance to retaining all observed dataflow with 48.7% fewer interface tokens. Overall, agent execution abstraction appears to require a tradeoff between efficiency and task performance. Suffice shows that it need not.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
LLM Agents, Agent Workflows, Agent Skills, Agent Abstraction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_pXCBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper explores how to encapsulate a segment of an agent's execution process into a reusable tool. The author uses controlled replay to examine which information affects the agent's decision-making, and then determines which steps the tool should include, as well as what needs to be passed in and returned. Finally, another model generates the tool code. Experiments use synthetic programs to check if the dependency judgments are correct, and then compare the task completion rate and interface size on AppWorld.

Critical strengths

S1. Considering which steps a tool includes and its input/output together, this approach is reasonable. When the scope of encapsulation changes, the information needed from the outside will also change. The two sides cannot be processed completely separately.

S2. The synthetic experiment has a clear correct answer and also takes into account the situation where multiple pieces of information must be used together and different pieces of information can be substituted for each other. This can directly check whether the method has identified the correct dependencies.

S3. Compared with the control group that retains all the observation information, Suffice uses a shorter interface under the same encapsulation scope, and the task completion rates measured on both sides are also relatively close. This comparison supports the value of simplifying the interface.

Critical weaknesses

W1. I still don't have a clear understanding of how much manual preparation is actually required for this process. For instance, how should the information in the execution record be broken down into variables, and how should a reasonable new history be constructed during replay? The appendix provides some requirements, but it doesn't clarify exactly how to do it in AppWorld. These are prerequisites for the subsequent search to run and will affect my judgment on the degree of automation of this method.

W2. Retaining sufficient information does not mean that the tool written by another model will make the original decision. Passing the interface check does not mean that the internal logic is correct. The current task completion rate indicates that the tool is useful, but it has not directly verified the decision preservation emphasized in the paper.

W3. The paper states that these tools can be reused in new tasks, but I didn't clearly understand if the execution records of the test tasks were used when generating the tools. If these records have been reviewed and the same task is run again, it still cannot be said that the tool can perform well on unseen tasks.

W4. The theoretical definition of sufficiency uses the action-distribution reconstruction error ρ\rho, but the replay check measures Δ\Delta, the maximum total variation distance between action distributions induced by histories that agree on the retained information. The appendix gives bounds relating the two, but when a nonzero error is allowed, why does this recover all the minimal sufficient sets as defined?

Questions for authors

See weaknesses.

Can you run the original agent and the generated tool multiple times from the same initial state, and compare their action distributions? This way, we can directly check how much of the original decision-making has been retained after encapsulation.

Reviewer_ixR3OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Suffice, an automated skill compilation approach that extracts reusable skills from agent execution trajectories. The key challenge addressed by the paper is how to determine the appropriate scope and input/output interfaces of compiled skills while reducing context overhead and preserving the original agent's decision-making capabilities. To address this challenge, the authors first formalize the problem in terms of identifying minimal sufficient information sets and derive several theoretical conditions for decision preservation. Building on this formulation, Suffice employs controlled replay and iterative input elimination to identify decision-relevant information and construct a dependency hypergraph, which is subsequently used to infer the scope and interfaces of reusable skills.

The authors evaluate Suffice on 200 synthetic programs and the AppWorld benchmark. On synthetic programs, Suffice achieves a Hyperedge F1 score of 0.974 and a boundary exact accuracy of 96.0%. On AppWorld, Suffice improves task success rates by up to 23.5% compared with ICAL. In addition, compared with retaining all observed dataflow, Suffice reduces interface token costs by 48.7% while maintaining comparable task performance.

Critical strengths

Clear problem formalization.

The paper provides a clear formulation of the scope and interface selection problem in agent workflow abstraction by introducing minimal sufficient information sets. This formulation offers a relatively unified and principled perspective for understanding information dependencies in agentic workflows and provides a useful foundation for reasoning about decision-preserving abstractions.

Critical weaknesses
  • Limited methodological novelty.

My primary concern is that the core method proposed in this paper appears to be closely related to classical delta debugging techniques, which have been extensively studied in software engineering for decades. In particular, the central idea of progressively removing candidate inputs and examining whether the resulting behavior changes is largely consistent with established input minimization approaches. While applying these techniques to agent skill compilation is potentially useful, the paper would benefit from a more thorough discussion of its connections to existing methods, as well as direct empirical comparisons with these established techniques. Without such comparisons, it is somewhat difficult to assess the methodological novelty of the proposed approach.

  • Limited theoretical novelty and unclear practical guarantees.

Although the paper places considerable emphasis on its theoretical contributions, I find that several of the main results appear relatively straightforward. Specifically, Theorem 1 seems to follow fairly directly from the definition of minimal sufficient information sets. Theorem 2 primarily relies on standard coupling and union-bound arguments, while Theorem 3 presents a relatively simple example of observational non-identifiability. While these results help formalize the problem, their technical novelty appears somewhat limited.

More importantly, I am not fully convinced that these theoretical results provide sufficiently strong guarantees that the generated skills can preserve the behavior of the original agents in practical settings. In particular, the dependency recovery guarantee relies on rather strong assumptions, including sufficient coverage of all reachable execution histories, a nonzero sampling probability for every reachable history, and exhaustive consideration of candidate information sets. These assumptions may be difficult to satisfy in realistic agentic workflows. I would therefore appreciate a more careful discussion of the practical implications and limitations of these theoretical guarantees, especially regarding the extent to which they can support the

Questions for authors

Please refer to the weakness section.

Reviewer_SCbrOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper focuses on the problem of turning agent execution into reusable skills. Specificly on wha tis the critical information to cross the skill boundary: too much and it kills efficiency, while too little hurts accuracy/decisions. The key insight of the authors is that the interface should be exact what each decision depends on. Find that requires interventions, not logs. Based on this insight, they build Suffice a systems that replays queries to find the minimal sufficient information in order to derive the right skill boundaries. They get good results on synthetic programs and AppWord (compared to ICAL). They also matched the performance of keep everything with about half the tokens used.

Critical strengths
  • Clear rule for what a skill's inputs & output should be. Past work is heuristic driven
  • Shows that past are not enough, you have to actively test what each decision depends on.
  • The synthetic tests are clean, Suffice wins where it should.
Critical weaknesses
  • Unclear why Suffice, a system that copies agents behavior can score so much better for AppWorld. There needs to be somethign else contributing to this result,
  • The experiments are rather small and wins come down to a few scenarios.
  • It would be nice to compare to the four quoted methods that are competitive to this work (AWM, ASI, TraceCompiler, and Skll-Disco).
  • The theorems are direct implications of the definitions, so I am not sure what is the value is. Did I miss something?
Questions for authors

The biggest question is where does the 30 point gain comes from: If you keep the compiler but use a simple interface (full history or some random subset), what do you get? How often are the skills inovked and what is SGC with and without them?

The paper would also greatly benefit from some better baselines, e.g., a comparison to TraceCompiler or another one of the related works you cite.

In 4b, do the replays by Suffice count as part of the history budget?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.