- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
TaReD addresses long-horizon tool-using tasks in settings where exposing the entire tool library is impractical. It organizes tools into a hierarchy of capabilities, discovers relevant capabilities on demand, and uses this information to guide recursive task decomposition. Detailed tool contracts are bound when an execution stage is ready, and references to intermediate results support dependencies between subtasks. The hierarchy contains 137 category nodes and 457 tool leaves. Evaluation on an AppWorld Test-C run set compares TaReD with ADAPT, ReAct, and Plan-and-Solve, alongside tool-awareness ablations and context-exposure analyses. TaReD achieves 69.3% TGC, compared with 51.3% for ReAct and 58.7% for an expandable flat tool subset.
Critical strengths
-
The paper identifies a useful connection between tool discovery and decomposition granularity. Capability information can help determine executable subtask boundaries rather than merely retrieve tools after a plan has already been formed. This makes the approach practically relevant to agents operating over large API libraries.
-
The expandable flat-subset ablation is a meaningful control beyond fixed one-shot retrieval. TaReD's 10.6-percentage-point TGC advantage over this condition suggests that the complete hierarchical configuration offers value beyond simply restoring access to omitted tools.
Critical weaknesses
-
The evidence does not isolate capability-aligned recursion from other changes in the tool interface. Hierarchical nodes provide curated summaries, input/output hints, preconditions, and cross-branch information. Table 2 does not compare against flat access to the same capability-card content, nor does it test non-recursive execution with the same hierarchy. The results therefore support the combined system but do not yet establish that hierarchical guidance improves recursive decomposition specifically.
-
The reported experiment and runtime policy are not specified precisely enough to assess reproducibility and resource comparability. The "adopted run set" lacks explicit task and scenario denominators, while the accounting of discovery and recursive operations within the 100-step and 600-second limits is unclear. Key decisions such as Sufficient, Direct, and ReviewOrRevise remain underspecified. These omissions affect the interpretation of both success rates and the claimed context-efficiency benefits.
Questions for authors
-
Clarification of the comparisons. Do the flat and hierarchical conditions receive equivalent semantic information, or do hierarchical capability cards add planning guidance unavailable to the flat conditions? Can you provide existing evidence separating hierarchical tool discovery from recursive execution, or explicitly limit the attribution to their combined effect?
-
Clarification of the evaluation and implementation. What are the exact task and scenario counts and selection rules for the adopted run set? How are tool discovery, model calls, and recursive operations charged to the step and time budgets? Please also provide the actual decision protocols or prompts for Sufficient, Direct, and ReviewOrRevise, with a real evaluation trace.
3 comments
Evaluation Protocol, Decision Mechanisms
2. Evaluation tasks and cost accounting
W2: “The "adopted run set" lacks explicit task and scenario denominators, while the accounting of discovery and recursive operations within the 100-step and 600-second limits is unclear.”
Q2 (tasks and budgets): “What are the exact task and scenario counts and selection rules for the adopted run set? How are tool discovery, model calls, and recursive operations charged to the step and time budgets?”
We selected 50 scenarios from AppWorld Test-C, with three official variants per scenario, yielding 150 task instances. Appendix E, Table 9, lists all task IDs grouped by scenario. TGC uses all 150 instances, while SGC uses all 50 scenarios and requires success on every variant.
Appendix D summarizes mean per-task costs over all adopted runs, including failures and timeouts. D.1, Table 7 and Figure 4, reports task success alongside model calls, API attempts, and duration; D.2, Table 8, reports cumulative model tokens. The metrics use the following definitions:
- LLM calls: logical model requests for tool selection, planning, execution, and review, excluding transport retries.
- API calls: native application API attempts, including authentication, completion, and failed attempts; documentation lookups are excluded.
- Time: recorded elapsed task-execution time in seconds, including model inference and tool execution.
- Input/Output tokens: cumulative provider-reported tokens supplied to/generated by the model across a task's calls, reported in thousands. Repeated context counts on every request; Total is their sum.
The statistics cover the main baselines, tool-awareness ablations, and supplementary conditions, including CUGA, CUGA-Tree, and both TaReD tool trees. They show that TaReD gains completion at additional inference cost relative to simpler systems, while combining higher completion with shorter recorded durations than the CUGA systems. Its token cost is concentrated on the input side. These task-level statistics complement Figure 3's average tool-context tokens per saved decision.
3. Actual decision protocols and a real trace
W2 (decision functions): “Key decisions such as Sufficient, Direct, and ReviewOrRevise remain underspecified.”
Q2 (protocols and trace): “Please also provide the actual decision protocols or prompts for Sufficient, Direct, and ReviewOrRevise, with a real evaluation trace.”
TaReD uses a shared decision loop at the root and each child task (Appendix B.1). Given the goal, constraints, dependencies, discovered capabilities, and execution evidence, the model selects the next action. The controller checks and applies it, then returns feedback. The following judgments operate within this loop; Appendix B.2 provides their detailed protocols.
- Sufficient: after discovery, determine whether capability information supports a plan with identifiable functional parts, outputs, and dependencies. Planning proceeds when these are clear; relevant gaps trigger further discovery. Exact tool schemas can be retrieved later.
- Direct: once information is sufficient, choose between executing a known sequence of related calls and computations as one stage, or delegating cohesive parts that need separate discovery or judgment to dependent child tasks. Missing schemas alone do not require decomposition.
- ReviewOrRevise: after a child failure, a review request, or pending work with no ready child, compare returned evidence with the remaining goal and dependencies. Keep a suitable plan or revise its unfinished work, reusing completed results. Failed obligations require explicit links to completed replacements before closure. Return the updated plan and continue, or stop as insufficient if no supported continuation exists.
Appendix B.3 follows adopted Test-C task eb5ad85_1, asking for year-to-date Amazon spending. Over 19 model decisions, TaReD separates order collection from aggregation, passes the collected records by reference, repairs a failed calculation that used the wrong amount field, and resolves the failed child before answering. The result passes both official checks. This trace connects the decision protocols to actual discovery, delegation, data handoff, and recovery.
We have prepared a code repository containing TaReD's detailed implementation. To preserve anonymity during review, we are not including the repository link here. If the paper is accepted, we will provide the link in the final version.
Thank you again for your thoughtful feedback and the effort you have devoted to reviewing our work. We welcome the opportunity to discuss any remaining questions. If you find that our responses and additional results have addressed your concerns, we would be grateful if you could consider a more favorable assessment of our work.
Dear Reviewer,
Thank you again for your time and thoughtful feedback. We would like to follow up on our response and ask whether you have any remaining questions or concerns. We welcome further discussion and would be glad to provide any additional clarification.
If our response and additional results have addressed your concerns, we would be grateful if you could consider a more favorable score in light of this evidence.
Thank you for your consideration.
Component Attribution
Dear Reviewer,
Thank you for recognizing TaReD's effectiveness and for your detailed and thoughtful review. We have carefully considered your feedback and clarified the information supplied to each condition, the scope of the performance attribution, and the exact evaluation accounting. Appendix C.2 adds transfer evidence, while Appendix B provides the decision protocols and a real execution trace. We address each point below.
1. Tool information and attribution of the gains
TaReD couples the tool-capability hierarchy with recursive task execution. Starting at the root node of the tool hierarchy, the agent discovers progressively finer functional categories through intermediate nodes. Their capability summaries can establish useful functional stages and dependencies before the agent inspects individual tool leaves. The agent can therefore decompose a goal into functional stages, then refine each subtask through further discovery and recursive execution, retrieving exact tool contracts when execution requires them. This coupling is a central design principle of TaReD: intermediate capability semantics guide the structure and granularity of recursive tasks, instead of requiring task boundaries to be inferred solely from individual APIs (Section 4; Appendices A.2–A.3 and B.2).
Both hierarchical and flat representations are built from the same tool library; shared exposed APIs have identical descriptions and execution contracts. The hierarchy adds intermediate nodes with functional summaries that explicitly support planning at different levels of abstraction. These category-level summaries are part of the design for guiding recursive decomposition. Appending the same summaries to an unordered flat context would retain their text, but would not by itself reproduce hierarchical navigation and its coupling to recursive task decomposition. Exposing all summaries at once could also increase context load and introduce irrelevant planning cues.
In the tool-awareness ablations (Table 2), we compare TaReD with full tool injection and an expandable tool subset under the same recursive execution framework. Full tool injection presents short descriptions of all tools as a flat list and selectively retrieves detailed tool information on demand. The expandable tool subset starts with detailed information for a predicted subset of relevant tools and allows missing tools to be discovered and added during execution. Like TaReD, both conditions can access descriptions and execution contracts across the entire tool library; they differ in how that information is organized and discovered. These results support the combined design of hierarchical representation, capability guidance, and discovery within recursive execution.
As discussed above, TaReD is designed to realize the full benefit of its tool hierarchy through recursive execution guided by hierarchical tool discovery. Nevertheless, in response to your request, we also evaluated the hierarchy in a non-recursive execution framework. Appendix C.2, Table 4, provides complementary transfer evidence. CUGA uses a fixed hierarchy of application-level planning, API selection through an LLM-based shortlister, and code planning and execution (Appendix C.1). CUGA-Tree retains CUGA's fixed, non-recursive execution framework and replaces its flat tool shortlister with navigation through the capability tree within each selected application. Within CUGA's fixed, non-recursive framework, the transferred design improves all three measures, extending the utility of hierarchical tool discovery beyond TaReD. The transfer results are summarized below: