acceptodds
Under review as a conference paper at ICLR 2027

TRACE: Task-conditioned Retrieval and Aggregation with Token-Budgeted Compilation for Executable Skill Programs in LLM Agents

Abstract

Skills provide a scalable abstraction for LLM agents by encapsulating reusable procedures, tool affordances, implementation details, validation logic, and failure-handling strategies. However, effectively leveraging large-scale skill repositories remains challenging: existing retrieval methods mainly optimize individual skill relevance, while existing compression methods often reduce context without preserving the execution structure required for multi-skill tasks. We propose TRACE, a task-conditioned skill retrieval, aggregation, and compilation framework that bridges the gap between skill retrieval and reliable execution. TRACE first constructs complementary skill bundles through requirement-conditioned graph routing with dynamic edge activation and then performs token-budgeted compilation to transform retrieved skills into compact executable programs containing dependencies, operations, validators, and fallback policies. For failed executions, TRACE further applies trace-guided repair to patch task-specific executable programs using execution traces and failure feedback without modifying the persistent skill library. Experiments on 51 multi-skill tasks show that TRACE achieves 92.16% execution success with only 2.18K estimated skill-input tokens on average, reducing skill-input cost by approximately 90% compared with raw top-20 skill injection. Moreover, trace-guided repair recovers two of four failed executions and improves the final success rate to 96.08%. Ablation studies further demonstrate the importance of role-balanced selection, validation information, and fallback guidance for reliable skill execution.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, LLM Agents, Agentic AI, Skill Learning, Retrieval-Augmented Generation, Program Synthesis, Tool Use

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.25 over 4 reviews, above 2% of the others.
Reviewer_gtnKOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes TRACE, a framework for multi-skill LLM agents that integrates requirement-conditioned graph retrieval, token-budgeted skill compilation, and trace-guided repair. It retrieves complementary skill bundles and transforms them into structured executable programs containing dependencies, operations, validators, and fallback policies. Experiments on 51 multi-skill tasks show that TRACE achieves 92.16% execution success with 2.18K estimated skill-input tokens on average, while trace-guided repair further improves the success rate to 96.08%.

Critical strengths

The paper addresses a practical challenge in multi-skill LLM agents by considering retrieval quality, context efficiency, and execution reliability within a unified framework. The structured compilation approach explicitly preserves execution-critical information, including dependencies, validation conditions, and fallback strategies. TRACE substantially reduces estimated skill-input tokens compared with raw skill injection while maintaining competitive execution performance. The ablation studies further demonstrate the importance of role-balanced selection, validators, and fallback guidance.

Critical weaknesses

The paper addresses well-known challenges in retrieval-augmented agents, including complementary skill retrieval, context compression, and execution-feedback-based refinement. However, the proposed framework appears to primarily combine existing techniques rather than introduce fundamentally new mechanisms. The paper does not clearly establish its technical novelty over straightforward combinations of graph-based retrieval, structured prompting, and iterative repair.

The key mechanisms of TRACE are described at a high level, without sufficient algorithmic details. In particular, it remains unclear how requirement-conditioned edge activation, evidence aggregation, and token-budgeted fragment selection are concretely performed. Without clearer formulations or implementation procedures, it is difficult to identify the specific technical contributions and assess their effectiveness independently.

The proposed framework critically relies on an offline skill graph and role-structured evidence extraction. However, the paper does not sufficiently explain how skill relations are constructed, how evidence fragments are assigned to functional roles, or how these structures can be generated for a new skill repository. Consequently, it is unclear whether the reported improvements stem from the proposed methods or from the quality of these intermediate structures, raising concerns about reproducibility and scalability.

The downstream execution evaluation is limited to 51 multi-skill tasks within a single coding-agent framework using one backbone model, with insufficient characterization of task diversity or variation across execution environments. Moreover, the paper does not report repeated trials, variance estimates, or statistical significance tests. Given the limited evaluation scope and the small success-rate difference between TRACErepair and raw top-20, it is difficult to assess the robustness and generalizability of the reported improvements.

Questions for authors

Please see the weaknesses.

Reviewer_fWxROpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes TRACE, a framework combining skill retrieval, compilation, and trace-guided repair for LLM agents. Experiments on 51 multi-skill tasks demonstrate reduced estimated token consumption while maintaining competitive execution success rates.

Critical strengths
  1. The paper addresses a practical problem of efficient skill retrieval and execution for LLM agents.
  2. The proposed framework significantly reduces estimated input token consumption while maintaining competitive task performance.
Critical weaknesses
  1. Incomplete module-level ablations. Although the paper provides ablations on validators, fallbacks, and role buckets, it lacks systematic ablations studies.
  2. Limited differentiation from existing methods. Stronger comparisons with the realted work are needed.
  3. Limited experimental validation. The experiments involve only 51 tasks, and TRACE with repair succeeds on just one additional task compared with raw top-20 injection.
  4. Presentation issues. Some figures contain very small text, and the tables exhibit inconsistent formatting and font sizes.
Questions for authors
  1. Could the authors provide more detailed ablations to clarify the contribution of each major module?
  2. What are the key methodological differences between TRACE and existing skill retrieval and compilation methods?
  3. Could the authors evaluate task success using functional correctness checks and a larger task?
  4. Could the authors improve the readability and consistency of the figures and tables?
Reviewer_FoFaOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces TRACE, a task conditioned skill retrieval and compilation method that retrieves task related skills based on a skill relationship graph and compiles those skills into one condensed skill .When execution fails, TRACE further repairs the program using execution traces. Experiments show that the proposed method achieves a success rate comparable to injecting all retrieved skills with a lower token cost.

Critical strengths

This paper targets a clear gap between skill retrieval and execution, and uses a task aware compression method to compress multiple dependent skills into one structured execution program. The method also incorporates execution results to repair the compressed program when execution fails.

Critical weaknesses
  1. Lack of critical details. For example, the paper does not explain where those constraints come from. Based on the example from figure 2, the user’s instruction is very high level. I am more curious about how this method derived the constraints based on these instructions. Several other components are also described only at a high level, including how the skill graph and its relation types are constructed. The construction of the evaluation set is also unclear, such as how the 51 tasks were selected, the size of the skill library, and the definition of the Easy and Hard tasks.

  2. The evaluation does not fully support the main claims. This paper uses different rubrics to measure task success. A task is marked as successful when the agent process exits with code 0 and no predefined failure markers appear, but the original SkillsBench uses deterministic verifiers to measure the task success, so it is unclear whether the tasks are actually solved. Trace guided repair also gives TRACE a second attempt with additional task success signals, while the baselines only get one attempt. In addition, the token accounting only covers the skill context given to the agent, so the cost of constructing and of the agent’s execution is not mentioned in the paper.

Questions for authors

See the critical weakness part. I think all weaknesses mentioned should be answered very carefully.

Reviewer_t9afOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents TRACE, a framework for retrieving and compiling skills for LLM agents. Given a task, TRACE uses requirement-conditioned graph routing to retrieve a bundle of skills, then compiles their contents into a compact execution plan. The method reduces estimated skill-input tokens by approximately 90% relative to raw top-20 skill injection, while achieving a similar initial execution success rate.

Critical strengths

● The paper addresses a practical problem: retrieving individually relevant skills does not necessarily produce a complete workflow, and compressing skill descriptions can remove information needed for execution. ● TRACE uses approximately 2.18K estimated skill-input tokens versus 21.81K for raw skill injection.

Critical weaknesses
  1. The contribution of graph retrieval is not convincingly demonstrated. Average Recall@20 improves from 0.645 for SkillRouter to 0.651 for TRACE, a gain of only 0.6%, without uncertainty estimates.
  2. The evaluation is too limited to support a robust performance advantage. There are only 51 tasks and one reported backend model, with no repeated-run analysis. Before repair, TRACE succeeds on one fewer task than raw injection; after repair, it succeeds on one more.
  3. Tokens are estimated from character counts, and the scope of the reported “total tokens” is not sufficiently clear.
Questions for authors
  1. With the compiler and execution settings fixed, how does downstream success differ between SkillRouter retrieval and TRACE retrieval?
  2. Does TRACE retain an advantage when all baselines receive the same retry budget and execution feedback?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.