acceptodds
Under review as a conference paper at ICLR 2027

From Skill Text to Skill Structure: The Scheduling-Structural-Logical Representation for Agent Skills

Abstract

Large language model (LLM) agents increasingly rely on reusable skills: capability packages that combine instructions, control flow, constraints, and tool calls. In current agentic systems, however, skills are still represented as text, mainly SKILL.md-style documents whose machine-usable evidence remains embedded largely in natural-language descriptions. As a result, skill-centered agentic systems face a representation problem: both managing skill collections and using skills during agent execution require reasoning over invocation interfaces, execution structure, and concrete side effects, but these signals are often entangled in a single textual surface. An explicit representation of skill knowledge may therefore help make these artifacts easier for machines to acquire and leverage. Drawing on Memory Organization Packets, Script Theory, and Conceptual Dependency from Schank and Abelson's classical work on cognitive linguistic representation, we introduce what is, to our knowledge, the first structured representation for agent skill artifacts that disentangles skill-level scheduling signals, scene-level execution structure, and logic-level action/resource-use evidence: the Scheduling-Structural-Logical (SSL) representation. We instantiate SSL with an LLM-based normalizer and evaluate it along two aspects: Skill Routing before a task, and Skill Execution during it. In Skill Routing, indexing candidate skills by their SSL views improves nDCG@10 over the raw view ( pp hard,  pp easy; 6.6% and 6.9% relative); In Skill Execution, an action card that annotates the task's source passages with SSL relations improves verified task success over raw execution ( pp on DeepSeek-V4.1-Flash,  pp on GLM-5.3-Flash; 6.8% and 13.8% relative) at no additional model calls. Together, the two aspects show that an explicit, source-grounded SSL representation helps both when structure selects what the agent sees and when it governs how the agent acts. We position SSL as a practical step toward more inspectable, reusable, and operationally actionable skill representations, and as a building block for future skill registries and agent runtimes.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Skill, Structure, Agent, Representation

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_r5USOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Taking some concepts from cognitive linguistic representation, the paper introduces the first structured representation for agent skill artifacts that disentangles skill-level scheduling signals, scene-level execution structure, and logic-level action/resource-use evidence. They call it Scheduling-Structural-Logical (SSL) representation.

In SSL, a skill artifact is converted by a normalizer into three layers: a scheduling record for skill-level signals, a structural graph of execution scenes, and a logical graph of atomic actions and resource evidence. It is basically a harness engineering effort to breakdown a single skill artefact into a more detailed representation.

Critical strengths

The paper's approach is inspired by some cognitive science work, which has been validated to some extent from previous literature.

Critical weaknesses
  1. The representation of the skills in 3 layers (a scheduling record for skill-level signals, a structural graph of execution scenes, and a logical graph of atomic actions and resource evidence) uses complex phrases like "scenes", "transition" which is not commonly found in previous harness work. I am not sure if it can be used off-the-shelf for any LLM on the market, or we require some further fine-tuning. Furthermore, I do not know if there is any practical value to implement such heuristic-driven changes to how skills are crafted. Perhaps, the bitter lesson has taught us that as models become better, we might not need to devote so much work in this area.
Questions for authors
  1. What is the performance difference if applied different ablation changes to the SSL representation? in Table 12, I see "The endpoint was chosen post hoc and all cells are exploratory. " What does this mean?
Reviewer_nDJ2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Scheduling-Structural-Logical (SSL), a three-layer representation that transforms natural-language agent skill documents into structured information about skill invocation, execution workflows, and atomic actions. Experiments on SkillRouter and SkillsBench show that SSL improves skill retrieval and task execution compared to raw skill representations.

Critical strengths

The paper addresses a practical problem in skill-based LLM agent systems: how to effectively retrieve and utilize skills as skill repositories grow larger. The proposed structured representation is potentially useful for both skill routing and execution.

Critical weaknesses
  • The SSL representation is generated using a single LLM normalizer, DeepSeek-V4.1-Flash. Since normalization is a central component of the proposed framework, it would be useful to evaluate whether different normalizer models produce consistent structures and comparable downstream performance.
  • The random and shuffled annotation basically provide very wrong information, weak results in this case are not very informative due to the noise introduced. Stronger competitive baselines would help isolate the contribution of the specific SSL design.
  • The routing improvements are concentrated at small retrieval windows and largely disappear at a window size of 100. Although the paper acknowledges this limitation, the underlying reasons remain unclear. Further analysis would help explain why SSL does not consistently outperform raw skill representations as the retrieval budget increases. Especially because this paper partially focuses on improving skill retrieval from a large pool.
Questions for authors

How often are the proposed SSL fields, such as expected inputs, outputs, action dependencies, and execution phases, explicitly specified in the original skill documents? It would be helpful to report field-level coverage statistics. Additionally, to what extent would an LLM naturally recover similar information without being constrained by the proposed SSL schema?

Reviewer_MJQxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SSL, a three-layer representation of agent skill documents, motivated by MOPs, Script Theory, and Conceptual Dependency

  • An LLM normalizer converts skill text into invocation records, scene-level workflows, and action/resource facts.
  • SSL is used for candidate retrieval and execution-time action cards. Experiments on SkillRouter and SkillsBench report better routing at small candidate windows and higher mean verified execution reward.
Critical strengths

I see practical value in a reusable, source-linked schema and in evaluating its downstream use rather than only its format.

  • Routing holds the encoder and raw-text reranker fixed, making the shortlist comparison interpretable.
  • Execution is assessed with executable tests on two agent models.
  • The paper reports unsuccessful delivery variants and explicitly acknowledges that the Structural layer remains unevaluated.
Critical weaknesses

The reported gains are promising. What they do not yet show is that SSL’s three-layer representation is the thing producing them, as opposed to clearer instructions and selective presentation of the same material.

  • The contribution is not sufficiently distinguished from better skill writing. The execution study does not include a same-model rewrite that clarifies requirements, separates examples from mandatory steps, and makes verification explicit. Such a baseline would address the stated failure modes without introducing a separate representation. Routing does include prose condensations (summ_long, summ_short) as controls, so the gap is specifically on the execution side. The cognitive theories are acknowledged as design analogies, which is reasonable, but the experiments do not identify a benefit specific to the proposed decomposition. The evidence supports improving skill presentation more clearly than it supports a distinct representation-level contribution.

  • Section 3 describes three layers, but the organizing principle is less clear than the names suggest. Scheduling separates information by function, i.e. when and how to invoke the skill. Structural and Logical instead separate procedural content by granularity, phases versus actions. Appendix A.4 gives a splitting rule for logic steps (split when the source supports a change in action type, resource boundary, effect, or control-flow outcome); no comparable rule exists for scenes. A scene is described as a coherent execution phase with its own goal, data contract, and exit conditions, which leaves the boundary to the normalizer’s judgment. When should verification be its own scene, and when is it a step inside another one? Table 8 also caps a record at 1–4 scenes and 1–6 logic steps in total, so in practice the “graph” is a short list, and the scene level is usually a single partition of a handful of steps.

    What the structural layer buys is not shown either. The authors acknowledge that its intended consumers are not evaluated. The one place it enters an experiment is Table 12, where appending the scene–logic relations or the per-action facts to the scheduling card pushes the gold skills down on both encoders. Where the layer was tested, it hurt. I kept looking for a result in which preserving scene boundaries and transitions changes an outcome, and could not find one.

    The execution experiment cannot supply that result, because of how the card is built. SSL represents phase- and action-level transitions, whereas the card in Section 4.2.1 exposes local bindings among input, instrument, and result for each passage. If the transition edges were altered while the selected passages and action attributes stayed fixed, would the agent receive a different prompt? If not, a correct workflow and an incorrect one produce the same card and the same reward. The hard validator in Section 3.3 does not close this gap: it checks that transition targets name an in-scope node, not that the source supports the transition. Two procedures could each be grounded in the source and still be linked in the wrong order, and neither valid records nor improved execution would expose it.

  • The execution comparison does not cleanly isolate the effect of annotations. Section 4.2.1 states that card_flat and card_ssl contain exactly the same passages. Table 13 says otherwise: the quote cap per entry is 220 characters for card_ssl and 900 for card_flat, so the two cards differ by construction, with a 15% length-parity tolerance on top. Appendix I.2 shows what this means for one task. Only the SSL card’s quoted span stops before a misleading Python 3.11 example, and on DeepSeek that is the difference between reward 0.0 and 1.0. Removing an example is a separate intervention from adding relational annotations. Matching total card length does not control for which instructions the agent sees, and a per-entry cap four times larger on the flat side does not make for a clean control. Until passage selection and truncation are held constant, the improvement cannot be attributed to SSL relations.

Questions for authors
  1. What was held constant in the execution comparison? Table 13 gives card_ssl a 220-character quote cap and card_flat a 900-character cap. Across the 76 tasks, how many card_flat entries exceed 220 characters, and how many card pairs differ in quoted spans or truncation boundaries? Does the gain survive when passages, ordering, and verification reminders are identical and only the annotation lines differ?
  2. What decides a scene boundary? A.4 states when a logic step is split; what is the equivalent rule for scenes? When the same skill admits several reasonable decompositions, which one is chosen? Is there any task in either experiment where preserving scene boundaries and transitions changes the outcome?
  3. Section 4.2.1 says normalization uses the source skill files alone. Is passage selection for the card also independent of the task instruction? Same skill, two different tasks: same card, or does the highlighted content move?
  4. How consistent is normalization under equivalent descriptions? Have the authors run repeated extraction, or a small paired audit on meaning-preserving paraphrases, to check whether the same act_type, resource_scope, and branching conditions come back?
  5. What exactly is reduced by the claimed 11.3× factor? The ratio matches average input tokens (1,941 versus 172 per candidate), but Section 5.3 calls it the embedding index. If each skill keeps one fixed-dimensional vector, shorter input text does not change the number or size of the stored vectors. Clarifying which quantity was measured would make the efficiency claim easier to assess.
Reviewer_4cNLOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the representation bottleneck in LLM-based agentic systems, where reusable skills are typically stored as unstructured, text-heavy documents (e.g., SKILL.md) that entangle invocation interfaces, execution workflows, and atomic resource operations. To disentangle these signals, the authors introduce the Scheduling-Structural-Logical (SSL) representation, a three-layer JSON graph schema inspired by classical cognitive linguistic theories (Memory Organization Packets, Script Theory, and Conceptual Dependency). The SSL representation is instantiated via an LLM-based normalizer and evaluated across two critical agentic phases: Skill Routing and Skill Execution. Experiments on the SkillRouter and SkillsBench benchmarks demonstrate that leveraging the SSL scheduling layer improves short-list retrieval precision (nDCG@10), while injecting source-grounded SSL logical annotations into execution prompts significantly boosts verified task success rates without increasing model call overhead.

Critical strengths

The primary strength of this work lies in its strong theoretical grounding and highly practical, zero-overhead empirical gains. By drawing on Schank and Abelson’s classical cognitive linguistic frameworks, the authors provide a principled, human-interpretable justification for disentangling skill artifacts into scheduling, structural, and logical layers, rather than relying on ad-hoc schema design. Furthermore, the evaluation is rigorous and economically compelling; the authors conduct thorough length-matched and field-ablation control experiments to prove that the routing gains stem from structural information rather than mere token compression, and the execution experiments demonstrate that SSL-annotated "action cards" significantly improve agent success rates on SkillsBench while actually reducing the average number of model calls, proving the representation's immediate utility for real-world agent runtimes.

Critical weaknesses

The paper suffers from a few notable limitations regarding the generalizability and dynamic applicability of the SSL representation. First, as the authors acknowledge, SSL is strictly a static extraction from text, meaning it cannot inherently capture dynamic runtime behaviors, conditional resource access, or state-dependent branching that only emerge during execution, potentially limiting its use in highly complex, non-deterministic environments. Second, the entire pipeline relies heavily on the fidelity of an LLM-based normalizer, which introduces risks of hallucination, over-regularization, or omission of critical nuances from obfuscated source texts. Finally, while the Scheduling and Logical layers are thoroughly evaluated for routing and execution respectively, the Structural layer (the scene-level execution graph) is entirely left without downstream consumers in this study, leaving a significant gap in validating the full three-layer hypothesis, and the evaluation is restricted to a single benchmark family for each task, raising questions about cross-domain transferability.

Questions for authors

please see weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.