acceptodds
Under review as a conference paper at ICLR 2027

Embodied Skill Bench: Benchmarking How Well Agents with Embodied Skills Perform on Physical Tasks

Abstract

LLM agents are increasingly used as high-level controllers for robots, drawing on a growing body of VLA policies, perception models, and procedural know- how from earlier robot runs. These modules are hard for agents to reuse because they differ in interfaces, embodiment assumptions, and conditions of use, and how much they actually help on physical tasks has not been measured. In this paper, we first package them as embodied skills, which extend agent skills by making embodiment assumptions and conditions of use explicit, and build a repository of 250 action, perception, and experience skills from 1,052 existing resources. On this basis we introduce EMBODIEDSKILLBENCH, which runs an agent in a closed observation-action loop and records success, actions, tokens, and wall-clock time. It supports three modes: native control, skill-only execution, and Agent-Directed Reuse (ADR), a mode we propose in which the agent keeps its native controls and decides for each skill whether to invoke, modify, or ignore it. Across four agents on 100 perturbed LIBERO-Pro tasks, we find that skills help an agent only as far as they exceed what it can already do. Skill-only execution lifts GPT-5.6- Sol from 11% to 63% and Kimi-K3 and Claude-Opus-5 by about 20 points, while cutting their token usage by 58% to 85%. Yet all four agents then end up between 61% and 64%, and GPT-6-Astra falls from 92% under native control to 61%. This suggests that as agents grow stronger, the role of embodied skills shifts from supplying missing capability to reducing cost: GPT-6-Astra already solves most tasks through native control, so restricting it to skills removes more than it adds. Under ADR, which lets it keep native control and call skills selectively, GPT-6-Astra recovers to 85%, within 7 points of native control, while using 14.1% fewer tokens and 11.9% less time. With Jev as a skill choosing motion directions and gripper actions, it reaches 81% with 50% fewer tokens and 57% less time.

Primary area:
datasets and benchmarks
Keywords:
Embodied AI, Large Language Model Agents, Agentic Systems, Robot Manipulation, Benchmarking

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_njV3OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces EmbodiedSkillBench, a benchmark for studying how reusable embodied skills affect LLM agents on manipulation tasks. The authors define a common skill format and build a repository of 250 skills from 1,052 existing resources, including action, perception, and experience skills. The benchmark evaluates native control, skill-based execution, and Agent-Directed Reuse (ADR) on 100 perturbed LIBERO-Pro tasks. The main finding is that skills are most useful when they provide capabilities that the base agent lacks. For weaker agents, they substantially improve task success and reduce execution cost. For a stronger agent, however, prescribed skill workflows can hurt performance, and skills are more useful as optional tools for delegation. I see the main contribution as an empirical study of the marginal utility of reusable embodied capabilities across agents with different levels of native competence.

Critical strengths
  1. The paper brings a broad set of heterogeneous embodied resources into a common format and provides a benchmark for studying how agents use them. Beyond the size of the repository itself, this gives a useful common setting for comparing skill selection, composition, and reuse across agents.

  2. The most interesting result is that skills are not uniformly helpful. They fill capability gaps for weaker agents, but can constrain a stronger agent when used through a prescribed workflow. The GPT-6-Astra experiments suggest a useful shift from capability augmentation toward selective, efficiency-oriented delegation as base agents become stronger.

Critical weaknesses
  1. The main results lack uncertainty estimates. The experiments appear to use a single rollout/seed per task and configuration (Clean100, seed 26), without repeated runs or confidence intervals. Given the stochasticity of LLM agents and learned robot policies, it is difficult to judge the robustness of some reported gaps, especially smaller differences such as 85% versus 92%. Multiple seeds or other uncertainty estimates would make the conclusions more convincing.

  2. The experiments do not isolate the value of the embodied-skill abstraction itself. This is my main concern. In the primary comparisons, the skill-enabled agent often receives additional VLA controllers, perception modules, and procedural guidance, while the no-skill baseline mainly uses native low-level robot control. As a result, the experiments clearly show that additional embodied capabilities can help, but they do not show how much of the gain comes from the proposed skill representation itself. A more controlled comparison would keep the underlying modules fixed and expose them either as raw tools/APIs or through the proposed skill format. This would help separate the value of the underlying VLA/perception modules from the value of skill packaging, instructions, and reuse mechanisms.

Questions for authors

Please see weaknesses.

4 comments
AuthorsOpenReview ↗

Same-tool baseline

W2: The experiments do not isolate the value of the embodied-skill abstraction itself. A controlled comparison should keep the underlying modules fixed and expose them as raw tools/APIs or through the proposed skill format, separating module gains from skill packaging, instructions, and reuse mechanisms.

We add a controlled same-tool, no-skill comparison on LIBERO-Pro Clean100 for gpt-5.6-sol, Kimi-K3, and claude-opus-5, finding that skill guidance improves overall success and reduces action, token, and time costs for all three models (Table 1). In the added baseline, agents access the same underlying models and tools through their APIs as in the with-skill setting, allowing us to evaluate the additional value of the embodied-skill abstraction and its procedural guidance while holding tool availability fixed. We rename the previous no-skill setting native control to distinguish it from this new baseline. Table 1 summarizes the evaluation across the four task families and overall, and results vary by model and task family.

Table 1. Added “no skill” experiments on LIBERO-Pro Clean100 for gpt-5.6-sol, kimi-k3, and opus-5. SR is in percent, Tok. in millions, and Time in minutes.

ModelSkillL10 SRAct.Tok.TimeGoal SRAct.Tok.TimeObject SRAct.Tok.TimeSpatial SRAct.Tok.TimeOverall SRAct.Tok.Time
gpt-5.6-solnative control0.030.046.00411.1114.321.074.71010.824.228.005.69212.8925.025.885.11513.4311.026.045.35412.01
no skill16.733.925.23138.1157.121.823.03121.2370.814.212.18816.0354.216.542.19816.3250.021.633.15722.85
with skill41.74.831.8468.7967.91.610.4552.9279.21.250.4422.8162.51.960.5283.3263.02.380.8034.40
Kimi-K3native control4.221.253.47169.6346.415.542.54548.4175.016.422.17630.2145.818.923.13252.4443.017.932.81950.10
no skill12.522.293.18764.1357.113.862.07839.7179.212.361.26522.0350.020.782.31436.8850.017.192.20640.65
with skill41.75.211.54119.3060.75.681.34817.2075.01.210.3354.9870.85.171.49019.6562.04.371.18515.36
claude-opus-5native control8.39.213.35625.1346.47.252.29520.0370.811.753.19618.0550.09.712.52616.7444.09.392.82119.99
no skill20.88.063.27220.8953.66.431.84115.3283.312.832.30513.1237.512.921.90412.1849.09.912.31115.38
with skill41.75.291.24912.4660.74.250.5366.5483.31.210.1922.5470.83.040.6677.7164.03.480.6567.28
AuthorsOpenReview ↗

Repeated runs and uncertainty

W1: The main results lack uncertainty estimates and appear to use a single rollout/seed per task and configuration. Given LLM and policy stochasticity, repeated runs or confidence intervals are needed to assess robustness, especially for gaps such as 85% versus 92%.

We extend the GPT-6 Astra evaluation on LIBERO-Pro Clean100 across three random seeds. Table 2 reports the mean and sample standard deviation for native control, same-tool no skill, with skill, and agent-directed reuse (ADR). Native control achieves the highest mean success rate, while the with-skill setting uses fewer actions, tokens, and agent minutes. ADR recovers much of the success lost under the with-skill setting.

Table 2. GPT-6 Astra on LIBERO-Pro Clean100 across three random seeds. Entries report mean ± sample standard deviation across seeds. SR is in percent, Tok. in millions, and Time in minutes. Act. counts state-changing calls.

Overall

SettingSR (%)Act.Tok. (M)Time (min)
With skill58.0±2.62.33±0.130.510±0.0632.26±1.00
No skill85.7±1.29.36±0.860.861±0.1714.05±1.14
Native control89.3±2.516.71±0.331.975±0.3069.58±9.47
Agent-directed reuse87.7±2.513.74±0.760.880±0.9477.88±8.84

L10

SettingSR (%)Act.Tok. (M)Time (min)
With skill31.9±4.83.11±0.380.810±0.1703.29±1.52
No skill72.2±2.416.31±1.781.708±0.4907.35±2.84
Native control72.2±2.426.50±0.893.690±0.83415.44±15.92
Agent-directed reuse61.1±2.425.17±3.141.617±1.75113.58±14.90

Goal

SettingSR (%)Act.Tok. (M)Time (min)
With skill59.5±7.41.94±0.220.318±0.0321.70±0.79
No skill86.9±2.18.33±0.720.743±0.1553.91±1.07
Native control96.4±3.613.12±1.461.380±0.1187.05±6.26
Agent-directed reuse94.0±4.110.80±1.030.768±0.8176.85±7.71

Object

SettingSR (%)Act.Tok. (M)Time (min)
With skill80.6±2.41.32±0.020.334±0.0141.81±0.80
No skill95.8±4.24.69±2.060.363±0.2061.83±0.69
Native control94.4±4.814.06±1.171.387±0.1977.40±6.83
Agent-directed reuse98.6±2.46.97±2.330.281±0.1662.73±1.77

Spatial

SettingSR (%)Act.Tok. (M)Time (min)
With skill59.7±2.43.00±0.570.609±0.1292.32±0.93
No skill87.5±4.28.26±0.500.652±0.0703.12±0.34
Native control93.1±2.413.75±0.521.542±0.3508.86±9.43
Agent-directed reuse95.8±4.212.53±2.410.874±1.0778.53±11.19

We extend the no-skill/with-skill comparison to RoboCasa365 and to RoboDojo for gpt-5.6-sol. Skill guidance improves overall success on both benchmarks, with gains varying by task category. RoboCasa365 shows gains on Atomic Seen and Composite Seen, while Composite Unseen remains unchanged. RoboDojo shows gains on Open and Memory and decreases on Generalization and Long Horizon. These experiments provide evidence for transfer across simulation environments.

Table 3. With skill vs. no skill for gpt-5.6-sol/high, evaluated across three random seeds on RoboCasa365 and three random layouts on RoboDojo. Entries report mean ± sample standard deviation across seeds or layouts. SR is in percent, Tok. in millions, and Time in minutes. Act. counts agent-level actions.

BenchmarkTask categoryNo skill SRAct.Tok.TimeWith skill SRAct.Tok.Time
RoboCasa365Atomic Seen59.3±11.62.41±0.320.291±0.0922.24±0.0972.2±19.23.26±1.510.405±0.1392.17±0.53
Composite Seen12.5±12.510.92±0.952.926±1.03111.47±3.3722.9±3.69.60±1.161.483±0.1619.65±2.27
Composite Unseen10.4±9.511.21±0.583.088±1.17112.33±4.3810.4±3.610.54±1.301.842±0.42611.28±2.62
Overall28.7±9.97.95±0.502.029±0.7348.42±2.4836.7±4.67.62±0.331.210±0.1227.48±1.45
RoboDojoGeneralization19.4±4.83.81±2.211.864±0.67112.10±1.0113.9±12.77.44±0.820.242±0.01812.81±1.10
Open8.3±7.29.13±2.491.926±0.59311.66±1.1525.0±0.06.83±0.800.215±0.0329.55±1.56
Precision0.0±0.02.83±2.131.472±0.48310.08±1.420.0±0.08.25±0.780.284±0.04611.23±0.58
Long Horizon12.5±12.57.13±3.032.118±0.86615.51±1.498.3±7.27.79±1.090.235±0.01714.23±1.83
Memory0.0±0.013.39±4.993.634±1.06415.83±1.5211.1±9.66.67±1.200.180±0.0329.98±1.60
Overall9.5±2.46.63±1.772.102±0.50712.81±1.1011.9±4.87.44±0.570.235±0.01411.75±0.44

Dear Reviewer,

I hope this message finds you well. As the discussion period is nearing its end with less than two days remaining, I wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are of great importance to us, and we’re eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

Thank you for the detailed rebuttal and additional experiments. The new same-tool baseline and multi-seed and cross-environment evaluations address most of my concerns. I appreciate the authors' efforts and have decided to update my assessment to weak accept.

Reviewer_TWEnOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper highlights that, given the heterogeneity of embodied modules, current embodied benchmarks lack a unified framework for reusable skills in agent control. The authors therefore propose EmbodiedSkillBench to evaluate agents using reusable embodied modules through a unified framework that supports planning, low-level control, and feedback. Multiple agents and configurations are evaluated.

Critical strengths
  1. This paper points out a critical drawback in current robotic benchmarks for agents concerning the heterogeneous interfaces and usage assumptions of embodied modules, which make their reuse and composition challenging.

  2. Various experiments and analyses are conducted on the proposed benchmark across multiple agents and skill configurations, showing both the benefits of skill reuse in task performance and efficiency and its potential to constrain stronger agents.

Critical weaknesses
  1. Although the authors conduct comprehensive experiments on LIBERO-Pro, all evaluations are performed within a single simulation benchmark and robot embodiment, without testing transfer to other simulators or real robots. Given the substantial differences among simulators and the sim-to-real gap, these experiments do not establish whether the proposed skill abstraction supports reuse across simulators or embodiments, or instead relies heavily on a fixed simulation setting. The authors should demonstrate that the proposed approach works in real-robot settings or can be transferred to a broader range of embodiments or simulators.

  2. This paper does not distinguish between the gains from external tools and those from the skill abstraction. When agents are provided with skills, they receive not only skill documentation but also access to external resources, including VLA and perception tools. The authors should conduct experiments in which agents have access to the exact same tools through standard APIs without wrapping them as embodied skills.

Questions for authors
  1. As mentioned in Weakness 1, can the proposed abstraction and skill structure, as well as the existing embodied skills, be readily transferred to real robots or other simulation environments?

  2. Given the finding that Astra is constrained by skill use, under what conditions should an agent rely on embodied skills rather than native control to better manage the trade-off between task success and execution efficiency?

4 comments
AuthorsOpenReview ↗

Same-tool baseline

W2: The paper does not distinguish gains from external tools from those of the skill abstraction. Agents should have access to the exact same tools through standard APIs without wrapping them as embodied skills.

We add a controlled same-tool, no-skill comparison on LIBERO-Pro Clean100 for gpt-5.6-sol, Kimi-K3, and claude-opus-5, finding that skill guidance improves overall success and reduces action, token, and time costs for all three models (Table 1). In the added baseline, agents access the same underlying models and tools through their APIs as in the with-skill setting, allowing us to evaluate the additional value of the embodied-skill abstraction and its procedural guidance while holding tool availability fixed. We rename the previous no-skill setting native control to distinguish it from this new baseline. Table 1 summarizes the evaluation across the four task families and overall, and results vary by model and task family.

Table 1. Added “no skill” experiments on LIBERO-Pro Clean100 for gpt-5.6-sol, kimi-k3, and opus-5. SR is in percent, Tok. in millions, and Time in minutes.

ModelSkillL10 SRAct.Tok.TimeGoal SRAct.Tok.TimeObject SRAct.Tok.TimeSpatial SRAct.Tok.TimeOverall SRAct.Tok.Time
gpt-5.6-solnative control0.030.046.00411.1114.321.074.71010.824.228.005.69212.8925.025.885.11513.4311.026.045.35412.01
no skill16.733.925.23138.1157.121.823.03121.2370.814.212.18816.0354.216.542.19816.3250.021.633.15722.85
with skill41.74.831.8468.7967.91.610.4552.9279.21.250.4422.8162.51.960.5283.3263.02.380.8034.40
Kimi-K3native control4.221.253.47169.6346.415.542.54548.4175.016.422.17630.2145.818.923.13252.4443.017.932.81950.10
no skill12.522.293.18764.1357.113.862.07839.7179.212.361.26522.0350.020.782.31436.8850.017.192.20640.65
with skill41.75.211.54119.3060.75.681.34817.2075.01.210.3354.9870.85.171.49019.6562.04.371.18515.36
claude-opus-5native control8.39.213.35625.1346.47.252.29520.0370.811.753.19618.0550.09.712.52616.7444.09.392.82119.99
no skill20.88.063.27220.8953.66.431.84115.3283.312.832.30513.1237.512.921.90412.1849.09.912.31115.38
with skill41.75.291.24912.4660.74.250.5366.5483.31.210.1922.5470.83.040.6677.7164.03.480.6567.28
AuthorsOpenReview ↗

Transfer and skill delegation

W1 / Q1: Evaluations cover a single simulation benchmark and robot embodiment. Can the proposed abstraction and skill structure, as well as the existing embodied skills, be readily transferred to real robots or other simulators or embodiments?

We extend the no-skill/with-skill comparison to RoboCasa365 and to RoboDojo for gpt-5.6-sol. Skill guidance improves overall success on both benchmarks, with gains varying by task category. RoboCasa365 shows gains on Atomic Seen and Composite Seen, while Composite Unseen remains unchanged. RoboDojo shows gains on Open and Memory and decreases on Generalization and Long Horizon. These experiments provide evidence for transfer across simulation environments.

Table 2. With skill vs. no skill for gpt-5.6-sol/high, evaluated across three random seeds on RoboCasa365 and three random layouts on RoboDojo. Entries report mean ± sample standard deviation across seeds or layouts. SR is in percent, Tok. in millions, and Time in minutes. Act. counts agent-level actions.

BenchmarkTask categoryNo skill SRAct.Tok.TimeWith skill SRAct.Tok.Time
RoboCasa365Atomic Seen59.3±11.62.41±0.320.291±0.0922.24±0.0972.2±19.23.26±1.510.405±0.1392.17±0.53
Composite Seen12.5±12.510.92±0.952.926±1.03111.47±3.3722.9±3.69.60±1.161.483±0.1619.65±2.27
Composite Unseen10.4±9.511.21±0.583.088±1.17112.33±4.3810.4±3.610.54±1.301.842±0.42611.28±2.62
Overall28.7±9.97.95±0.502.029±0.7348.42±2.4836.7±4.67.62±0.331.210±0.1227.48±1.45
RoboDojoGeneralization19.4±4.83.81±2.211.864±0.67112.10±1.0113.9±12.77.44±0.820.242±0.01812.81±1.10
Open8.3±7.29.13±2.491.926±0.59311.66±1.1525.0±0.06.83±0.800.215±0.0329.55±1.56
Precision0.0±0.02.83±2.131.472±0.48310.08±1.420.0±0.08.25±0.780.284±0.04611.23±0.58
Long Horizon12.5±12.57.13±3.032.118±0.86615.51±1.498.3±7.27.79±1.090.235±0.01714.23±1.83
Memory0.0±0.013.39±4.993.634±1.06415.83±1.5211.1±9.66.67±1.200.180±0.0329.98±1.60
Overall9.5±2.46.63±1.772.102±0.50712.81±1.1011.9±4.87.44±0.570.235±0.01411.75±0.44

Q2: Given that Astra is constrained by skill use, under what conditions should an agent rely on embodied skills rather than native control to manage the trade-off between task success and execution efficiency?

An agent should use embodied skills for precise manipulations that it struggles to execute through native control but that a VLA policy trained on similar motions can perform more accurately and efficiently, and for time-sensitive substeps that require rapid feedback. Our LIBERO-Pro moka-pot case illustrates the first condition: the agent repeatedly approached and closed the gripper using native control without retaining the pots, whereas the skill-based execution completed the transfers. This observation motivates delegating delicate grasp-and-lift phases to policies trained on similar interactions. Task scenarios requiring precise manipulation include plugging a charger into a power strip in RoboDojo and manipulating stove knobs in RoboCasa365. In familiar precision-demanding manipulations, embodied skills can outperform native control while also improving execution efficiency.

Embodied skills are also useful when the physical state changes rapidly and delayed reactions can compromise execution. RoboDojo provides task scenarios such as picking a moving object from a conveyor and pouring liquid from a bottle into a cup. In physical deployments of these scenarios, the robot must intercept the object before it passes out of reach, or promptly adjust the bottle's tilt as liquid flows to avoid spilling. Local policy inference can provide faster, more timely control. Delegating these substeps is therefore particularly attractive when repeated agent inference would delay the required response.

The agent should retain responsibility for semantic grounding, task decomposition, memory, spatial reasoning, and progress verification. For example, identifying the conveyor object specified by an image or deciding which pot remains to be moved. It should delegate a clearly specified motor subtask, inspect the resulting state, and revise the plan or use native control when the policy selects the wrong object or encounters an unfamiliar geometry.

Dear Reviewer,

I hope this message finds you well. As the discussion period is nearing its end with less than two days remaining, I wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are of great importance to us, and we’re eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

I appreciate the authors' efforts during the rebuttal period and the additional experiments. The same-tool baselines and cross-environment evaluations substantially address my concerns. Therefore, I raise my score to 3.

Reviewer_hpvpOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents EmbodiedSkillBench, a framework for designing skills and using them in agent execution of physical tasks to improve performance. It begins by describing the need for skills, the designed evaluation (the benchmark itself being a key contribution), and the skill framework. It then goes through results that use the benchmark to demonstrate the value of skills in context.

Critical strengths

Strengths:

  • The paper is quite well-written. It's clear and every part is explained and signposted well
  • Experimental structure, e.g. research questions, are valuable and well executed
  • The paper has been done well, it just doesn't seem like a new idea
Critical weaknesses

Main weakness: this paper does not seem novel. The tasks are from other benchmarks and the skill concept and extraction method are extensively documented in prior work, even using LLMs. The results from recent models may be unique but I don't think this warrants a conference publication. The related work does attempt to position the paper against prior work, but neither comparison of different agents nor linking of skills to task outcomes and execution effort is novel. The former is again just a matter of what experiments were run, and the latter is true of any framework where actions have explicit prerequisites and effects. The question of whether skills help as context is important but is also studied in [1] and works like it; executing the benchmarking of such ability is useful but not novel.

Secondary weakness: it's not clear that the benchmark is a worthwhile contribution even aside from novelty. It's a collection of existing tasks and a standard agent loop, which is the right setup for benchmarking skills but not a novel setup.

[1] "One demo is all it takes: planning domain derivation from LLMs from a single demonstration". Jinbang Huang, Yixin Xiao, Zhanguang Zhang, Mark Coates, Jianye Hao, Yingxue Zhang. ICLR 2026.

Questions for authors

To differentiate from other embodied skill frameworks, the related work section says "these resources differ in their interfaces and usage assumptions, making their combination a separate challenge". Could you say more about the actual differences beyond minor formatting? What challenges of combination does this work uniquely solve?

5 comments

W1: The paper does not seem novel: tasks come from other benchmarks, and the skill concept and extraction method are extensively documented in prior work. Neither comparing agents, linking skills to task outcomes and execution effort, nor studying whether skills help as context is novel.

We agree that reusable skills and the use of skill-related knowledge as context to support agent planning and execution have been explored in prior work. Our contribution is an evaluation framework and empirical study of how general-purpose agents select, combine, and execute heterogeneous embodied resources.

Specifically, our added same-tool, no-skill baseline (Table 1) distinguishes access to embodied modules from the additional value of skill-based guidance, while comparisons with agent-directed reuse examine the consequences of prescribing workflows versus allowing agents to control resource use. PDDLLM [1] demonstrates how knowledge derived from demonstrations can support robotic planning. EmbodiedSkillBench goes beyond asking whether skill knowledge is useful as context to examine the broader question of when agents should rely on embodied resources and how they should adjust their execution strategies in response to feedback.

Our comparisons show that skill guidance can improve success and reduce execution effort even when agents have access to the same underlying tools, while prescribed workflows can constrain agents that are capable of adapting their own execution strategy. Together with the case studies, these findings motivate selective delegation: using embodied skills for familiar, precision-demanding and time-sensitive motor substeps while retaining agent control over semantic grounding, spatial understanding,memorizing, planning, and recovery. This provides an empirically grounded explanation of how embodied skills and agent autonomy interact in physical tasks.

Table 1. Added “no skill” experiments on LIBERO-Pro Clean100 for gpt-5.6-sol, kimi-k3, and opus-5. SR is in percent, Tok. in millions, and Time in minutes.

ModelSkillL10 SRAct.Tok.TimeGoal SRAct.Tok.TimeObject SRAct.Tok.TimeSpatial SRAct.Tok.TimeOverall SRAct.Tok.Time
gpt-5.6-solnative control0.030.046.00411.1114.321.074.71010.824.228.005.69212.8925.025.885.11513.4311.026.045.35412.01
no skill16.733.925.23138.1157.121.823.03121.2370.814.212.18816.0354.216.542.19816.3250.021.633.15722.85
with skill41.74.831.8468.7967.91.610.4552.9279.21.250.4422.8162.51.960.5283.3263.02.380.8034.40
Kimi-K3native control4.221.253.47169.6346.415.542.54548.4175.016.422.17630.2145.818.923.13252.4443.017.932.81950.10
no skill12.522.293.18764.1357.113.862.07839.7179.212.361.26522.0350.020.782.31436.8850.017.192.20640.65
with skill41.75.211.54119.3060.75.681.34817.2075.01.210.3354.9870.85.171.49019.6562.04.371.18515.36
claude-opus-5native control8.39.213.35625.1346.47.252.29520.0370.811.753.19618.0550.09.712.52616.7444.09.392.82119.99
no skill20.88.063.27220.8953.66.431.84115.3283.312.832.30513.1237.512.921.90412.1849.09.912.31115.38
with skill41.75.291.24912.4660.74.250.5366.5483.31.210.1922.5470.83.040.6677.7164.03.480.6567.28

[1] Jinbang Huang, Yixin Xiao, Zhanguang Zhang, Mark Coates, Jianye Hao, and Yingxue Zhang. One Demo Is All It Takes: Planning Domain Derivation with LLMs from A Single Demonstration. ICLR, 2026.

AuthorsOpenReview ↗

Benchmark contribution

W2: It is unclear whether the benchmark is a worthwhile contribution even aside from novelty: it combines existing tasks and a standard agent loop, which is appropriate for benchmarking skills but not a novel setup.

We agree that the underlying tasks and observe-act loop are established. We retain them deliberately so that task objectives and success criteria remain independent of our resource packaging. EmbodiedSkillBench is designed to make how agents use heterogeneous embodied resources systematically observable and comparable.

To this end, EmbodiedSkillBench brings action, perception, and experience resources into a shared evaluation protocol, recording resource invocations and feedback alongside official task success and execution effort. This allows researchers to inspect how agents combine resources and respond to execution feedback. For example, how an agent continues after an unmet grasp condition. On fixed tasks, comparisons among native control, access to the same external tools without skill guidance, skill-guided execution, and agent-directed reuse further distinguish the roles of resource availability, procedural guidance, and control allocation.

The additional RoboCasa and RoboDojo evaluations (Table 2) extend this analysis beyond the original environment, while repeated-seed evaluations assess result stability.

Table 2. With skill vs. no skill for gpt-5.6-sol/high, evaluated across three random seeds on RoboCasa365 and three random layouts on RoboDojo. Entries report mean ± sample standard deviation across seeds or layouts. SR is in percent, Tok. in millions, and Time in minutes. Act. counts agent-level actions.

BenchmarkTask categoryNo skill SRAct.Tok.TimeWith skill SRAct.Tok.Time
RoboCasa365Atomic Seen59.3±11.62.41±0.320.291±0.0922.24±0.0972.2±19.23.26±1.510.405±0.1392.17±0.53
Composite Seen12.5±12.510.92±0.952.926±1.03111.47±3.3722.9±3.69.60±1.161.483±0.1619.65±2.27
Composite Unseen10.4±9.511.21±0.583.088±1.17112.33±4.3810.4±3.610.54±1.301.842±0.42611.28±2.62
Overall28.7±9.97.95±0.502.029±0.7348.42±2.4836.7±4.67.62±0.331.210±0.1227.48±1.45
RoboDojoGeneralization19.4±4.83.81±2.211.864±0.67112.10±1.0113.9±12.77.44±0.820.242±0.01812.81±1.10
Open8.3±7.29.13±2.491.926±0.59311.66±1.1525.0±0.06.83±0.800.215±0.0329.55±1.56
Precision0.0±0.02.83±2.131.472±0.48310.08±1.420.0±0.08.25±0.780.284±0.04611.23±0.58
Long Horizon12.5±12.57.13±3.032.118±0.86615.51±1.498.3±7.27.79±1.090.235±0.01714.23±1.83
Memory0.0±0.013.39±4.993.634±1.06415.83±1.5211.1±9.66.67±1.200.180±0.0329.98±1.60
Overall9.5±2.46.63±1.772.102±0.50712.81±1.1011.9±4.87.44±0.570.235±0.01411.75±0.44
AuthorsOpenReview ↗

Resource interfaces and composition

Q1: What are the actual differences in resource interfaces and usage assumptions beyond minor formatting? What challenges of combination does this work uniquely solve?

Embodied resources differ in their capability boundaries, operational roles, and execution requirements. Combining them is difficult because agents may struggle to infer which modules can reliably handle the current task, how modules with different roles should cooperate, and whether one module's outcome satisfies the next module's requirements. Effective composition therefore requires capability-aware selection, role coordination, and verification of execution handoffs.

First, an agent may have limited knowledge of a module's capability boundaries. For a post-trained policy, a task description alone does not reveal which objects, geometries, or manipulations it can handle reliably. With multiple modules available, choosing an appropriate tool for the current physical conditions becomes difficult, even when several tools appear relevant to the same subgoal.

Second, resources operate at different execution granularities and leave different physical states. A learned controller may execute a bounded sequence of action chunks, whereas native control may perform a local motion or gripper operation. Their composition requires checking whether the resulting state supports the next operation—for example, whether an object is actually held before beginning transport.

Third, resources have different operational roles. A perception module provides evidence about the scene, an action module changes the physical state, and an experience resource provides procedural guidance. Combining them requires identifying which resources provide executable capabilities and which supply information or instructions for using those capabilities.

EmbodiedSkillBench makes these requirements explicit in its skill packages and interaction protocol. We consolidate documented capabilities, embodiment assumptions, and conditions of use from source papers, model-use documentation, and experience cards into agent-readable skill descriptions and invocation guidance. For execution handoffs, interface declarations retain coordinate conventions, input requirements, and execution limits, while the shared observation-action protocol returns fresh observations and feedback after each action. Verification and recovery instructions guide the agent in checking the resulting state before selecting the next operation. Action, perception, and experience resources are also explicitly typed; experience skills identify the executable action and perception dependencies required to carry out their procedures. Together, these mechanisms give agents concrete information for resource selection and coordination and make their use measurable under a common evaluation setup.

Our experiments motivate selective combination: delegate familiar, precision-demanding and time-sensitive motor substeps to embodied policies while retaining agent control over semantic grounding, planning, and feedback-driven adjustments. Native control remains useful when the agent needs to correct or adapt execution beyond a policy's effective behavior.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary and Appreciation

Dear all,

We sincerely thank everyone for their time, thoughtful feedback, and support throughout the review and discussion process. Your input has helped us strengthen the experimental evidence and clarify the contributions of EmbodiedSkillBench.

During the rebuttal, we added controlled same-tool, no-skill comparisons to distinguish the value of skill guidance from access to embodied resources, extended our evaluations to RoboCasa365 and RoboDojo, and conducted repeated runs with uncertainty estimates to assess result stability. We also clarified the novelty and scope of our contribution, the challenges of combining heterogeneous embodied resources, and the conditions under which skill guidance can benefit agents.

We are grateful for the reviewers’ positive feedback on our additional experiments and clarifications, and for their acknowledgment of our efforts to address their concerns. We especially appreciate their time in revisiting our responses and engaging in the discussion.

Thank you again for your careful consideration and the time devoted to reviewing and discussing our work. We are grateful for the opportunity to improve the paper through this process.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.