acceptodds
Under review as a conference paper at ICLR 2027

Explore, Execute, Evolve: A Skill Acquisition and Reuse Loop for Embodied Agents

Abstract

Vision-language-action and world-action models have demonstrated impressive capabilities in robotics, yet generalization to unseen tasks remains challenging. More recently, general-purpose multimodal agents have shown great potential for zero-shot robotic task solving. However, they often incur high execution costs by reasoning and exploring the physical world from scratch. To reduce these costs, we introduce RoboSkill, a framework that connects skill acquisition and reuse through an Explore, Execute, Evolve loop. Within this loop, the agent explores to gather task-relevant information, executes tasks while adapting to feedback, and evolves its skill library based on execution records. It then reuses these skills to guide exploration and execution in the next cycle, closing the loop. To improve loop efficiency, we complement vision with tactile feedback to reduce uncertainty during physical interaction. We further augment textual guidance with reusable code to reduce reasoning overhead during skill reuse. On LIBERO-10, RoboSkill improves first-episode success rates by 12.5–25.0 percentage points and reduces average runtime by 7.6–72.4% across four agents. On real robots, it improves success rates by 8.3 percentage points and reduces average runtime for successful trials by at least 14.4%.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied Agent, skills, Manipulation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_ZfzaOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents RoboSkill, a framework that structures embodied agent learning through an "Explore–Execute–Evolve" loop. Agents explore their environments to overcome issues, execute actions using visual and tactile feedback, and convert their experience into structured, reusable skills with textual guidance, code, metadata, and failure cases. These skills are then retrieved and adapted for downstream tasks. RoboSkill is evaluated on both the LIBERO-10 benchmark and physical robots, and it demonstrates strong capabilities in skill reuse, cross-task and cross-agent transfer, and continuous skill refinement.

Critical strengths
  1. The algorithm is sound:

    • The "Explore, Execute, Evolve" loop is clean and well-motivated for skill acquisition and reuse.
    • The use of force/torque signals from tactile feedback complements vision, and reusable code (vs. text-only instructions) reduces the reasoning burden during reuse.
    • They demonstrate that skills are agent-agnostic to some extent (Section 4.4).
  2. Empirical results are convincing:

    • The evaluation is very broad (i.e. 4 agents, 10 LIBERO-10 tasks, cross-task transfer, cross-agent transfer, 12 real-robot tasks, significant ablations...).
    • The main LIBERO results are very strong, both in terms of success rate and average time.
    • Even if not as strong as simulation results, the real-robot evaluation is also valuable, as agents improve success rate with respect to the baseline.
Critical weaknesses
  1. The "Method" section could be improved for clarity:
    • As far as I understood, how sub-goals are selected is not specified.
    • It is not explained how the agent detects it requires additional information. Failure seems to be the trigger, but the mechanism of how the agent decides what addition information requires is not defined.
    • The definition of skill should probably be earlier in the paper to make it easier to understand. Until Section 3.4 there is not such definition, but you speak about skills already in the abstract.
    • It is not clear to me how skills are built. The paper should specify more clearly what is the process for skill extraction. Is it through prompting? If so, what prompts are being used?
    • No example of what a skill looks like. Would be useful to see one given it is a central part of the algorithm.
    • Section 3.3. has a grammar error: "Throughout this paper, tactile feedback broadly including pressure and force/torque signals". This sentence should be fixed.

2- One of the main motivations for this work is that agents waste time and physical interaction by repeatedly solving tasks from scratch. However, as far as I understand, the results do not include the time for acquiring the skills, it measures the time it takes the agent when the skills are available (Appendix B). This should be emphazised in the main text (or, alternatively, the reported time in the tables should capture as well the time it takes to obtain the skills).

Questions for authors
  • How does the agent detect that it requires additional knowledge? And how does it identify unresolved issues?
  • According to the Section 3.2, the agent explores until it "resolves the exploration subgoal or reveals a need for further investigation". Does that mean that if, while solving one sub-task it detects it requires additional information, it abandons exploration for the initial sub-task and explores to resolve the new additional issue? Does it ever go back to the original sub-task? Is the last issue always prioritized? This should probably be clarified in the paper.
  • How are conflicting skills resolved?
  • What is the latency of executing code + API calls? Is this acceptable for real-time control?
6 comments
AuthorsOpenReview ↗

Response to W1.1, W1.2 & Q1, Q2: Agent-Directed Exploration and Nested Subgoals

Thank you for these questions. Our framework combines prompting with a state-machine-based harness to support agent-directed exploration. Before each model call, the harness adds a brief reminder to the input context, without retaining it in the conversation history:

Main goal: <goal task>
Current goal: <current task>
If you cannot resolve the current task, you may create a subtask
to acquire the information needed to proceed.

The harness tracks both goals and treats the agent as exploring when the current goal differs from the main goal.

Subgoal selection and information needs (W1.1, W1.2; Q1). The agent identifies missing information and selects exploration subgoals from execution feedback, without a separate detector or predefined selection rules. Exploration does not require a preceding failure; the agent may also identify missing information before attempting an action.

Nested exploration (Q2). The same agent can temporarily shift to a new subgoal, which takes priority while the previous goal remains pending. Once resolved, the agent resumes the previous goal with the findings retained in context. Deeper nesting is supported but was not observed in our experiments. We will clarify these mechanisms in Section 3.2.

Example: placing two moka pots on the stove.

Main task: grasp and place the pots
    │ Grasp unsuccessful:
    │ “The finger-offset model is inconsistent.”
    ▼
Exploration subgoal: determine the fingertip offset
    │ Agent chooses to touch the pot’s knob with the closed gripper
    │ → Executes contact probe (steps 248–268)
    │ → Updates the gripper geometry estimate
    ▼
Resume main task with the updated estimate
      “Grasp confirmed — the pot is lifted …
       Now transporting to the stove.”

This trace illustrates the agent identifying an information gap, choosing a diagnostic subgoal, and returning to the original task.

AuthorsOpenReview ↗

Response to W1.4: Skill Extraction and Prompts

Thank you for the question. Skill extraction is prompt-based. After successful task execution, the harness continues the same agent session, with further robot actions disabled. The agent reviews its conversation, executed code, feedback logs, and saved observations to produce:

  1. Textual guidance: successful strategies, failures, corrections, and supporting evidence.
  2. Reusable code: a hierarchical program index and modules copied verbatim from code executed during the run.
  3. Supporting records: action records, keyframes, and scene-specific parameters with instructions for recomputing them in a new scene.

The extraction prompt includes the following instructions (translated excerpts; omissions marked):

Keep the current session and workspace. Do not call step() or reset().
Review only the actual execution of your own task/seed ...
Summarize the successful path, important failures, and causal corrections ...
Separate scene-specific values from methods for recomputing them ...
Extract a reusable hierarchical program from code you actually executed.
Copy the corresponding source files verbatim ...

The harness checks the package structure, recorded-frame references, and code-file consistency. Failed checks are returned to the agent for repair before export. We will clarify this procedure and provide the full extraction prompt in the revised manuscript.

AuthorsOpenReview ↗

Response W1.5: A Concrete RoboSkill Example

Thank you for the suggestion. We will include a concrete example. The following is a small placement subskill within the RoboSkill package acquired from “turn on the stove and put the moka pot on it.” It lowers the grasped pot onto the burner and detects contact from stalled vertical motion.

Textual guidance (translated; excerpted):

After reaching the estimated placement height, descend gradually with a small command magnitude of 0.05–0.08. Use stalled vertical motion to determine whether the pot has seated on the support surface.

Executable code (original logic; logging omitted and explanatory comments added):

zs = [float(c.pos[2])]                 # Record the initial end-effector height.
for i in range(25):                    # Limit descent to at most 25 actions.
    c.step([0, 0, -0.06, 0, 0, 0, 1]) # Move downward slightly while keeping the gripper closed.
    zs.append(float(c.pos[2]))         # Read and store the updated height after the action.
    if len(zs) > 6 and abs(zs[-1] - zs[-6]) < 0.0012: # Detect less than 1.2 mm of vertical progress.
        break                         # Stop descending when the motion stalls.

Supporting records and parameters. The full package contains 12 action records and 11 keyframes, linking the guidance and code to the source execution. It also records scene-specific parameters, including the burner height (0.9303 m) and end-effector-to-pot-base offset (0.1506 m), together with instructions for remeasuring them in a new scene.

The text explains the operation, the code implements the action–feedback loop, and the supporting records preserve execution evidence and guidance for adapting scene-dependent parameters.

Reviewer_crerOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RoboSkill stores textual guidance, code, and execution records in reusable experience packages. A general-purpose coding agent selects packages, adapts them into programs that control the robot through an SDK, and updates the library through session-end reflection. Evaluation covers LIBERO-10, 12 physical-robot tasks, cross-task and cross-agent reuse, and ablations.

Critical strengths
  • Transferring Astra's packages to Sol improves physical-robot success from 4/30 to 21/30 trials across six tasks, demonstrating practical benefits from sharing experience across models.
  • Evaluation spans multiple models, simulation and hardware, with code, selection, and update analyses. Negative outcomes and non-monotonic updates are explicitly reported.
Critical weaknesses
  1. The methodological novelty is insufficient. Explore–Execute–Evolve combines task-directed information gathering, feedback-driven code generation, and session-end reflection. Reusable code libraries and accumulated repair knowledge are already central to CaP-X and ASPIRE, respectively.

    RoboSkill's contribution is an incremental reorganization of these mechanisms into experience packages. The code and update ablations establish that retaining experience is useful. They do not demonstrate a substantive advance in skill learning, and the empirical gains provide insufficient new insight to compensate for the limited methodological novelty.

  2. The claimed skills lack convincing evidence of generalizable control guidance. Language-conditioned controllers, including VLAs, can ground reusable language guidance into scene-dependent actions. This makes the abstraction of experience into transferable guidance a central question. RoboSkill evaluates text/code packages through a coding-agent/SDK stack; even its text-only ablation retains code generation as the execution mechanism.

    The paper does not demonstrate that the same discovered guidance remains effective across diverse scenes without accumulating new task-specific packages. Most evaluations supply a target-task package, and Table 5 shows mixed cross-task transfer. These results establish useful experience reuse within the coding framework, but provide insufficient evidence that skill discovery produces broadly applicable control guidance.

Questions for authors
  1. How does session-end consolidation change skill construction or adaptation relative to the cited code-library and repair-memory approaches, and which experimental finding supports that distinction?
  2. Which discovered skill provides reusable guidance across substantially different scenes? Please show the same guidance, how the executor grounds it into different actions, and whether additional task-specific packages are required.
2 comments
AuthorsOpenReview ↗

Response to Weakness 1 & Question 1: Novelty and the Role of Session-End Consolidation

Thank you for the comment. We agree that session-end consolidation is shared with ASPIRE, and we do not claim its timing as a methodological distinction. Our focus is how the resulting experience supports an embodied agent in autonomously completing a task. All CaP comparisons below use a baseline adapted from ASPIRE’s open-source approach, with task-specific programs developed and then frozen for evaluation.

AspectASPIRE-based CaP (frozen program)RoboSkill used by an online agent
PreparationDevelop and test a task-specific program before evaluation.No separate target-task program-development phase is required.
Retained artifactA complete task program with implemented decision and recovery rules.Control code, execution evidence, decision rationales, and parameter-recomputation guidance.
Execution decisionsThe program selects subsequent actions through implemented branches.The agent interprets feedback and decides subsequent actions.
Scene adaptationRecompute quantities using implemented procedures.The agent uses stored guidance to select and perform measurements for the current scene.

Concrete example: placing two moka pots on a stove. We inspected artifacts generated with the same model, Astra:

ASPIRE-based CaP (frozen program)Online agent and its saved RoboSkill
Placement decisionIf descent fails to reach the target and position error exceeds 10 mm, return failure; otherwise release.After descent exhausted its local budget, the agent interpreted motion and force changes as support-contact evidence, released the pot, and visually verified placement.
Where the decision residesAn explicit conditional branch in the task program.Agent judgment during execution, subsequently retained as guidance alongside control code and evidence.

The saved guidance states:

Exhausting a local motion budget does not necessarily indicate failure; use contact evidence and measured pose to decide whether to transition from position control to release.

This judgment was not implemented as an automatic contact-based release rule in the saved control module. It was preserved for an agent to interpret and apply in subsequent execution.

We also compared this ASPIRE-based CaP baseline with a plain online agent on original LIBERO layouts and modified layouts of the same tasks using LIBERO-Plus:

MethodOriginal LIBERO (50 trials)Modified layouts (50 trials)Success-rate drop
ASPIRE-based CaP32/50 (64%)20/50 (40%)24 percentage points
Plain agent50/50 (100%)44/50 (88%)12 percentage points

These results support the value of retaining the agent in the execution loop. The example illustrates what RoboSkill preserves to support that agent: executable tools together with evidence and guidance for deciding how to use them.

AuthorsOpenReview ↗

Response to Weakness 2 & Question 2: Cross-Task Reuse of RoboSkill Guidance and Code

Existing evaluation. We already evaluate cross-task reuse in Table 5, where the agent receives a RoboSkill package acquired from a different task. Below, we show how the transferred guidance and code affect execution in a concrete case.

Task. Transfer a RoboSkill package from “turn on the stove and place one moka pot” to “place two moka pots on an already-on stove.”

Transferred RoboSkill contents (translated and abbreviated):

  • Grasping guidance: use the lid knob; remeasure its center in the current scene; verify the grasp using the gripper opening.
  • Recorded calibration: the fingertips are approximately 8 mm below the end-effector reference point.
  • Reusable code: geom.py and ctrl.py provide perception and feedback-control utilities.
Key decisionWithout RoboSkillWith RoboSkill
Where to grasp?Grasps the pot body, rotating the gripper by 22.5° to align with its faces.Grasps the lid knob, following the source guidance, and verifies the grip.
How to determine grasp height?Incorrect fingertip offset: 99.4 mm → empty grasp → recalibration to ≈7 mm.Reuses the 8 mm source calibration and remeasures the current knob position.
How to execute?Builds perception and control code during the run.Reuses geom.py and ctrl.py unchanged, adding code for two placement slots.

Cost on the same target task and seed: 808 steps without RoboSkill → 414 with RoboSkill, a reduction of 394 steps (48.8%). Both runs succeed within one episode.

Textual guidance informs the grasping strategy; reusable code supplies perception and control implementations. Scene-specific positions and placement slots are adapted without adding a target-task package. This comparison measures the combined package, not text alone.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.