acceptodds
Under review as a conference paper at ICLR 2027

StateSkill-Agent: An Effective and Efficient Embodied Agent via State-conditioned Skill

Abstract

In this work, we aim to develop an embodied MLLM agent that understands and solves embodied tasks by learning, abstracting, and reusing transferrable and generalizable behavioral patterns. To this end, we propose StateSkill, a new skill representation for embodied agents, which introduces the concept of “finite-state machines” into skill modeling for effective and efficient skill construction and learning. The core idea of StateSkill is to model embodied interaction experiences as a finite-state machine, where a skill is organized around a set of task-progress states, state transitions, and state-conditioned guidance, enabling embodied agents to dynamically adjust their skills as the task progresses while keeping skill contexts compact and stage-relevant. We further develop StateSkill-Evolve, an experience-driven evolution mechanism that iteratively refines StateSkill through retrospective analysis of new successful and failed state-guided trajectories. Upon this, we introduce StateGRPO, which integrates StateSkill evolution and model optimization via state-conditioned GRPO into a unified iterative learning loop, enabling state-conditioned skills and model capabilities to co-evolve and reinforce each other. Extensive experiments demonstrate the effectiveness of our proposed methods and trained model (i.e., StateSkill-Agent) across various embodied tasks.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied Agents, Skill Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_qhmeOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

StateSkill represents reusable robot skills as progress states with language guidance, evidence-based transition gates, and action preferences. An offline constructor extracts skills from trajectories, StateSkill-Evolve revises them using execution feedback and paired validation, and StateGRPO trains the agent using success and state-relative interaction efficiency. Evaluation covers LIBERO-PRO, RoboCasa365, physical manipulation, and controlled component comparisons.

Critical strengths
  • Controlled evidence for execution improvements. With the same underlying knowledge, Table 10 improves success from 47.0% to 54.5% with progress organization and phase-specific guidance, 60.0% with evidence gates, and 63.5% with action priors. These comparisons support concrete design choices beyond gains over an unassisted model.

  • Useful revision and reward results. Localized revision reaches 68.4% versus 65.0% for whole-skill reflection (Table 12). Across ten training repetitions, the progress-relative reward improves success over global efficiency shaping by 1.8 percentage points, with a reported 95% interval of [0.69, 2.91] (Table 11).

Critical weaknesses
  1. The state-representation claims exceed the defined mechanism. Formally, the state variable denotes a task-progress phase; environmental observations and retained execution facts enter the transition gate separately. This is a valid controller definition. However, the introduction extends its significance to representing changing agent and environment states without establishing the corresponding abstraction. The paper should distinguish execution-progress tracking from environmental-state representation and substantiate its contribution at the appropriate level.

    Table 10's A0–A1 comparison combines progress organization with selective guidance; Table 12 does not separate graph changes from guidance, gate, or prior updates. Table 8 provides only a schematic edit. The distinctive contribution of learning or revising the skill structure therefore remains unestablished.

    The novelty claim also needs narrower positioning: CEDAR (August 2026) learns reusable embodied skill automata, while HEXIS (September 2026) combines FSMs, local instructions, and trace-driven graph updates. These contemporaneous works inform positioning; their absence as baselines is not a rejection criterion.

  2. The claimed benefit of co-evolution is not established. Table 4 adds StateGRPO to skill revision, whereas Table 11 holds skills fixed. Neither isolates alternation against one refinement stage followed by policy training under matched budgets. The validated RL contribution is a progress-relative efficiency reward used through trajectory-level advantages; it does not establish the claimed mutually reinforcing effect of repeated skill and policy updates.

Questions for authors
  1. Which actual structural edits were accepted, and what evidence distinguishes their contribution from updating guidance, gates, or priors on a fixed graph?
  2. Do existing results isolate alternation from sequential skill refinement and policy training? If not, please narrow the co-evolution claim accordingly.
2 comments

We thank the reviewer for recognizing the controlled evidence for execution improvements, state-localized revision, and the progress-relative reward. Below, we clarify the state representation and report supplementary evidence on structural edits and co-evolution.

W1 & Q1: State representation, structural revisions, and novelty positioning.

We thank the reviewer for requesting clarification of the state representation and evidence for structural revisions. Your comments have helped us explain the progress representation and structural revisions more clearly and better distinguish StateSkill from related frameworks.

1. What the StateSkill state represents.

The active state qtq_t represents progress within an embodied skill. In Equation (3), environmental evidence et+1e_{t+1} and retained execution facts mtm_t determine how the gate updates this progress and the retained facts. For example, grasp verification identifies the current execution stage, while observations and feedback establish whether the object is held or the grasp was missed, determining the subsequent transition and guidance. We have updated Appendix B.1 to clarify how task-progress states are updated using environmental evidence.

2. Skill revisions and a concrete structural example.

In any Evolve phase, StateSkill-Evolve can revise a skill's state graph and state-conditioned knowledge, including language guidance, transition gates, and action priors. As one example, a revision to a pick-and-place skill added a grasp-verification state and a recovery transition back to VLA grasping, enabling a retry after a verified missed grasp within the attempt budget.

Compared with the pre-revision skill, the illustrated revision improved overall success by 2.3 percentage points on LIBERO-PRO with Qwen3.5-9B. Model parameters and the language guidance, action priors, and gate conditions of unchanged states were held fixed. We have added a before-and-after illustration of this example in Appendix C.5.

To evaluate the additional contribution of structural revisions, we have added a fixed-graph ablation (C1) to Appendix F.4, Table 17. C1 uses the same state-localized diagnosis as the full method (C2), but fixes the initial state set and transition edges while allowing updates to language guidance, action priors, and gate conditions on existing transitions. Both conditions use the same initial skills, model checkpoint, development data, 10-step refinement budget, and validation and final-evaluation protocols, with StateGRPO disabled.

Table 17 (reproduced from the revised manuscript). Skill-revision comparison on LIBERO-PRO with Qwen3.5-9B. Repairs and regressions are paired counts relative to C0; token usage is measured in millions per episode during deployment.

ConditionSuccess (%) ↑Repairs ↑Regressions ↓Tokens/ep. (M) ↓
C0: Fixed skill63.5——0.60
C1: Fixed-graph revision65.332180.56
C2: State-localized revision68.451120.47

The full method (C2) reaches 68.4%, compared with 65.3% for the fixed-graph variant (C1), an improvement of 3.1 percentage points. This comparison supports the additional benefit of allowing structural revisions beyond knowledge updates on a fixed graph.

3. Positioning relative to CEDAR and HEXIS.

CEDAR studies how language-guided embodied agents can satisfy persistent constraints during task execution. To make skill behavior verifiable and composable with constraints, it represents both skills and constraints as automata and combines them into controllers that enforce the learned constraints. HEXIS focuses on improving agents' compliance with skill instructions. Because repeatedly inferring control decisions can lead to omitted or incorrect steps, it compiles skills into extended finite-state machines that separate local skill knowledge from control flow, with revisions validated through static checks and trace replay.

Our work focuses on representing, reusing, and improving behavioral knowledge for embodied MLLM agents. This requires skills to represent evolving physical interactions while keeping guidance relevant to the current execution stage. We therefore organize embodied experience into task-progress states equipped with language guidance, evidence-based transition gates, and trajectory-derived action priors, loading guidance for the active state to reduce redundant context. We further use this shared progress representation to localize Evolve revisions and define StateGRPO's efficiency rewards, allowing skill knowledge and model capabilities to improve together through interaction. We have expanded Appendix G.3 with a discussion of CEDAR and HEXIS and these methodological distinctions.

W2 & Q2: Co-evolution under matched budgets and StateGRPO evidence.

We thank the reviewer for requesting clarification of the co-evolution procedure and its evaluation. Your suggestions have helped us improve the manuscript through a clearer co-evolution protocol and a fuller evaluation of alternating updates and the StateGRPO reward design.

1. Co-evolution procedure and supplementary cycles.

In our framework, a co-evolution cycle consists of StateSkill-Evolve with the model fixed, followed by StateGRPO with the refined skills fixed. Our original main results use one cycle of 10 Evolve steps followed by 100 StateGRPO optimizer updates. The same per-cycle budgets apply to the supplementary alternating schedule. One candidate is generated and evaluated per Evolve step. Only candidates satisfying the validation acceptance criterion are applied.

To examine whether further co-evolution cycles remain beneficial, we conducted two additional cycles, each starting from the model and skills produced by the preceding cycle. Table R1a reports the success rates after each phase.

Table R1a. Success rates across co-evolution cycles on LIBERO-PRO with Qwen3.5-9B (%). Cycle 1 corresponds to the original Table 4; cycles 2 and 3 are supplementary experiments. Cycle gain is measured in percentage points relative to the start of that cycle.

CycleBefore EvolveAfter EvolveAfter StateGRPOCycle gain (pp)
1 (original)63.568.470.9+7.4
2 (supplementary)70.972.373.1+2.3
3 (supplementary)73.173.974.5+1.4

Success increases from 70.9% after the first cycle to 74.5% after the third, an additional 3.6 percentage points. Both skill refinement and model training continue to improve performance, with diminishing gains across cycles. These diminishing gains may partly reflect limitations of the current model, leaving some difficult tasks unresolved despite further refinement and training. We have updated Appendix E.2 with the original schedule and these supplementary results (Table 11).

2. Equal-budget comparison.

To assess the benefit of alternating skill refinement and model training under matched budgets, we compared the two schedules on LIBERO-PRO with Qwen3.5-9B, starting from the same model checkpoint and initial StateSkills. Both schedules use 30 Evolve steps and 300 StateGRPO optimizer updates. The non-alternating baseline performs all 30 Evolve steps with the model fixed, then freezes the resulting skills for all 300 training updates. The alternating schedule uses three cycles, each with 10 Evolve steps followed by 100 training updates.

The non-alternating baseline reaches 68.8% after the 30 Evolve steps and 72.6% after fixed-skill training. Alternating co-evolution reaches 74.5%, an improvement of 1.9 percentage points under matched refinement-step and training-update budgets (Table R1b).

Table R1b. Equal-budget comparison on LIBERO-PRO with Qwen3.5-9B.

Optimization scheduleTotal Evolve stepsTotal training updatesFinal success (%)
One-time refinement → fixed-skill training3030072.6
Alternating skill–model co-evolution3030074.5

We have added this comparison to Appendix E.2, Table 11(b).

3. StateGRPO reward design and evidence.

StateGRPO compares interaction counts within the same progress state across successful trajectories for the same task and environment seed. It combines the resulting state-efficiency reward with task success into a trajectory reward, from which the trajectory-level group-relative advantage is computed. We have clarified this computation in Appendix D.1.

The reward ablation in Appendix F.3, Table 16 uses fixed initial StateSkills, disabled skill evolution, and a shared model initialization.

Table R2. Reward ablation on LIBERO-PRO with Qwen3.5-9B.

Training variantSuccess (%)Interaction turnsTokens/episode (M)
GRPO (task-success reward)67.026.00.56
GRPO + global efficiency reward67.522.90.48
StateGRPO (task success + state-aware efficiency)69.323.20.49

Success rates for GRPO + global efficiency reward and StateGRPO are means over ten training repetitions.

The state-aware reward improves success by 2.3 percentage points over success-only training. Across ten training repetitions, its mean improvement over global efficiency is 1.80 percentage points (95% confidence interval: [0.69, 2.91]), with similar interaction counts (23.2 vs. 22.9) and token usage (0.49M vs. 0.48M) to global efficiency.

We thank the reviewer again for the suggestions, which have improved both the presentation of StateSkill and the evidence supporting its design.

Reviewer_unhvOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces StateSkill, which organizes embodied interaction past experiences into reusable skills. Each skill breaks an operation into stages, with guidance on what to do, which actions to favor, and when to move to the next stage.

  • StateSkill-Evolve refines these skills using both successful and failed attempts.
  • StateGRPO rewards the model for completing tasks and using fewer interactions at each stage.

The system alternates between improving the skills and training the high-level multimodal model, while keeping the low-level VLA unchanged. The paper reports improvements in success rates and execution efficiency on LIBERO-PRO, RoboCasa365, and real-world robot tasks, with the largest gains on Qwen3.5-9B.

Critical strengths
  • StateSkill addresses a practical problem. A robot finishing an action does not necessarily mean it has achieved the intended result. StateSkill makes this distinction explicit and uses transition gates to decide whether to proceed or attempt recovery. The paper also separates local skill completion from overall task success, which is determined by the environment rather than the agent’s own judgment.

  • Small model gains are substantial with lower execution costs. Compared with Harness VLA using the same high-level model, StateSkill improved a lot on LIBERO-PRO an RoboCasa365. Appendix G.2 also reports a reduction in average E2E execution time from 8.3 to 1.7 minutes, including perception, reasoning, execution, and retries.

  • Ablations help explain the improvements. This paper examine the contributions of state-based guidance, transition gates, and action priors. Ten training runs support the benefit of state-based rewards, while reporting both fixes and regressions gives a clearer picture of skill revision.

Critical weaknesses
  • Implementation details are missing. It is unclear how generated skills become executable rules or how much manual work is involved. Missing training settings and validation budgets also limit reproducibility.

  • The generalization claims need clearer task splits. Different test seeds do not mean new tasks. The paper should clarify which tasks were excluded from skill development and how much adaptation the real-world experiments required.

  • It is unclear what additional benefits the repeated co-evolution would bring. This paper needs to be compared with a simpler approach: first improve the skills, then freeze them, and then train the model with the same budget. Without such a comparison, the additional benefits might only come from the model training itself. The experiments also did not show how much improvement came from modifying the state diagram itself rather than adjusting the guidance, gating, or action priors.

Questions for authors
  • Could you clarify how generated skills become executable rules, how much manual work is needed, and the main training settings?

  • Which tasks were excluded from skill development, and what adaptation was needed for the real-world experiments?

  • Does repeated co-evolution outperform improving the skills once and then training the model with the same budget?

4 comments

W1 & Q1: Executable skill implementation, manual involvement, and training settings.

We thank the reviewer for requesting clearer implementation and training details. Your comments have helped us document skill construction, automatic validation, and training settings more clearly, making our experiments easier to reproduce.

1. Executable skills and manual involvement.

GPT-5.5 constructs StateSkills from exploration trajectories. Given the configured action and feedback interfaces, the constructor and offline updater directly generate and revise executable Python gate code, without manual rule translation, skill editing, or per-candidate approval. At runtime, these gates evaluate observations, executor feedback, and retained facts to remain in the current state, follow a transition, or return a local skill outcome (Equation 3). The active state determines the corresponding guidance and action preferences.

Numerical thresholds and retry limits are refined during StateSkill-Evolve and remain fixed during each StateGRPO phase and final evaluation. Candidate revisions are accepted automatically when consistency and behavioral checks pass and full-task success improves on the same validation instances (Appendix C.3). We have updated Appendices A–C with the gate-code generation, runtime execution, parameter refinement, and automatic validation procedures.

2. Training settings and validation budget.

Table R1 summarizes the original LIBERO-PRO settings. Initial skill construction uses 80 trajectories, while Evolve uses 240 trajectories collected from 80 task configurations across seeds 11–13. Candidate validation uses a separate set of 80 task–seed instances, with seed 14. We have updated Appendix C.3 with the evolution and validation settings and Appendix E.3 (Table 12) with the training configuration and experience budgets.

Table R1. Training and skill-evolution settings for the original LIBERO-PRO experiments.

ItemSetting
Training model and frozen componentsQwen3.5-9B planner; VLA executor and SAM3 frozen
Fine-tuning method and target modulesLoRA on all linear projections in the language decoder
LoRA rank / alpha16 / 32
OptimizerAdamW
Learning rate, original cycle1e-5
StateGRPO training environment seeds11, 12, 13
GRPO group size4 independently sampled trajectories for the same task and environment seed
Batch per training update16 groups × 4 trajectories = 64 trajectories
Training update budget100 optimizer updates
Refinement–training cycles1 in the original experiments; 3 in total including the supplementary experiments
Candidate generation1 candidate skill revision generated and evaluated at each refinement step
Evolve stopping criterionCompletion of 10 refinement steps per phase
Candidate-validation set80 task configurations × 1 validation seed (seed 14) = 80 task–seed instances
Experience budget for skill construction80 trajectories
Experience budget for StateSkill-Evolve240 trajectories (80 task configurations × 3 seeds, 11–13)

W2 & Q2: Task-level exclusion and adaptation for real-world deployment.

We thank the reviewer for requesting clearer task splits and real-world adaptation details. Your suggestions have helped us distinguish task-level exclusion from seed separation and describe the real-world deployment protocol more clearly.

1. Task and seed allocation.

Both benchmarks exclude evaluation seeds from all development stages and fix skills and model parameters before evaluation. Seed separation tests new initializations, while RoboCasa365 Composite-Unseen additionally tests tasks excluded throughout development. Tables R2a and R3a list environment seeds, while Tables R2b and R3b separately list the task groups or suites used at each stage. In these tables, the first three stage columns belong to Training, which covers the full development process, including candidate validation; Testing denotes final evaluation only. — denotes exclusion from that stage across all seeds. Evolve refers to experience collection, while validation determines candidate acceptance. We have updated Appendix C.4, Tables 8 and 9 with the task-group/suite and seed allocations for RoboCasa365 and LIBERO-PRO, respectively. Appendix E.1, Table 10 provides the full RoboCasa365 Target50 task list.

All 16 RoboCasa365 Composite-Unseen tasks were excluded from initial skill construction, StateSkill-Evolve, candidate validation, and StateGRPO training across all seeds, including seed 0. These tasks were reserved for final evaluation. Development used all 18 Atomic-Seen and 16 Composite-Seen tasks, as summarized in Tables R2a and R2b. RoboCasa365 evaluation comprises 340 episodes, including 80 on Composite-Unseen.

Table R2a. RoboCasa365 environment seeds by stage.

Task groupTraining: ConstructionTraining: Evolve / StateGRPOTraining: ValidationTesting: Final evaluation
Atomic-Seen011–13141–10
Composite-Seen011–13141–5
Composite-Unseen———1–5

Table R2b. RoboCasa365 task groups used at each stage.

StageTraining: ConstructionTraining: Evolve / StateGRPOTraining: ValidationTesting: Final evaluation
Task groupsAtomic-Seen + Composite-SeenAtomic-Seen + Composite-SeenAtomic-Seen + Composite-SeenAtomic-Seen + Composite-Seen + Composite-Unseen

For LIBERO-PRO, Long (LIBERO-10) tasks are excluded from initial skill construction but participate in Evolve, candidate validation, and StateGRPO. Final evaluation covers 80 task/swap configurations × 10 seeds = 800 episodes. Tables R3a and R3b summarize the allocation.

Table R3a. LIBERO-PRO environment seeds by stage.

SuiteTraining: ConstructionTraining: Evolve / StateGRPOTraining: ValidationTesting: Final evaluation
Spatial011–13141–10
Object011–13141–10
Goal011–13141–10
Long—11–13141–10

Table R3b. LIBERO-PRO suites used at each stage.

StageTraining: ConstructionTraining: Evolve / StateGRPOTraining: ValidationTesting: Final evaluation
SuitesSpatial + Object + GoalSpatial + Object + Goal + LongSpatial + Object + Goal + LongSpatial + Object + Goal + Long

2. Adaptation for real-world deployment.

For real-world deployment, we replace the simulation interfaces with real-robot interfaces and use a frozen π0.5 VLA executor. The simulation-derived StateSkills retain essentially the same state definitions, guidance, gate logic, and numerical thresholds, with task-specific objects, destinations, and completion criteria supplied within this shared execution structure. We have updated Appendix H with these deployment details.

W3: Co-evolution and structural revisions.

We thank the reviewer for requesting further details on co-evolution and skill revisions. Your questions have helped us clarify the co-evolution schedule and strengthen its evaluation with additional cycles, an equal-budget comparison, and a concrete structural-revision example.

1. Co-evolution procedure and supplementary cycles.

In our framework, a co-evolution cycle consists of StateSkill-Evolve with the model fixed, followed by StateGRPO with the refined skills fixed. Our original main results use one cycle of 10 Evolve steps followed by 100 StateGRPO optimizer updates. The same per-cycle budgets apply to the supplementary alternating schedule. One candidate is generated and evaluated per Evolve step. Only candidates satisfying the validation acceptance criterion are applied.

To examine whether further co-evolution cycles remain beneficial, we conducted two additional cycles, each starting from the model and skills produced by the preceding cycle. Table R4a reports the success rates after each phase.

Table R4a. Success rates across co-evolution cycles on LIBERO-PRO with Qwen3.5-9B (%). Cycle 1 corresponds to the original Table 4; cycles 2 and 3 are supplementary experiments. Cycle gain is measured in percentage points relative to the start of that cycle.

CycleBefore EvolveAfter EvolveAfter StateGRPOCycle gain (pp)
1 (original)63.568.470.9+7.4
2 (supplementary)70.972.373.1+2.3
3 (supplementary)73.173.974.5+1.4

Success increases from 70.9% after the first cycle to 74.5% after the third, an additional 3.6 percentage points. Both skill refinement and model training continue to improve performance, with diminishing gains across cycles. These diminishing gains may partly reflect limitations of the current model, leaving some difficult tasks unresolved despite further refinement and training. We have updated Appendix E.2 with the original schedule and these supplementary results (Table 11).

2. Skill revisions and a concrete structural example.

In any Evolve phase, StateSkill-Evolve can revise a skill's state graph and state-conditioned knowledge, including language guidance, transition gates, and action priors. As one example, a revision to a pick-and-place skill added a grasp-verification state and a recovery transition back to VLA grasping, enabling a retry after a verified missed grasp within the attempt budget.

Compared with the pre-revision skill, the illustrated revision improved overall success by 2.3 percentage points on LIBERO-PRO with Qwen3.5-9B. Model parameters and the language guidance, action priors, and gate conditions of unchanged states were held fixed. We have added a before-and-after illustration of this example in Appendix C.5.

To evaluate the additional contribution of structural revisions, we have added a fixed-graph ablation (C1) to Appendix F.4, Table 17. C1 uses the same state-localized diagnosis as the full method (C2), but fixes the initial state set and transition edges while allowing updates to language guidance, action priors, and gate conditions on existing transitions. Both conditions use the same initial skills, model checkpoint, development data, 10-step refinement budget, and validation and final-evaluation protocols, with StateGRPO disabled.

Table 17 (reproduced from the revised manuscript). Skill-revision comparison on LIBERO-PRO with Qwen3.5-9B. Repairs and regressions are paired counts relative to C0; token usage is measured in millions per episode during deployment.

ConditionSuccess (%) ↑Repairs ↑Regressions ↓Tokens/ep. (M) ↓
C0: Fixed skill63.5——0.60
C1: Fixed-graph revision65.332180.56
C2: State-localized revision68.451120.47

The full method (C2) reaches 68.4%, compared with 65.3% for the fixed-graph variant (C1), an improvement of 3.1 percentage points. This comparison supports the additional benefit of allowing structural revisions beyond knowledge updates on a fixed graph.

Q3: Equal-budget comparison.

To assess the benefit of alternating skill refinement and model training under matched budgets, we compared the two schedules on LIBERO-PRO with Qwen3.5-9B, starting from the same model checkpoint and initial StateSkills. Both schedules use 30 Evolve steps and 300 StateGRPO optimizer updates. The non-alternating baseline performs all 30 Evolve steps with the model fixed, then freezes the resulting skills for all 300 training updates. The alternating schedule uses three cycles, each with 10 Evolve steps followed by 100 training updates.

The non-alternating baseline reaches 68.8% after the 30 Evolve steps and 72.6% after fixed-skill training. Alternating co-evolution reaches 74.5%, an improvement of 1.9 percentage points under matched refinement-step and training-update budgets (Table R4b).

Table R4b. Equal-budget comparison on LIBERO-PRO with Qwen3.5-9B.

Optimization scheduleTotal Evolve stepsTotal training updatesFinal success (%)
One-time refinement → fixed-skill training3030072.6
Alternating skill–model co-evolution3030074.5

We have added this comparison to Appendix E.2, Table 11(b).

We thank the reviewer again for the constructive feedback, which has made the revised manuscript clearer and more complete.

Reviewer_B8ZiOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates how reusable skills can support more reliable embodied task execution. Its proposed representation, StateSkill, associates task-progress states with language guidance, transition conditions, and action preferences. StateSkill-Evolve updates these representations from execution experience, while StateGRPO trains the high-level MLLM using task success and relative efficiency within states. The low-level execution policies remain frozen. Evaluation on LIBERO-PRO, RoboCasa365, and real-world manipulation tasks shows improvements in success rate and deployment efficiency. Additional experiments examine knowledge presentation, transition verification, reward design, and skill revision.

Critical strengths
  1. Strong practical gains. With Qwen3.5-9B, StateSkill-Agent improves LIBERO-PRO success from 25.8% to 68.4% without policy fine-tuning, while also reducing deployment time. These results support the practical value of state-structured execution.
  2. Informative controlled experiments. The same-knowledge ablations help isolate the benefits of state-conditioned guidance, transition gates, and action priors. Repeated training runs further support the benefit of the state-aware reward.
Critical weaknesses
  1. Task-level generalization remains unclear. Evaluation seeds are held out, but task overlap across skill construction, evolution, validation, and RL training is insufficiently specified. In particular, it is unclear whether RoboCasa Composite-Unseen tasks were excluded from all development stages, limiting the conclusions about transfer to unseen tasks.
  2. The benefit of co-evolution is not sufficiently isolated. Existing ablations support skill refinement and policy training, but do not establish that repeated alternation outperforms refining skills once and then training with fixed skills. Actual structural revisions and an equal-budget comparison are needed to substantiate this claim.
Questions for authors

Additional Limitations 3. Novelty could be positioned more precisely. The paper should better distinguish its contribution from existing skill-learning and co-evolution frameworks, particularly the additional value of explicit state structures and executable gates. 4. Reproduction details are incomplete. Key training parameters, evolution rounds, candidate budgets, and validation-set sizes are missing. Comparable experience and refinement budgets for the reproduced baselines also need clarification. 5. The reward introduces an underexplored trade-off. State-relative efficiency can favor trajectories with more total interaction turns. The rationale and sensitivity to state granularity and extreme count ratios deserve further analysis. 6. Main-result uncertainty is insufficiently characterized. Although the reward ablation includes repeated runs, the main comparisons lack uncertainty estimates, particularly for the small GPT-5.5 gains and real-world results. 7. Real-world transfer evidence has a limited scope. The required environment-specific adaptations are insufficiently documented, and the five-subtask sequence is qualitative rather than a repeated evaluation of long-horizon reliability. Questions For Authors

  1. Were the RoboCasa Composite-Unseen tasks excluded from every development stage, including runs with different seeds? Please specify task and seed allocations.
  2. How many co-evolution rounds were performed, and which accepted revisions actually changed the state graph?
  3. Is there an equal-budget comparison against one-time skill refinement followed by fixed-skill policy training? This would materially affect my assessment of the co-evolution claim.
7 comments

We sincerely thank the reviewer for carefully reading our paper and for recognizing the practical gains and the value of the controlled experiments. We appreciate the specific concerns about task generalization, co-evolution, and the scope of the empirical evidence. Your constructive suggestions have helped us improve the manuscript by clarifying the experimental protocol, strengthening the empirical evaluation, and making the scope of our findings more explicit.

W1 & Q1: Task-level generalization and task–seed allocation.

We thank the reviewer for highlighting the distinction between task-level exclusion and seed-level separation. This comment has helped us make the task split and the evidence for task-level generalization clearer in the revised manuscript. All 16 RoboCasa Composite-Unseen tasks were excluded from every development stage of our method, including initial skill construction, StateSkill-Evolve, candidate validation, and StateGRPO training, across all environment seeds, including seed 0. No trajectories, task-specific memories, or experience summaries from these tasks were used during development. Development used all 18 Atomic-Seen and 16 Composite-Seen tasks. Skills and model parameters were fixed before final evaluation.

Table R1a lists the environment seeds, while Table R1b separately lists the task groups used at each stage. In both tables, the first three stage columns belong to Training, which covers the full development process, including candidate validation for acceptance. The last column, Testing, denotes final evaluation only.

Table R1a. RoboCasa365 environment seeds by stage.

Task groupTraining: ConstructionTraining: Evolve / StateGRPOTraining: ValidationTesting: Final evaluation
Atomic-Seen011–13141–10
Composite-Seen011–13141–5
Composite-Unseen———1–5

Table R1b. RoboCasa365 task groups used at each stage.

StageTraining: ConstructionTraining: Evolve / StateGRPOTraining: ValidationTesting: Final evaluation
Task groupsAtomic-Seen + Composite-SeenAtomic-Seen + Composite-SeenAtomic-Seen + Composite-SeenAtomic-Seen + Composite-Seen + Composite-Unseen

— denotes exclusion from that stage across all seeds. Evolve refers to experience collection. The same 34 seen tasks participate in Training and Testing with disjoint seeds; all 16 Composite-Unseen tasks participate only in Testing. Thus, seen-task results evaluate new initializations, while Composite-Unseen additionally evaluates task-level generalization. Final evaluation comprises 340 episodes, including 80 on Composite-Unseen. We have revised Appendix C.4, Tables 8 and 9 to separate environment-seed allocation from the task groups or suites used at each stage for RoboCasa365 and LIBERO-PRO, respectively. Appendix E.1, Table 10 provides the full RoboCasa365 task list.

W2 & Q2: Co-evolution and structural revisions.

1. Co-evolution procedure and supplementary cycles.

We thank the reviewer for requesting further details on the co-evolution procedure and its evaluation. Your suggestions have helped us improve the manuscript by clarifying the co-evolution procedure and strengthening its empirical evaluation. In our framework, a co-evolution cycle consists of StateSkill-Evolve with the model fixed, followed by StateGRPO with the refined skills fixed. Our original main results use one cycle of 10 Evolve steps followed by 100 StateGRPO optimizer updates. The same per-cycle budgets apply to the supplementary alternating schedule. One candidate is generated and evaluated per Evolve step. Only candidates satisfying the validation acceptance criterion are applied.

To examine whether further co-evolution cycles remain beneficial, we conducted two additional cycles, each starting from the model and skills produced by the preceding cycle. Table R2a reports the success rates after each phase.

Table R2a. Success rates across co-evolution cycles on LIBERO-PRO with Qwen3.5-9B (%). Cycle 1 corresponds to the original Table 4; cycles 2 and 3 are supplementary experiments. Cycle gain is measured in percentage points relative to the start of that cycle.

CycleBefore EvolveAfter EvolveAfter StateGRPOCycle gain (pp)
1 (original)63.568.470.9+7.4
2 (supplementary)70.972.373.1+2.3
3 (supplementary)73.173.974.5+1.4

Success increases from 70.9% after the first cycle to 74.5% after the third, an additional 3.6 percentage points. Both skill refinement and model training continue to improve performance, with diminishing gains across cycles. These diminishing gains may partly reflect limitations of the current model, leaving some difficult tasks unresolved despite further refinement and training. We have updated Appendix E.2 with the original schedule and these supplementary results (Table 11).

2. Structural revisions (Q2).

Your question has helped us explain more precisely what structural refinement changes and how we evaluate its contribution.

In any Evolve phase, StateSkill-Evolve can revise a skill's state graph and state-conditioned knowledge, including language guidance, transition gates, and action priors. As one example, a revision to a pick-and-place skill added a grasp-verification state and a recovery transition back to VLA grasping, enabling a retry after a verified missed grasp within the attempt budget.

Compared with the pre-revision skill, the illustrated revision improved overall success by 2.3 percentage points on LIBERO-PRO with Qwen3.5-9B. Model parameters and the language guidance, action priors, and gate conditions of unchanged states were held fixed. We have added a before-and-after illustration of this example in Appendix C.5.

To evaluate the additional contribution of structural revisions, we have added a fixed-graph ablation (C1) to Appendix F.4, Table 17. C1 uses the same state-localized diagnosis as the full method (C2), but fixes the initial state set and transition edges while allowing updates to language guidance, action priors, and gate conditions on existing transitions. Both conditions use the same initial skills, model checkpoint, development data, 10-step refinement budget, and validation and final-evaluation protocols, with StateGRPO disabled.

Table 17 (reproduced from the revised manuscript). Skill-revision comparison on LIBERO-PRO with Qwen3.5-9B. Repairs and regressions are paired counts relative to C0; token usage is measured in millions per episode during deployment.

ConditionSuccess (%) ↑Repairs ↑Regressions ↓Tokens/ep. (M) ↓
C0: Fixed skill63.5——0.60
C1: Fixed-graph revision65.332180.56
C2: State-localized revision68.451120.47

The full method (C2) reaches 68.4%, compared with 65.3% for the fixed-graph variant (C1), an improvement of 3.1 percentage points. This comparison supports the additional benefit of allowing structural revisions beyond knowledge updates on a fixed graph.

Q3: Equal-budget comparison.

We thank the reviewer for suggesting a comparison under matched budgets, which has strengthened our evaluation of alternating skill refinement and model training. We compared the two schedules on LIBERO-PRO with Qwen3.5-9B, starting from the same model checkpoint and initial StateSkills. Both schedules use 30 Evolve steps and 300 StateGRPO optimizer updates. The non-alternating baseline performs all 30 Evolve steps with the model fixed, then freezes the resulting skills for all 300 training updates. The alternating schedule uses three cycles, each with 10 Evolve steps followed by 100 training updates.

The non-alternating baseline reaches 68.8% after the 30 Evolve steps and 72.6% after fixed-skill training. Alternating co-evolution reaches 74.5%, an improvement of 1.9 percentage points under matched refinement-step and training-update budgets (Table R2b).

Table R2b. Equal-budget comparison on LIBERO-PRO with Qwen3.5-9B.

Optimization scheduleTotal Evolve stepsTotal training updatesFinal success (%)
One-time refinement → fixed-skill training3030072.6
Alternating skill–model co-evolution3030074.5

We have added this comparison to Appendix E.2, Table 11(b).

Additional Limitation 3: Novelty relative to skill-learning and co-evolution frameworks.

We thank the reviewer for suggesting a clearer comparison with existing frameworks. This suggestion has helped us better explain the contribution of StateSkill relative to prior skill-learning and co-evolution approaches.

SkillRL studies how LLM agents can turn past experience into reusable knowledge that improves their policies. Because raw trajectories contain redundant and noisy information, it distills experience into a hierarchical skill library, retrieves relevant general and task-specific heuristics, and recursively evolves the library alongside the policy during reinforcement learning. COSPLAY focuses on consistent decision making in long-horizon interactive tasks, where agents need to discover, retain, and reuse skills across episodes. This motivates a co-evolution framework in which a decision agent learns skill retrieval and action generation, while a skill-bank agent extracts and refines skills and their contracts from rollouts.

Our work focuses on representing, reusing, and improving behavioral knowledge for embodied MLLM agents. This requires skills to represent evolving physical interactions while keeping guidance relevant to the current execution stage. We therefore organize embodied experience into task-progress states equipped with language guidance, evidence-based transition gates, and trajectory-derived action priors, loading guidance for the active state to reduce redundant context. We further use this shared progress representation to localize Evolve revisions and define StateGRPO's efficiency rewards, allowing skill knowledge and model capabilities to improve together through interaction. We have expanded Appendix G.3 to discuss these distinctions.

We evaluate these mechanisms using the same underlying skill knowledge on LIBERO-PRO with Qwen3.5-9B (Appendix F.2, Table 15).

Execution conditionSuccess (%)Erroneous advancement (%)
Full-knowledge text47.018.0
Explicit states and state-conditioned guidance; MLLM-assessed progression54.513.0
Evidence-based transition gates, without the action prior60.06.5
Full StateSkill, including the action prior63.56.0

State organization and state-conditioned guidance jointly improve success from 47.0% to 54.5%. Adding evidence-based gates raises success to 60.0% and halves erroneous advancement from 13.0% to 6.5%. Adding the action prior further improves success to 63.5%, demonstrating the contributions of the execution mechanisms.

Comments on the paper

1 comment

We sincerely thank all reviewers for their time and thoughtful, constructive feedback. We appreciate your recognition of StateSkill’s practical benefits for effective and efficient embodied task execution. Your comments have helped us clarify the state-conditioned skill representation, strengthen the evaluation of skill–model co-evolution, and improve the overall quality of our manuscript. We have revised the manuscript accordingly and provide point-by-point responses below. We hope our responses and revisions address your concerns, and we welcome further questions and discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.