- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
StateSkill represents reusable robot skills as progress states with language guidance, evidence-based transition gates, and action preferences. An offline constructor extracts skills from trajectories, StateSkill-Evolve revises them using execution feedback and paired validation, and StateGRPO trains the agent using success and state-relative interaction efficiency. Evaluation covers LIBERO-PRO, RoboCasa365, physical manipulation, and controlled component comparisons.
Critical strengths
-
Controlled evidence for execution improvements. With the same underlying knowledge, Table 10 improves success from 47.0% to 54.5% with progress organization and phase-specific guidance, 60.0% with evidence gates, and 63.5% with action priors. These comparisons support concrete design choices beyond gains over an unassisted model.
-
Useful revision and reward results. Localized revision reaches 68.4% versus 65.0% for whole-skill reflection (Table 12). Across ten training repetitions, the progress-relative reward improves success over global efficiency shaping by 1.8 percentage points, with a reported 95% interval of [0.69, 2.91] (Table 11).
Critical weaknesses
-
The state-representation claims exceed the defined mechanism. Formally, the state variable denotes a task-progress phase; environmental observations and retained execution facts enter the transition gate separately. This is a valid controller definition. However, the introduction extends its significance to representing changing agent and environment states without establishing the corresponding abstraction. The paper should distinguish execution-progress tracking from environmental-state representation and substantiate its contribution at the appropriate level.
Table 10's A0–A1 comparison combines progress organization with selective guidance; Table 12 does not separate graph changes from guidance, gate, or prior updates. Table 8 provides only a schematic edit. The distinctive contribution of learning or revising the skill structure therefore remains unestablished.
The novelty claim also needs narrower positioning: CEDAR (August 2026) learns reusable embodied skill automata, while HEXIS (September 2026) combines FSMs, local instructions, and trace-driven graph updates. These contemporaneous works inform positioning; their absence as baselines is not a rejection criterion.
-
The claimed benefit of co-evolution is not established. Table 4 adds StateGRPO to skill revision, whereas Table 11 holds skills fixed. Neither isolates alternation against one refinement stage followed by policy training under matched budgets. The validated RL contribution is a progress-relative efficiency reward used through trajectory-level advantages; it does not establish the claimed mutually reinforcing effect of repeated skill and policy updates.
Questions for authors
- Which actual structural edits were accepted, and what evidence distinguishes their contribution from updating guidance, gates, or priors on a fixed graph?
- Do existing results isolate alternation from sequential skill refinement and policy training? If not, please narrow the co-evolution claim accordingly.
2 comments
W2 & Q2: Co-evolution under matched budgets and StateGRPO evidence.
We thank the reviewer for requesting clarification of the co-evolution procedure and its evaluation. Your suggestions have helped us improve the manuscript through a clearer co-evolution protocol and a fuller evaluation of alternating updates and the StateGRPO reward design.
1. Co-evolution procedure and supplementary cycles.
In our framework, a co-evolution cycle consists of StateSkill-Evolve with the model fixed, followed by StateGRPO with the refined skills fixed. Our original main results use one cycle of 10 Evolve steps followed by 100 StateGRPO optimizer updates. The same per-cycle budgets apply to the supplementary alternating schedule. One candidate is generated and evaluated per Evolve step. Only candidates satisfying the validation acceptance criterion are applied.
To examine whether further co-evolution cycles remain beneficial, we conducted two additional cycles, each starting from the model and skills produced by the preceding cycle. Table R1a reports the success rates after each phase.
Table R1a. Success rates across co-evolution cycles on LIBERO-PRO with Qwen3.5-9B (%). Cycle 1 corresponds to the original Table 4; cycles 2 and 3 are supplementary experiments. Cycle gain is measured in percentage points relative to the start of that cycle.
| Cycle | Before Evolve | After Evolve | After StateGRPO | Cycle gain (pp) |
|---|---|---|---|---|
| 1 (original) | 63.5 | 68.4 | 70.9 | +7.4 |
| 2 (supplementary) | 70.9 | 72.3 | 73.1 | +2.3 |
| 3 (supplementary) | 73.1 | 73.9 | 74.5 | +1.4 |
Success increases from 70.9% after the first cycle to 74.5% after the third, an additional 3.6 percentage points. Both skill refinement and model training continue to improve performance, with diminishing gains across cycles. These diminishing gains may partly reflect limitations of the current model, leaving some difficult tasks unresolved despite further refinement and training. We have updated Appendix E.2 with the original schedule and these supplementary results (Table 11).
2. Equal-budget comparison.
To assess the benefit of alternating skill refinement and model training under matched budgets, we compared the two schedules on LIBERO-PRO with Qwen3.5-9B, starting from the same model checkpoint and initial StateSkills. Both schedules use 30 Evolve steps and 300 StateGRPO optimizer updates. The non-alternating baseline performs all 30 Evolve steps with the model fixed, then freezes the resulting skills for all 300 training updates. The alternating schedule uses three cycles, each with 10 Evolve steps followed by 100 training updates.
The non-alternating baseline reaches 68.8% after the 30 Evolve steps and 72.6% after fixed-skill training. Alternating co-evolution reaches 74.5%, an improvement of 1.9 percentage points under matched refinement-step and training-update budgets (Table R1b).
Table R1b. Equal-budget comparison on LIBERO-PRO with Qwen3.5-9B.
| Optimization schedule | Total Evolve steps | Total training updates | Final success (%) |
|---|---|---|---|
| One-time refinement → fixed-skill training | 30 | 300 | 72.6 |
| Alternating skill–model co-evolution | 30 | 300 | 74.5 |
We have added this comparison to Appendix E.2, Table 11(b).
3. StateGRPO reward design and evidence.
StateGRPO compares interaction counts within the same progress state across successful trajectories for the same task and environment seed. It combines the resulting state-efficiency reward with task success into a trajectory reward, from which the trajectory-level group-relative advantage is computed. We have clarified this computation in Appendix D.1.
The reward ablation in Appendix F.3, Table 16 uses fixed initial StateSkills, disabled skill evolution, and a shared model initialization.
Table R2. Reward ablation on LIBERO-PRO with Qwen3.5-9B.
| Training variant | Success (%) | Interaction turns | Tokens/episode (M) |
|---|---|---|---|
| GRPO (task-success reward) | 67.0 | 26.0 | 0.56 |
| GRPO + global efficiency reward | 67.5 | 22.9 | 0.48 |
| StateGRPO (task success + state-aware efficiency) | 69.3 | 23.2 | 0.49 |
Success rates for GRPO + global efficiency reward and StateGRPO are means over ten training repetitions.
The state-aware reward improves success by 2.3 percentage points over success-only training. Across ten training repetitions, its mean improvement over global efficiency is 1.80 percentage points (95% confidence interval: [0.69, 2.91]), with similar interaction counts (23.2 vs. 22.9) and token usage (0.49M vs. 0.48M) to global efficiency.
We thank the reviewer again for the suggestions, which have improved both the presentation of StateSkill and the evidence supporting its design.
We thank the reviewer for recognizing the controlled evidence for execution improvements, state-localized revision, and the progress-relative reward. Below, we clarify the state representation and report supplementary evidence on structural edits and co-evolution.
W1 & Q1: State representation, structural revisions, and novelty positioning.
We thank the reviewer for requesting clarification of the state representation and evidence for structural revisions. Your comments have helped us explain the progress representation and structural revisions more clearly and better distinguish StateSkill from related frameworks.
1. What the StateSkill state represents.
The active state qt represents progress within an embodied skill. In Equation (3), environmental evidence et+1 and retained execution facts mt determine how the gate updates this progress and the retained facts. For example, grasp verification identifies the current execution stage, while observations and feedback establish whether the object is held or the grasp was missed, determining the subsequent transition and guidance. We have updated Appendix B.1 to clarify how task-progress states are updated using environmental evidence.
2. Skill revisions and a concrete structural example.
In any Evolve phase, StateSkill-Evolve can revise a skill's state graph and state-conditioned knowledge, including language guidance, transition gates, and action priors. As one example, a revision to a pick-and-place skill added a grasp-verification state and a recovery transition back to VLA grasping, enabling a retry after a verified missed grasp within the attempt budget.
Compared with the pre-revision skill, the illustrated revision improved overall success by 2.3 percentage points on LIBERO-PRO with Qwen3.5-9B. Model parameters and the language guidance, action priors, and gate conditions of unchanged states were held fixed. We have added a before-and-after illustration of this example in Appendix C.5.
To evaluate the additional contribution of structural revisions, we have added a fixed-graph ablation (C1) to Appendix F.4, Table 17. C1 uses the same state-localized diagnosis as the full method (C2), but fixes the initial state set and transition edges while allowing updates to language guidance, action priors, and gate conditions on existing transitions. Both conditions use the same initial skills, model checkpoint, development data, 10-step refinement budget, and validation and final-evaluation protocols, with StateGRPO disabled.
Table 17 (reproduced from the revised manuscript). Skill-revision comparison on LIBERO-PRO with Qwen3.5-9B. Repairs and regressions are paired counts relative to C0; token usage is measured in millions per episode during deployment.
The full method (C2) reaches 68.4%, compared with 65.3% for the fixed-graph variant (C1), an improvement of 3.1 percentage points. This comparison supports the additional benefit of allowing structural revisions beyond knowledge updates on a fixed graph.
3. Positioning relative to CEDAR and HEXIS.
CEDAR studies how language-guided embodied agents can satisfy persistent constraints during task execution. To make skill behavior verifiable and composable with constraints, it represents both skills and constraints as automata and combines them into controllers that enforce the learned constraints. HEXIS focuses on improving agents' compliance with skill instructions. Because repeatedly inferring control decisions can lead to omitted or incorrect steps, it compiles skills into extended finite-state machines that separate local skill knowledge from control flow, with revisions validated through static checks and trace replay.
Our work focuses on representing, reusing, and improving behavioral knowledge for embodied MLLM agents. This requires skills to represent evolving physical interactions while keeping guidance relevant to the current execution stage. We therefore organize embodied experience into task-progress states equipped with language guidance, evidence-based transition gates, and trajectory-derived action priors, loading guidance for the active state to reduce redundant context. We further use this shared progress representation to localize Evolve revisions and define StateGRPO's efficiency rewards, allowing skill knowledge and model capabilities to improve together through interaction. We have expanded Appendix G.3 with a discussion of CEDAR and HEXIS and these methodological distinctions.