acceptodds
Withdrawn from ICLR 2027

SEES: A Self-Evolving Embodied System via Failure-Guided VLA Policy Adaptation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recent vision-language-action (VLA) policies demonstrate promising generalization across diverse short-horizon tasks. However, they remain unreliable on long-horizon tasks, partly because the large-scale training data is biased toward single-stage manipulation tasks that are cheaper to demonstrate. A single weak atomic skill can cause failures across multiple multi-stage tasks. To address such failures, existing methods often require experts to identify the bottleneck and provide additional demonstrations, making the improvement costly and potentially impractical after deployment. To this end, we present a Self-Evolving Embodied System (SEES) that learns from failures and improves the VLA policy without additional expert demonstrations. SEES decomposes long-horizon tasks into atomic tasks and routes them to corresponding family policies. Each family consists of related atomic skills that share one VLA adapter. During execution, the system automatically monitors atomic-task outcomes to identify the most frequently failing atomic skills as the current bottlenecks. To overcome these bottlenecks, SEES constructs tailored RL tasks in simulation by restoring previously encountered states and generating task-specific success criteria with an LLM. Online RL updates the shared family adapters to promote positive transfer among related atomic skills and cumulative improvement across evolution rounds. Extensive experiments show that SEES can be integrated with different VLA backbones to progressively improve their long-horizon performance. We also observe continued improvement on unseen tasks, providing evidence of transfer beyond the evolution settings.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Self-Evolving Robots, Continual Robot Learning, Failure-Driven Adaptation, Long-Horizon Mobile Manipulation, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_16nCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

SEES adapts VLA policies through a failure-driven loop. It decomposes long-horizon tasks, identifies bottleneck skills, and constructs local PPO tasks using saved simulator states and LLM-generated success predicates. Related skills share a family-level LoRA on a frozen backbone. Experiments on RoboCasa365 and LIBERO report improvements over initial policies, with focused comparisons of family sharing and bottleneck selection.

Critical strengths

The system connects recurring skill failures to targeted training and shared updates, providing a coherent approach to improving long-horizon execution without additional expert demonstrations.

Experiments span two backbones and two benchmarks, with direct controls for sharing and bottleneck selection. LIBERO’s disjoint training, approval, and sealed evaluation states strengthen the reported improvements.

Critical weaknesses

Weak starting baselines. LIBERO starts at only 53.1% and reaches 65.4%, while supervised OpenVLA-OFT reports 94.5–97.1% under its own protocols. The low baseline requires explanation, and the results do not establish whether the approach remains useful for stronger starting policies.

Limited effectiveness for a primarily system-level contribution. RoboCasa composite-task gains over the base policies are only 1.1–2.9 percentage points, with final success rates of 6.1–21.8%. The novelty mainly lies in integrating existing components, making stronger effectiveness evidence important. All experiments are simulated; the autonomous improvement loop, which relies on state restoration and simulator-state rewards, remains unvalidated on real robots.

Questions for authors
  1. Why do the LIBERO SFT checkpoints start at 53.1%? Please specify their training, input, and evaluation settings, and clarify whether stronger starting policies have been tested.

  2. Are matched-interaction controls available against ordinary PPO or uniform/random target selection across multiple RoboCasa tasks? Such evidence could materially change the assessment.

  3. Please reconcile the six-to-seven RL targets stated in Section 4.3 with the 7/9/9/9 counts implied by Table 2. Can per-task outcomes and independent evolution repeats or appropriate uncertainty estimates be provided?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 16nC — Part 1/2

Response to Reviewer 16nC — Part 1/2

Starting checkpoints and stronger baselines (W1 / Q1)

Thank you for your thoughtful review and constructive suggestions. Manuscript Table 3 tests family-level adapter sharing, while Table 4 tests failure-guided bottleneck selection. We added simple multi-task PPO controls to both: one globally shared LoRA over nine atomic tasks (Table 1, expanding manuscript Table 3), and uniform training over reached subtasks (Table 2, expanding manuscript Table 4). The proposed designs exceed these controls by 5.34 and 6.27 percentage points overall, respectively, under matched budgets within each experiment.

Our original LIBERO policies are GR00T SFT checkpoints released by RLinf; we did not train them ourselves. To address weak starting policies, the new fixed-7/3 comparison includes both GR00T and official OpenVLA-OFT checkpoints, with matched interactions within each backbone and suite (Table 3 in Part 2, supplementing manuscript Table 2). OpenVLA-OFT + SEES reaches 97.69% overall success versus 97.03% for Base, 96.64% for Uniform PPO, and 96.19% for Random: gains of 0.66, 1.05, and 1.50 points. It exceeds Uniform in all four suites, including Long (96.04% vs. 95.07%). These results support effectiveness with strong starting policies, although gains are modest near the performance ceiling.

Evaluation uses native LIBERO success predicates on 15 sealed initial states × five inference seeds per task. GR00T checkpoints are suite-specific. In Object/Spatial/Goal/Long order, GR00T uses horizons of 240/240/320/480, action chunks of 5/5/5/10, and four denoising steps. OpenVLA-OFT uses horizons of 280/220/300/520 and action chunks of eight.

Matched-interaction controls (Q2)

Within each RoboCasa ablation, methods share the starting checkpoint, LoRA configuration, PPO recipe, parallel-environment count, rollout epochs, horizon, and termination/loss-masking rules. Total PPO outer steps summed across adapters are matched, ensuring equal fixed-horizon interaction budgets. Target selection across five composite tasks uses 80 steps per task and method, in two 40-step rounds. Simple multi-task distributes this total across reached subtasks, not 40 steps per subtask or family. Random uniformly selects one reached subtask per round; Bottleneck selects the one with the most attributed failures. Policies are evaluated on paired held-out scenes.

Simple multi-task raises mean success from 20.40% to 50.80%, showing that extra RL helps. Bottleneck reaches 57.07% under the same budget, exceeding Simple multi-task, Random (41.07%), and Highest-success (21.33%) by 6.27, 16.00, and 35.74 points. This supports failure-guided allocation over uniform, random, or highest-success selection. Table 1 separately matches 90 total PPO outer steps across sharing schemes: Semantic family reaches 73.87% overall versus 68.53% for Simple multi-task and 71.42% for Random groups, supporting semantic sharing beyond joint RL alone.

Table 1. RoboCasa365 LoRA sharing: expanded manuscript Table 3. Success (%) over nine trained and six RL-held-out sibling tasks, 50 rollouts/task. Adapted values are means over three independent training seeds; ± denotes the 95% CI half-width in percentage points. Base has no further training and one evaluation run. Best means are bold.

MethodRL-trained tasks (9)Sibling tasks (6)Overall
Base70.8964.3368.27
Atomic-specific72.37 ± 0.6464.3369.16 ± 0.38
Random groups75.41 ± 1.6965.44 ± 1.7271.42 ± 0.51
Simple multi-task69.11 ± 0.5567.67 ± 2.1968.53 ± 1.15
Semantic family78.30 ± 2.2367.22 ± 0.4873.87 ± 1.44

Table 2. RoboCasa365 target selection: expanded manuscript Table 4. Mean success (%) across five composites; the same three-seed and CI convention applies.

MethodMean success (%)
Base20.40
Random41.07 ± 2.29
Highest-success21.33 ± 0.57
Simple multi-task50.80 ± 0.99
Bottleneck57.07 ± 1.52

System-level effectiveness and physical validation (W2)

Together, Tables 1–3 support semantic sharing and failure-guided allocation beyond additional RL alone. In a preliminary RoboDojo real-to-sim check, we reconstructed a scene from real-robot SFT images and successfully executed the corresponding SFT actions in simulation. This establishes simulation-side feasibility, not real-robot adaptation or sim-to-real transfer. The complete failure-guided adaptation loop remains unvalidated on physical robots, which we acknowledge as a key limitation.

Please ask further questions if anything remains unclear or inadequately addressed; we welcome further discussion.

AuthorsOpenReview ↗

Reviewer 16nC — Part 2/2

Response to Reviewer 16nC — Part 2/2

RL task counts, selection records, and uncertainty (Q3)

Thank you for pointing out the inconsistency. The correct original counts in manuscript Table 2 are 7/9/9/9: Object selected 1+3+3 targets, whereas the other suites selected 3+3+3. These counts include PPO participation in rejected attempts. We will correct the outdated “six-to-seven” statement.

Our new LIBERO experiment fixes seven adaptation-eligible and three held-out tasks per suite before training, with identical splits across methods and both backbones. Held-out tasks never participate in failure ranking, target selection, PPO, or checkpoint selection, including rejected attempts. They are held out from adaptation, not necessarily unseen during pretraining.

Each round, SEES selects the eligible task with the lowest current diagnostic success rate. Ties use the smallest zero-based task ID within the suite, and reselection is allowed. Table 5 shows round-wise selections from one training run per backbone–suite pair, not a seed aggregate.

We repeat the complete LIBERO evolution process with three independent training seeds, not merely different inference seeds. Table 3 supplements manuscript Table 2, reporting each suite's eligible and held-out results separately. GR00T + SEES achieves 58.07 ± 0.94% overall success versus 54.72 ± 0.72% for Uniform PPO; OpenVLA-OFT achieves 97.69 ± 0.96% versus 96.64 ± 0.71%. All ± values are 95% confidence-interval half-widths in percentage points, not standard deviations. Base receives no additional training and has one evaluation run.

The expanded RoboCasa ablations also use three independent training seeds and report 95% confidence intervals (Tables 1 and 2 in Part 1). Limited rebuttal time and compute prevented repeating the complete main RoboCasa evolution pipeline.

Table 3. LIBERO success rates (%) under the fixed 7/3 split: supplementary results for manuscript Table 2. Three-seed means; Mean averages four suites.

SuiteMethodGR00T: Eligible (7)GR00T: Held-out (3)GR00T: OverallOpenVLA-OFT: Eligible (7)OpenVLA-OFT: Held-out (3)OpenVLA-OFT: Overall
ObjectBase Model68.3860.8966.1397.14100.0098.00
Random68.89±1.7965.04±2.5567.73±2.0196.70±0.9899.41±1.2797.51±1.06
Uniform72.57±1.2570.07±3.1971.82±1.4996.38±1.8999.85±0.6497.42±1.34
SEES78.86±1.4276.59±2.3078.18±1.5797.97±1.9799.41±2.5598.40±1.75
SpatialBase Model41.7141.3341.6097.33100.0098.13
Random39.17±1.0942.67±2.2140.22±1.3897.08±1.5298.22±1.9197.42±1.49
Uniform39.68±0.2744.15±1.6941.02±0.6997.59±1.3799.11±1.1098.04±1.25
SEES42.67±2.5043.26±2.3042.84±1.0698.22±1.9199.26±1.2798.53±1.72
GoalBase Model41.1448.4443.3396.7699.1197.47
Random43.17±1.7950.96±0.6445.51±1.3495.62±1.4296.00±2.9295.73±1.84
Uniform44.19±2.3747.85±1.6945.29±1.5795.81±0.8296.59±2.3096.04±1.16
SEES45.59±0.7253.63±3.3748.00±1.1997.65±1.4598.07±1.6997.78±0.51
LongBase Model62.4859.5661.6095.2492.8994.53
Random61.90±1.7162.67±2.9262.13±2.0193.27±1.9796.00±2.2194.09±1.67
Uniform60.13±1.1962.22±1.1060.76±1.1695.75±1.4593.48±0.6495.07±1.19
SEES64.06±3.1561.33±1.9163.24±2.7896.13±2.6995.85±3.1996.04±1.88
MeanBase Model53.4352.5653.1796.6298.0097.03
Random53.29±1.0555.33±0.9653.90±1.0295.67±1.2897.41±1.1596.19±1.24
Uniform54.14±1.1356.07±0.4254.72±0.7296.38±0.5997.26±1.0496.64±0.71
SEES57.79±1.5858.70±0.5758.07±0.9497.49±1.3298.15±0.5797.69±0.96

Table 5. Per-round bottleneck selections. One run per backbone–suite pair. IDs are zero-based within each suite; ties use the smallest ID.

BackboneSuiteRound 1Round 2Round 3
GR00TObject524
GR00TSpatial431
GR00TGoal360
GR00TLong097
OpenVLA-OFTObject861
OpenVLA-OFTSpatial451
OpenVLA-OFTGoal302
OpenVLA-OFTLong069

Task descriptions. Object: 1 = cream cheese into basket; 2 = salad dressing into basket; 4 = ketchup into basket; 5 = tomato sauce into basket; 6 = butter into basket; 8 = chocolate pudding into basket. Spatial: 1 = black bowl next to ramekin onto plate; 3 = black bowl on cookie box onto plate; 4 = black bowl from drawer onto plate; 5 = black bowl on ramekin onto plate. Goal: 0 = open middle drawer; 2 = wine bottle onto cabinet top; 3 = open top drawer and place bowl inside; 6 = cream cheese into bowl. Long: 0 = alphabet soup and tomato sauce into basket; 6 = white mug onto plate and chocolate pudding to the right of the plate; 7 = alphabet soup and cream cheese into basket; 9 = yellow-and-white mug into microwave and close its door.

Please ask further questions if anything remains unclear or inadequately addressed; we welcome further discussion.

Reviewer_xuaQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SEES improves robot policies by identifying and training frequently failing skills. It breaks long tasks into subtasks and uses a learned termination head to track completion. It then selects bottlenecks, restores simulator states from before those subtasks, and uses an LLM to generate success checks. Related skills share a LoRA adapter, so training it with PPO on a failing skill can also improve other skills. This cycle repeats without additional expert demonstrations. RoboCasa365 experiments test the full system with two VLA backbones. LIBERO experiments test the adaptation process. Performance improves on the tasks used for RL training, while some tasks left out of RL training also improve.

Critical strengths
  1. S1: SEES uses execution failures to decide which skills to train. It restores simulator states from just before the selected subtask, so training can start there without repeating earlier stages. This lets the robot practice weak skills in situations it has already encountered.

  2. S2: The results show some transfer to tasks outside RL training. Both RoboCasa365 backbones improve on Composite-Unseen, which is excluded from adaptation and checkpoint selection. All three LIBERO-Object tasks listed as untrained in the accepted evolution chain also improve. The authors also check the termination head and generated success functions against simulator ground truth. These checks help evaluate whether the system can identify failures and assign rewards reliably.

Critical weaknesses
  1. W1: It is unclear how much value SEES adds beyond additional RL training. The main comparisons use policies without adaptation. The comparison with random bottleneck selection covers only one composite task. A comparison across more tasks, with the same interaction and validation budgets, would help establish the value of failure-based selection. The benefit of skill grouping is also unclear. RoboCasa365 does not compare family adapters with one adapter shared across all adaptation tasks. LIBERO uses one adapter per suite, so it does not answer this question either. These missing comparisons make it hard to tell which design choices drive the gains.

  2. W2: I have two concerns about the transfer results. Section 4.3 says that six to seven tasks per LIBERO suite receive RL training. But the averages in Table 2 suggest a split of nine trained tasks and one untrained task for Spatial, Goal, and Long. Which count is correct? Are tasks from rejected rounds counted? If only one task in each of these three suites was not used for RL training, then the transfer evidence for these suites comes from just three tasks. Some gains are also small: RLDX-1 goes from 6.0% to 7.1% on Composite-Unseen. The paper gives no confidence intervals or results from independent training runs. For each LIBERO suite, the five seeds test the policy from a single evolution run. They do not show whether repeating the training would give similar gains.

Questions for authors
  1. Q1: Does SEES outperform training all tasks equally or randomly selecting tasks to train? Please compare these approaches across more tasks, using the same total environment interactions and checkpoint-validation budget. How many families are updated in each round, and what is the total training cost?

  2. Q2: Which tasks were used for RL training in each LIBERO suite? Section 4.3 says six to seven tasks per suite. But the numbers in Table 2 suggest nine for Spatial, Goal, and Long. Which is correct? Were any “untrained” tasks used in training runs whose results were later discarded?

  3. Q3: If you repeat the full evolution process with different training seeds, do you get similar gains? Please also provide confidence intervals for the performance differences, especially the smaller gains on Composite-Unseen.

  4. Q4: Have you checked the generated success functions on the bottleneck subtasks actually used for RL, including trajectories from the updated policies? The current check uses Atomic-Seen trajectories from the base policy. Also, when the termination head controls subtask switching, how often do its errors cause the system to select the wrong bottleneck?

2 comments
AuthorsOpenReview ↗

Response to Reviewer xuaQ — Part 1/2

Response to Reviewer xuaQ — Part 1/2

Matched-budget controls and contributions of system design (W1 / Q1)

Thank you for your careful review and helpful suggestions. In our manuscript, Table 3 examines the effectiveness of family-level adapter sharing, while Table 4 examines the effectiveness of prioritizing failure bottlenecks. To address your concern about gains from additional RL alone, we added simple multi-task PPO controls to both experiments: uniformly training one shared LoRA on all nine atomic tasks in Table 1 (expanding manuscript Table 3), and distributing training uniformly across all reached subtasks in Table 2 (expanding manuscript Table 4). We additionally compare random task groupings and extend target selection to five composite tasks. Semantic family sharing exceeds global sharing and random grouping by approximately 5.34 and 2.45 percentage points, respectively; Bottleneck exceeds simple multi-task PPO and Random selection by 6.27 and 16.00 points. These matched-budget controls support the contributions of semantic sharing and failure-guided allocation beyond additional RL training alone.

Within each ablation, starting checkpoints and total PPO outer steps summed across adapters are matched. Table 2 allocates 80 PPO outer steps per task and method, divided into two rounds of 40 steps. Simple multi-task PPO distributes this budget across all reached subtasks, rather than receiving a separate 40-step budget for each family. All adaptation methods also use identical numbers of parallel environments and rollout epochs, the same rollout horizon, and the same termination and loss-masking rules, ensuring the same fixed-horizon PPO environment-interaction budget. All adaptation variants are trained with three independent random seeds, and we report mean success rates with 95% confidence intervals. The Base policy receives no additional training and is evaluated in a single evaluation run.

In the main GR00T RoboCasa experiment, we train 7, 8, and 7 family adapters in evolution rounds 1, 2, and 3, respectively. Selected bottleneck tasks belonging to the same family jointly update one shared adapter. Each active family receives 40 PPO outer steps per round; families without a selected bottleneck remain unchanged.

The main three-round GR00T evolution uses 776.13 NVIDIA A800 (80 GB) GPU-hours, broken down in Table 4. These costs describe the main experiment, not the additional ablations.

Table 1. LoRA-sharing ablation on RoboCasa365: expanded manuscript Table 3. Success (%) on nine trained tasks, six RL-held-out siblings, and all 15 tasks; 50 rollouts/task. Values are mean ± 95% CI half-width (percentage points) over three training seeds; Base is a point estimate. Best means are bold.

MethodRL-trained tasks (9)Sibling tasks (6)Overall
Base70.8964.3368.27
Atomic-specific72.37 ± 0.6464.3369.16 ± 0.38
Random groups75.41 ± 1.6965.44 ± 1.7271.42 ± 0.51
Simple multi-task69.11 ± 0.5567.67 ± 2.1968.53 ± 1.15
Semantic family78.30 ± 2.2367.22 ± 0.4873.87 ± 1.44

Table 2. Target-selection ablation on RoboCasa365: expanded manuscript Table 4. Mean success (%) across five composite tasks. Three-seed means and 95% CI half-widths are reported as in Table 1; Base is a point estimate.

MethodMean success (%)
Base20.40
Random41.07 ± 2.29
Highest-success21.33 ± 0.57
Simple multi-task50.80 ± 0.99
Bottleneck57.07 ± 1.52

Table 4. Compute cost for three-round GR00T evolution run on RoboCasa365. Compute is measured in NVIDIA A800 (80 GB) GPU-hours.

ComponentA800 GPU-hours
PPO training683.35
Failure collection36.02
Composite verification56.76
Reported subtotal776.13

Predicate validation and bottleneck-selection reliability (Q4)

We checked the predicates used for actual RL bottlenecks against existing videos and sampled RL-training rollout videos. After cross-checking with Astra, no predicate errors were observed in the inspected cases. These checks cover sampled training trajectories rather than exhaustively validating every updated policy. Bottleneck selection identifies the atomic task with the highest aggregated failure count for each composite task. Although individual termination predictions can be incorrect, occasional errors did not change which task had the largest failure count in our evaluated runs. Across all three evolution rounds for both GR00T N1.5 and RLDX-1 on RoboCasa, no incorrect bottleneck selections were observed, yielding 100% agreement with the simulator oracle.

If any part of our response remains unclear or does not fully address your concerns, please feel free to ask further questions. We welcome further discussion.

AuthorsOpenReview ↗

Response to Reviewer xuaQ — Part 2/2

Response to Reviewer xuaQ — Part 2/2

LIBERO task exposure, evaluation protocol, and uncertainty (W2 / Q2 / Q3)

The original 7/9/9/9 counts reflect 1+3+3 Object targets versus 3+3+3 elsewhere, including rejected PPO attempts. We will correct the outdated “six-to-seven.” Because this left too few tasks to assess transfer reliably, we reran LIBERO with seven eligible and three held-out tasks per suite, using a split fixed before training and shared across methods and backbones. Held-out tasks are excluded from failure ranking, target selection, PPO, and checkpoint selection, including rejected attempts. “Held-out” refers to this adaptation experiment, not to pretraining.

SEES chooses the eligible task with the lowest current diagnostic success rate each round; ties use the smallest zero-based suite-local ID. Table 5 lists one training run per backbone–suite pair, not a seed aggregate. On both GR00T and OpenVLA-OFT, we compare SEES, Random, and Uniform PPO on a common validation panel, with matched outer steps, parallel environments, rollout epochs, and horizons within each backbone and suite, giving equal fixed-horizon interaction budgets.

We repeated full LIBERO evolution with three independent training seeds, not just inference seeds, and observed consistent improvements. Table 3 replaces the original comparison: mean held-out success rises from 52.56% to 58.70% for GR00T (+6.14 percentage points), and from 98.00% to 98.15% for OpenVLA-OFT (+0.15 points). Limited rebuttal time and compute prevented repeating the complete main RoboCasa evolution pipeline; the expanded ablations nevertheless report 95% confidence intervals (Tables 1 and 2 in Part 1), supporting these comparisons.

Table 3. LIBERO success rates (%) under the fixed 7/3 split. Adapted results are means over three training seeds; ± denotes the 95% CI half-width in percentage points. Base is not adapted and evaluated once. Mean averages four suites.

SuiteMethodGR00T: Eligible (7)GR00T: Held-out (3)GR00T: OverallOpenVLA-OFT: Eligible (7)OpenVLA-OFT: Held-out (3)OpenVLA-OFT: Overall
ObjectBase Model68.3860.8966.1397.14100.0098.00
Random68.89±1.7965.04±2.5567.73±2.0196.70±0.9899.41±1.2797.51±1.06
Uniform72.57±1.2570.07±3.1971.82±1.4996.38±1.8999.85±0.6497.42±1.34
SEES78.86±1.4276.59±2.3078.18±1.5797.97±1.9799.41±2.5598.40±1.75
SpatialBase Model41.7141.3341.6097.33100.0098.13
Random39.17±1.0942.67±2.2140.22±1.3897.08±1.5298.22±1.9197.42±1.49
Uniform39.68±0.2744.15±1.6941.02±0.6997.59±1.3799.11±1.1098.04±1.25
SEES42.67±2.5043.26±2.3042.84±1.0698.22±1.9199.26±1.2798.53±1.72
GoalBase Model41.1448.4443.3396.7699.1197.47
Random43.17±1.7950.96±0.6445.51±1.3495.62±1.4296.00±2.9295.73±1.84
Uniform44.19±2.3747.85±1.6945.29±1.5795.81±0.8296.59±2.3096.04±1.16
SEES45.59±0.7253.63±3.3748.00±1.1997.65±1.4598.07±1.6997.78±0.51
LongBase Model62.4859.5661.6095.2492.8994.53
Random61.90±1.7162.67±2.9262.13±2.0193.27±1.9796.00±2.2194.09±1.67
Uniform60.13±1.1962.22±1.1060.76±1.1695.75±1.4593.48±0.6495.07±1.19
SEES64.06±3.1561.33±1.9163.24±2.7896.13±2.6995.85±3.1996.04±1.88
MeanBase Model53.4352.5653.1796.6298.0097.03
Random53.29±1.0555.33±0.9653.90±1.0295.67±1.2897.41±1.1596.19±1.24
Uniform54.14±1.1356.07±0.4254.72±0.7296.38±0.5997.26±1.0496.64±0.71
SEES57.79±1.5858.70±0.5758.07±0.9497.49±1.3298.15±0.5797.69±0.96

Table 5. Per-round bottleneck selections. One run per backbone–suite pair; IDs are zero-based within each suite and ties use the smallest ID.

BackboneSuiteRound 1Round 2Round 3
GR00TObject524
GR00TSpatial431
GR00TGoal360
GR00TLong097
OpenVLA-OFTObject861
OpenVLA-OFTSpatial451
OpenVLA-OFTGoal302
OpenVLA-OFTLong069

Task descriptions. Object: 1 = cream cheese into basket; 2 = salad dressing into basket; 4 = ketchup into basket; 5 = tomato sauce into basket; 6 = butter into basket; 8 = chocolate pudding into basket. Spatial: 1 = black bowl next to ramekin onto plate; 3 = black bowl on cookie box onto plate; 4 = black bowl from drawer onto plate; 5 = black bowl on ramekin onto plate. Goal: 0 = open middle drawer; 2 = wine bottle onto cabinet top; 3 = open top drawer and place bowl inside; 6 = cream cheese into bowl. Long: 0 = alphabet soup and tomato sauce into basket; 6 = white mug onto plate and chocolate pudding to the right of the plate; 7 = alphabet soup and cream cheese into basket; 9 = yellow-and-white mug into microwave and close its door.

Please ask further questions if anything remains unclear or inadequately addressed; we welcome further discussion.

Reviewer_yg2iOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SEES, an embodied system designed to continuously improve Vision-Language-Action (VLA) policies using failure feedback. The system first decomposes long-horizon tasks into atomic tasks, uses a learned termination head to monitor subtask completion, and aggregates execution outcomes to identify recurring bottlenecks. It then resets to the simulation state prior to the bottleneck task execution, leverages an LLM to generate success criteria, and updates a shared LoRA adapter for the corresponding skill family using PPO. The authors demonstrate performance improvements across two VLA backbones on RoboCasa365 and four LIBERO suites, including on several tasks that were not involved in RL training. The main contribution of the paper is the integration of failure attribution, training task construction, and policy adaptation into a closed-loop system.

Critical strengths
  1. Compared to applying RL directly to the entire long-horizon task, restoring pre-task states for selected bottlenecks and training atomic skills represents a rational and practical strategy for resource allocation.

  2. Rather than merely detecting failures, the proposed method constructs training tasks, updates policies, and reuses them in subsequent executions, forming a testable and iterative improvement pipeline.

  3. The paper validates two VLA backbones on RoboCasa365 and observes adaptation gains across four LIBERO suites, with the average success rate on LIBERO increasing from 53.1% to 65.4%.

Critical weaknesses
  1. While the main results show that SEES improves performance after incorporating RL training, they do not sufficiently prove that its failure-guided strategy and skill-family design are more effective than simple multi-task RL under the same training budget. Although the authors provide an ablation on target selection, it only covers a single task, leaving the generality of the conclusion unverified.

  2. Table 3 compares independently trained atomic task LoRAs against a shared skill-family LoRA trained on multiple tasks, simultaneously altering both the parameter sharing structure and the training data composition. Consequently, the current results cannot disentangle whether the advantage primarily stems from semantic family partitioning or from joint training across multiple related tasks.

  3. All experiments are conducted in RoboCasa365 and LIBERO simulation environments, whereas key steps of the method rely on precise state resetting and queryable simulator states. In real-world scenarios, challenges such as environmental reconstruction, state estimation errors, and sim-to-real gaps may hinder directed RL training and policy transfer. Section 5 discusses potential avenues for real-world deployment (e.g., integrating real-to-sim techniques), but the full closed-loop system has not yet been verified on a physical robot.

Questions for authors
  1. Under the same total environment interaction budget, what is the gain of SEES compared to uniform multi-task PPO and random target selection? Could you extend the target selection ablation to multiple composite tasks and report the total compute cost, including failure collection and verification?

  2. How much additional benefit does semantic family partitioning provide over a globally shared LoRA or random groupings? If the independent LoRA baseline retains the base policy for untrained tasks, does the transfer comparison conclusion still hold?

  3. Are the composite task completion boundaries used by the termination head derived from existing datasets or additional annotations? Were the fixed atomic task specifications and family routing manually revised? How much total human effort is required to build the entire system?

  4. Are there currently any component-level validations on physical robots, such as termination detection, failure localization, or policy transfer following simulation adaptation? When real states cannot be precisely reset, how does the system construct effective bottleneck training environments and ensure that the updated adapters improve real-robot performance?

2 comments
AuthorsOpenReview ↗

Response to Reviewer yg2i — Part 1/2

Response to Reviewer yg2i

W1 / Q1: Matched-budget controls and compute

Thank you for your constructive review. Manuscript Table 3 tests family-level adapter sharing/transfer; Table 4 tests failure-guided bottleneck selection. We added simple multi-task PPO to both: one shared LoRA across nine tasks (Table 1), and uniform updates across reached subtasks (Table 2). We also added random grouping and expanded target selection to five composite tasks.

Within each ablation, methods share the starting checkpoint, parallel-environment count, rollout epochs/horizon, and termination/loss-masking rules. Total PPO outer steps, summed across adapters, are matched: 90 per method for sharing, and 80 per task/method (two 40-step rounds) for selection, ensuring equal fixed-horizon interaction budgets. Simple multi-task divides this budget across all reached subtasks; Random updates one randomly selected reached subtask per round. Semantic family and Bottleneck exceed their simple multi-task controls by 5.34 and 6.27 percentage points overall.

Adapted results are mean success rates over three independent training seeds; ± is the 95% CI half-width in percentage points. Base receives no further training and has one evaluation run.

Table 1. RoboCasa365 LoRA sharing: expanded manuscript Table 3. Success (%), nine RL-trained and six RL-held-out sibling tasks; 50 rollouts/task. Bold marks best means.

MethodTrained (9)Sibling (6)Overall (15)
Base70.8964.3368.27
Atomic-specific72.37 ± 0.6464.3369.16 ± 0.38
Random groups75.41 ± 1.6965.44 ± 1.7271.42 ± 0.51
Simple multi-task69.11 ± 0.5567.67 ± 2.1968.53 ± 1.15
Semantic family78.30 ± 2.2367.22 ± 0.4873.87 ± 1.44

Table 2. Target selection: expanded manuscript Table 4. Mean success (%) across five composite tasks.

MethodSuccess (%)
Base20.40
Random41.07 ± 2.29
Highest-success21.33 ± 0.57
Simple multi-task50.80 ± 0.99
Bottleneck57.07 ± 1.52

Table 4. Main three-round GR00T evolution on RoboCasa365. Recorded compute is at least 776.13 NVIDIA A800 (80 GB) GPU-hours.

ComponentA800 GPU-hours
PPO training683.35
Failure collection36.02
Composite verification56.76
Reported subtotal776.13

W2 / Q2: Semantic grouping and sibling transfer

Semantic family and Random groups use the same nine tasks, three LoRAs (three tasks each), ranks, and PPO budget; only partitioning changes. The global-sharing control has the same PPO budget. Semantic grouping improves overall success over random grouping by 2.45 points, supporting semantic partitioning beyond joint training alone.

For untrained siblings, Atomic-specific retains Base (64.33%); Semantic family reaches 67.22% (+2.89 points), so transfer is not an artifact of degraded atomic-specific adapters. All three multi-task variants improve sibling success over Base. Simple multi-task has slightly higher sibling success (67.67% vs. 67.22%), but Semantic family achieves the best target-task and overall success, showing more efficient adaptation with positive sibling transfer.

W3 / Q4: Physical robots and imperfect resets

We acknowledge that termination detection, failure localization, and post-adaptation transfer lack physical-robot validation. In a preliminary real-to-sim check, we reconstructed a scene from RoboDojo real-robot SFT images and completed the task by executing the associated SFT actions in simulation. This does not validate the full adaptation loop, transfer back, or real-robot improvement.

Future work will combine reconstruction and state estimation for bottleneck training under imperfect resets, using plausible initial-state distributions rather than exact reconstructed states. Adapter acceptance would require held-out reconstructions and physical-robot trials; real-world gains must be established empirically.

AuthorsOpenReview ↗

Response to Reviewer yg2i — Part 2/2

Q3: Specifications, labels, and human effort

We trained the termination head on existing atomic and composite demonstrations. For each atomic demonstration, we used the final frame as the completion frame. For composite demonstrations, we replayed the trajectories in simulation and ran an LLM-generated success check for each subtask to identify its completion frame. No additional manual frame annotations were needed.

We used LLMs to break composite tasks into subtasks and assign each subtask to a predefined skill family for adapter routing. We did not manually revise the resulting plans or family assignments.

To assess whether the agent could write usable success checks, we compared the checks’ outputs with what happened in a sample of demonstration and training-rollout videos, with LLM assistance. We found no discrepancies in the cases we inspected. These manual spot checks took less than 10 person-hours, excluding system development.

Please ask further questions if anything remains unclear; we welcome further discussion.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Follow-up on Our Response

Dear Reviewer,

Thank you for taking the time to review our paper. As the discussion period draws to a close, we’d appreciate any feedback on whether our response addressed your concerns. We’d be happy to clarify any remaining points.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.