- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents RAISE (Risk-Aware Imitation from Successful Experience), a way to use failed robot executions in imitation learning without imitating them. It first adapts a pretrained progress model on successful trajectories, so that it predicts the task stage and how far the task has progressed, and its prediction drops when the robot makes a mistake. The model is then used to find where failed rollouts start to go wrong. These moments, together with the moments just before errors in successful retries, serve as risk points. Similar moments in the existing demonstrations then get higher loss weights, without changing the policy or its action targets. RAISE improves success for several imitation learners in simulation and on a real robot.
Critical strengths
-
Using failures to decide where to focus, rather than what to imitate, is a sensible idea. Successful demonstrations can contain local errors, and failed rollouts are often correct until a decisive mistake, so simply keeping successes and discarding failures misses useful information. RAISE keeps every demonstration and its action target and only changes how much each sample counts, so it never imitates wrong actions.
-
It is a plug-in that works across very different learners. Since it only changes offline loss weights, it needs no change to the policy or to deployment. The same procedure improves BC, Diffusion Policy and two π0.5-based VLAs in 37 of 40 task-learner pairs (Table 1), and it also helps on a real robot.
Critical weaknesses
-
The comparison is not data-matched. The baselines train only on the 300 demonstrations, while RAISE also uses more than ten times as many extra successful trajectories for the progress model, references and calibration, plus several hundred failures (Appendix A.2, StackCube). These extra successes contain actions, so simply adding them to the training set might give a similar gain. This is not tested.
-
The pipeline is complex, and it is not clear which parts matter most. RAISE has many components and needs task-specific stage labels, but its output is only a weight between 1 and 2 per sample. The ablations do not remove the risk term itself (Table 24), so it is hard to tell how much the failure signal contributes.
Questions for authors
- Could you add a baseline that trains on the 300 demonstrations plus the extra clean successes, with uniform weights? If RAISE still clearly wins, the gain would come from the reweighting rather than from the extra data.
- In RAISE, a sample gets a higher weight if its state looks like a risky moment and its action moves the task forward. Could you add a variant that ignores risk and weights samples only by how much their action moves the task forward? If this variant does about as well as RAISE, the failures would add little.
- The progress model is adapted separately for each task. Would one model adapted jointly on all ten tasks work as well? This would make RAISE cheaper to apply to new tasks. This could be very interesting.
8 comments
Response to xAfz (2/4): Adding 1,000 clean demonstrations
We also tested a larger action-training dataset with added clean demonstrations on five tasks. With 1,300 action demonstrations, five-task mean stable success is 21.3% for BC and 41.1% for BC+RAISE; DP and DP+RAISE achieve 25.0% and 36.6%. Results are in Rebuttal Revision, H.2, Table 28; seed-level counts and paired gains are in H.4, Tables 33–34.
For each task, we separately collected 1,000 successful, retry-free trajectories using the same demonstrator, state distribution and action interface as the 300 expert demonstrations. The new trajectories are disjoint from those demonstrations and every auxiliary pool. This experiment tests weighting with 1,000 additional retry-free demonstrations. The all-auxiliary-success experiment in H.2, Table 29 tests two uses of the existing auxiliary successes; it uses a different set of added trajectories.
BC and BC+RAISE (Table 28)
| Task | BC 300 | BC+RAISE 300 | BC 1300 | BC+RAISE 1300 |
|---|---|---|---|---|
| Stack | 0.0 | 9.8 | 2.2 | 10.7 |
| Push | 2.3 | 31.8 | 6.7 | 33.5 |
| Lift | 61.2 | 73.0 | 62.7 | 74.3 |
| Peg | 10.3 | 43.2 | 14.7 | 46.7 |
| Apple | 19.8 | 38.0 | 20.3 | 40.2 |
| Mean | 18.7 | 39.2 | 21.3 | 41.1 |
DP and DP+RAISE (Table 28)
| Task | DP 300 | DP+RAISE 300 | DP 1300 | DP+RAISE 1300 |
|---|---|---|---|---|
| Stack | 39.2 | 43.8 | 41.5 | 44.3 |
| Push | 0.0 | 0.0 | 3.0 | 8.7 |
| Lift | 24.7 | 52.8 | 23.8 | 53.3 |
| Peg | 0.2 | 10.2 | 2.3 | 12.3 |
| Apple | 52.2 | 63.3 | 54.3 | 64.5 |
| Mean | 23.2 | 34.0 | 25.0 | 36.6 |
BC and DP use uniform weights; the suffix gives the action-demonstration count. Entries are three-training-seed mean Stable Success@3 (%); Mean weights the five tasks equally. Stack, Push, Lift, Peg and Apple denote StackCube, PushCube, LiftPegUpright, PegInsertionSide and PlaceApple, with horizons 250/50/200/100/100 environment steps. Seeds 0/1/2 each use the same 200 evaluation initial conditions. Stable success requires three consecutive evaluation states satisfying the completion criterion.
BC+RAISE minus BC and DP+RAISE minus DP (Table 34)
Action-demonstration counts are matched within each pair. Gains are in percentage points.
| Learner / action N | Seed 0 | Seed 1 | Seed 2 | Mean±SD |
|---|---|---|---|---|
| BC / 300 | 27.0 | 15.1 | 19.2 | 20.4±6.0 |
| BC / 1,300 | 19.6 | 19.0 | 20.7 | 19.8±0.9 |
| DP / 300 | 11.3 | 12.8 | 8.3 | 10.8±2.3 |
| DP / 1,300 | 12.1 | 10.8 | 12.0 | 11.6±0.7 |
For each seed, we compute within-task differences and average the five tasks equally. Mean±SD summarizes these three seed-level gains using sample SD. The 600 episodes per configuration comprise three models tested on the same 200 initial conditions. Differences use unrounded rates.
Across data scales, policy input representations, architecture, initialization, normalization, update budget and evaluation protocol stay fixed. BC uses 10,000 updates and DP 100,000, with batch size 1,024. Within each task and data scale, BC/BC+RAISE and DP/DP+RAISE share action demonstrations, targets, batch order and execution settings. Push's execution library expands from 300 to 1,300 trajectories, with its neighbor count and fusion rule fixed; each baseline–RAISE pair uses the same library.
Auxiliary progress models, references, calibration sets and risk anchors stay frozen across data scales. RAISE also uses auxiliary success and failure inputs, so matched action-demonstration counts do not imply equal total data budgets.
Adding action data improves the five-task means, with task-level exceptions: Lift DP drops from 24.7% to 23.8%. Push DP and DP+RAISE both achieve 0% with 300 demonstrations; with 1,300 they reach 3.0% and 8.7%. At both data scales, the paired five-task gains of BC+RAISE and DP+RAISE are positive for each training seed. Protocols are in H.1–H.2.
Response to xAfz (3/4): Risk, weight assignment and component ablations
We tested the requested control by fixing the progress model and setting the risk factor to 1, with the action-advancement gate and weight schedule unchanged. Rebuttal Revision, E.5, Table 24(a) reports these component results.
BC: 11 configurations
| Variant | Stack | Push | Lift | Peg | Apple | Mean |
|---|---|---|---|---|---|---|
| BC (uniform) | 0.0 | 2.3 | 61.2 | 10.3 | 19.8 | 18.7 |
| Retry risk only | 0.0 | 0.0 | 55.7 | 6.3 | 14.2 | 15.2 |
| Natural-failure risk only | 4.2 | 10.5 | 67.5 | 34.2 | 30.2 | 29.3 |
| No retry supervision | 2.7 | 8.3 | 58.5 | 31.5 | 25.7 | 25.3 |
| Progress-only localization | 0.0 | 0.0 | 58.2 | 8.7 | 16.5 | 16.7 |
| No advancement gate | 5.7 | 26.3 | 69.7 | 40.2 | 34.3 | 35.2 |
| Progress-only weighting | 3.5 | 15.5 | 64.0 | 20.2 | 27.2 | 26.1 |
| Global shuffle | 2.2 | 12.5 | 62.5 | 15.2 | 21.2 | 22.7 |
| Conditional shuffle | 2.7 | 10.7 | 61.5 | 16.5 | 22.5 | 22.8 |
| No-retry-risk shuffle | 2.3 | 9.7 | 60.7 | 14.7 | 20.5 | 21.6 |
| BC+RAISE | 9.8 | 31.8 | 73.0 | 43.2 | 38.0 | 39.2 |
DP: 11 configurations
| Variant | Stack | Push | Lift | Peg | Apple | Mean |
|---|---|---|---|---|---|---|
| DP (uniform) | 39.2 | 0.0 | 24.7 | 0.2 | 52.2 | 23.2 |
| Retry risk only | 36.5 | 0.0 | 20.5 | 0.0 | 44.5 | 20.3 |
| Natural-failure risk only | 40.3 | 0.0 | 47.5 | 6.8 | 56.3 | 30.2 |
| No retry supervision | 39.8 | 0.0 | 46.8 | 5.3 | 42.2 | 26.8 |
| Progress-only localization | 38.2 | 0.0 | 22.7 | 0.0 | 48.7 | 21.9 |
| No advancement gate | 39.7 | 0.0 | 48.5 | 8.7 | 59.2 | 31.2 |
| Progress-only weighting | 41.2 | 0.0 | 30.5 | 4.7 | 56.5 | 26.6 |
| Global shuffle | 38.5 | 0.0 | 25.7 | 2.2 | 53.2 | 23.9 |
| Conditional shuffle | 40.5 | 0.0 | 27.2 | 1.2 | 50.7 | 23.9 |
| No-retry-risk shuffle | 40.7 | 0.0 | 23.7 | 1.5 | 49.2 | 23.0 |
| DP+RAISE | 43.8 | 0.0 | 52.8 | 10.2 | 63.3 | 34.0 |
Entries are mean Stable Success@3 (%) across training seeds 0/1/2, with the same 200 evaluation initial conditions per model. Mean weights five tasks equally. StackCube, PushCube, LiftPegUpright, PegInsertionSide and PlaceApple use horizons 250/50/200/100/100 environment steps. Stable success requires completion at three consecutive evaluation states. Each configuration uses 300 action demonstrations, 10,000 BC or 100,000 DP updates, and batch size 1,024.
Control definitions
Progress-only weighting sets the sample-weight risk factor to 1 with the progress model fixed. Progress-only localization retains risk anchors and matching, using progress evidence alone to locate failures.
The two single-risk-source variants retain the full progress model and advancement gate; full RAISE takes the larger of the two risks. No retry supervision sets the retry-loss coefficient to zero, readapts the model, and rebuilds references, calibration, anchors and weights. No advancement gate sets the advancement factor to 1 while preserving anchors and matches.
Global shuffling preserves weight values and multiplicities but permutes their assignment to samples. Conditional shuffling constrains permutations by stage, within-stage progress and advancement. No-retry-risk shuffling exchanges weights only among samples with positive advancement and zero successful-retry risk. All controls retain their learner's action representation, controller, per-sample loss and weight-application schedule.
Paired gains of BC+RAISE or DP+RAISE (Table 34)
| Learner / control | Gains: seeds 0 / 1 / 2 | Mean±SD (pp) |
|---|---|---|
| BC / progress weighting | 12.7 / 12.4 / 14.2 | 13.1±1.0 |
| BC / global shuffle | 15.4 / 16.8 / 17.2 | 16.5±0.9 |
| DP / progress weighting | 6.3 / 11.1 / 5.0 | 7.5±3.2 |
| DP / global shuffle | 9.1 / 13.8 / 7.5 | 10.1±3.3 |
For each seed, we subtract the control's success rate from its learner's full RAISE success rate within each task, then average five tasks equally. SD is the sample standard deviation of those three seed-level gains; calculations use unrounded rates. Risk-factor gains are positive for all three five-task seed averages in both learners. For DP, progress-only weighting adds 3.3±1.7 percentage points over uniform training; the risk factor adds a further 7.5±3.2.
BC+RAISE and DP+RAISE have the highest five-task means in their respective tables. Retry-risk-only and progress-only localization have lower means than uniform training. All DP variants achieve 0% on Push at 50 steps. These mechanism results cover BC and DP on five tasks. Definitions, seed counts and paired statistics are in E.5, Table 24(a), and H.4, Tables 32–34; the common protocol is in H.1.
Response to xAfz (1/4): Training on all auxiliary successes
Thank you for proposing this control. We trained BC and DP with uniform weights on the 300 expert demonstrations plus every auxiliary successful trajectory, using three update budgets. The protocol and results are in Rebuttal Revision, H.2, Table 29.
Within the rebuttal time and compute budget, we use behavior cloning (BC) and Diffusion Policy (DP) on five tasks for detailed data and weight-assignment controls. RAISE weights each learner's per-sample loss, including supervised fine-tuning losses for π0.5 and OpenVLA-7B (Eq. 2; A.4–A.5). Table 24(a) reports BC/DP component controls; Table 24(b) additionally reports π0.5 component ablations on three tasks. Table 1 provides paired π0.5 results, and Table 2 reports the GTP-FA-OpenVLA-7B+RAISE combined system.
Successful-trajectory composition
Stack denotes StackCube; Lift denotes LiftPegUpright. The four auxiliary success pools are trajectory-disjoint and have compatible action records. Each complete trajectory is counted once, including trajectories with multiple retry events. The separately collected 1,000 clean demonstrations in the 1,300-demonstration study are excluded. Development and final-evaluation trajectories are also disjoint.
Update-budget comparison (Table 29)
The uniform columns report BC or DP trained on all N successful trajectories. The last column reports BC+RAISE or DP+RAISE, respectively, using 300 action demonstrations at U. Values are Stable Success@3 (%), mean±sample SD across training seeds 0/1/2. Each model uses the same 200 evaluation initial conditions per task. Stack and Lift have horizons of 250 and 200 environment steps. Stable success requires the completion criterion at three consecutive evaluation states within an episode.
U is 10,000 BC updates or 100,000 DP updates; batch size is 1,024. Within each learner, input and action representations, initialization, normalization, optimizer and execution control are matched. Normalization uses the 300 expert demonstrations.
Expanded-data runs retain optimizer state through 4U; DP also retains EMA state. After U, BC keeps learning rate 0.0003; DP warms up for 500 updates to 0.0001 and then cosine-decays through the extension. Checkpoints at U, 2U and 4U were fixed before evaluation: 12 independent training paths yield 36 checkpoint evaluations and 7,200 episodes.
At U, BC+RAISE and DP+RAISE exceed their uniform counterparts at 4U by 2.0/2.3 percentage points on Stack and 4.8/12.7 on Lift. Differences use unrounded rates. Seed variability is reported in the table, including the 9.5-point SD for Stack DP+RAISE.
The experiment compares two uses of the same auxiliary successes. RAISE additionally uses natural failures to compute offline weights: Stack has 489 natural failures, 352 of which produce confirmed anchors. The methods therefore have different total auxiliary inputs. The fixed-progress-model control in E.5, Table 24(a) isolates the risk factor's contribution. Training and evaluation details are in H.1–H.2.