- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper considers the problem of learning a credit assignment model leveraging successful and failed rollouts, as well as human interventions. Given human annotations on episode outcome, human correction, and failure timestep, the proposed approach learns the effectiveness logit and onset logit . These logits are then converted to revise value signals for RL with, e.g., RECAP, IQL and RLToken. The experiment results on Libero-Pro and real robot experiments show that the learned progress estimator outperforms other existing models such as RynnValue-8B, Robo-Dopamine 2.0-8B, Robometer-4B and SVM.
Critical strengths
The paper identifies an important assumption commonly made by progress modeling: progress is often modeled as uniformly across successful demonstrations. This is in general not true for successful but suboptimal trajectories. Moreover, most progress modeling algorithms cannot effectively use failure demonstrations, which this paper aims to address.
Critical weaknesses
-
The scalability of the proposed approach is questionable. The approach presented in this paper relies heavily on human demonstrations (6 seconds per trajectory per annotator). Moreover, the ECR models are trained per-task, meaning that new annotation needs to be obtained for each new task. This makes it difficult to use the approach for general purpose robotics progress modeling.
-
The experiment results are limited. Only 1 out of the 10 tasks from each Libero-Pro suite was evaluated, and over only 2 random seeds. The comparison with most baselines is not entirely fair: most models are held fixed and evaluated zero-shot, whereas the approach uses on-task annotated data to train task specific models. A more fair comparison would require fine-tuning the baseline models on the same set of data.
Questions for authors
- Are ECR models trained per-task?
- If all baselines are trained with the same set of annotated data, are they still performing worse than ECR?
- Since PACT (mentioned in the related work section) is closely related to ECR, why is it not selected as another baseline for comparison?
- Does the observation obtained from a single task from Libero-Pro suites also hold over the entire suite, i.e., all ten tasks?
- In Table 7, why are detection and policy scores for different models computed differently?
- How to speed up or reduce human annotation?
9 comments
Response to Reviewer wxiu [2/4]
3. Coverage of all ten Object tasks and independent training runs — W2 (task coverage and seeds) and Q4
“The experiment results are limited. Only 1 out of the 10 tasks from each Libero-Pro suite was evaluated, and over only 2 random seeds. … Does the observation obtained from a single task from Libero-Pro suites also hold over the entire suite, i.e., all ten tasks?”
ECR’s gains in average success rate and throughput persisted over the complete LIBERO-Pro Object suite. We evaluated all ten tasks under the Position perturbation setting, using three independent training seeds per method and 100 evaluation episodes per task for each trained policy.
RECAP, Robo-Dopamine 2.0 (FT) + RECAP, and ECR + RECAP were compared under matched training and evaluation conditions.
Table R1. Policy performance across all ten LIBERO-Pro Object tasks under Position perturbations. Mean ± sample SD across three independent training runs.
| Method | Success rate (%) ↑ | Speedup (×) ↑ | Throughput (tasks/h) ↑ |
|---|---|---|---|
| RECAP | 63.5 ± 1.8 | 1.42 ± 0.03 | 62.0 ± 1.9 |
| Robo-Dopamine 2.0 (FT) + RECAP | 87.8 ± 1.4 | 1.67 ± 0.02 | 114.0 ± 2.3 |
| ECR + RECAP | 97.4 ± 1.1 | 1.91 ± 0.02 | 154.4 ± 3.0 |
ECR improved the average success rate by 9.6 percentage points and throughput by 35.4% over Robo-Dopamine 2.0 (FT) + RECAP. Its mean success rate and throughput were higher on nine of the ten tasks. Results for individual tasks are provided in Appendix F.1, Table 15.
Response to Reviewer wxiu [3/4]
4. Baselines trained on target-task data — W2 (baseline adaptation) and Q2
“The comparison with most baselines is not entirely fair: most models are held fixed and evaluated zero-shot, … A more fair comparison would require fine-tuning the baseline models on the same set of data. … If all baselines are trained with the same set of annotated data, are they still performing worse than ECR?”
ECR maintains its advantage in transition effectiveness and policy performance after all three baselines are fine-tuned on target task data. We fine-tuned RynnValue [4], Robo-Dopamine 2.0 [5], and Robometer [6] on the same trajectory splits as ECR across four simulation and four real-world tasks, retaining their native training objectives. All models were evaluated on the same held-out autonomous transitions using the same dense transition labels and metrics. Downstream policy comparisons used RECAP with identical learning settings.
Table R2. Transition effectiveness on held-out trajectories across four simulation and four real-world tasks (Appendix Table 16). Four-task macro-averages: five-fold mean ± sample standard deviation (%).
| Method | Sim. AUROC ↑ | Sim. AP ↑ | Sim. Success AUROC ↑ | Sim. Failure AUROC ↑ | Real AUROC ↑ | Real AP ↑ | Real Success AUROC ↑ | Real Failure AUROC ↑ |
|---|---|---|---|---|---|---|---|---|
| RynnValue (FT) | 68.2 ± 0.7 | 70.7 ± 1.2 | 66.1 ± 2.0 | 72.7 ± 1.9 | 68.0 ± 2.0 | 64.2 ± 3.2 | 67.3 ± 1.5 | 68.5 ± 4.6 |
| Robo-Dopamine 2.0 (FT) | 81.9 ± 0.9 | 84.0 ± 1.7 | 79.0 ± 0.9 | 85.0 ± 2.5 | 75.5 ± 1.6 | 73.2 ± 2.3 | 73.7 ± 1.6 | 75.6 ± 3.1 |
| Robometer (FT) | 73.4 ± 1.2 | 75.4 ± 2.2 | 72.2 ± 1.5 | 77.5 ± 3.7 | 68.5 ± 1.8 | 67.0 ± 3.4 | 66.9 ± 2.5 | 74.3 ± 3.0 |
| ECR | 97.1 ± 0.4 | 96.8 ± 0.5 | 97.3 ± 0.4 | 96.2 ± 1.1 | 93.5 ± 2.5 | 89.8 ± 3.8 | 94.5 ± 2.1 | 91.6 ± 4.4 |
ECR outperforms all three fine-tuned baselines in both AUROC and AP across the simulation and real-world evaluations. Its macro-average AUROCs reach 97.1 ± 0.4% and 93.5 ± 2.5%, respectively (Table R2).
Table R3a. Simulation policy performance within RECAP (100 trials/task). Trial counts are per method; task columns report success rates (%, ↑).
| Method | Spatial | Object | Goal | Long | Mean success rate (%) | Speedup on successful episodes (×) | Throughput (tasks/h) |
|---|---|---|---|---|---|---|---|
| RynnValue (FT) | 58.0 | 81.0 | 48.0 | 95.0 | 70.5 | 1.38 | 80.9 |
| Robo-Dopamine 2.0 (FT) | 79.0 | 90.0 | 68.0 | 85.0 | 80.5 | 1.43 | 109.0 |
| Robometer (FT) | 72.0 | 81.0 | 61.0 | 87.0 | 75.3 | 1.26 | 85.9 |
| ECR | 91.0 | 99.0 | 98.0 | 98.0 | 96.5 | 1.66 | 173.5 |
Table R3b. Real-world policy performance within RECAP (15 trials/task). Task columns report success rates (%, ↑).
| Method | Put | Store | Insert | Fasten | Mean success rate (%) | Speedup on successful episodes (×) | Throughput (tasks/h) |
|---|---|---|---|---|---|---|---|
| RynnValue (FT) | 86.7 | 60.0 | 66.7 | 26.7 | 60.0 | 1.27 | 46.1 |
| Robo-Dopamine 2.0 (FT) | 93.3 | 80.0 | 60.0 | 53.3 | 71.7 | 1.33 | 57.4 |
| Robometer (FT) | 80.0 | 66.7 | 60.0 | 40.0 | 61.7 | 1.26 | 41.1 |
| ECR | 100.0 | 86.7 | 80.0 | 66.7 | 83.3 | 1.69 | 82.8 |
ECR achieves higher policy success than all three fine-tuned baselines on each of the eight tasks. Mean success rates reach 96.5% in simulation and 83.3% on the real robot, with the highest aggregate speedup and throughput in both domains (Tables R3a–R3b). These results show that ECR’s advantage after baseline fine-tuning extends from local transition discrimination to downstream policy performance.
For the onset model itself, Table 4 compares ECR variants under matched supervision. Relative to Direct, Event-CDF improves simulation AP from 88.1% to 96.8% and policy success from 88.8% to 96.5%.
References
[4] Huang et al. RynnValue: Scaling Robotic Value Foundation Models with Temporal Distance. arXiv:2608.09853, 2026.
[5] Xu et al. Robo-Dopamine 2.0: History-Conditioned and OOD-Aware Process Reward Modeling for Robotic Manipulation. arXiv:2608.15680, 2026.
[6] Liang et al. Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisons. In RSS, 2026.
Response to Reviewer wxiu [1/4]
We thank the reviewer for the careful review and constructive feedback. We address the questions in detail below.
1. Annotation efficiency and cost — W1 (annotation cost) and Q6
Additional temporal annotation is required only for failed trajectories, which account for 11.1% of our training data. Each task contains 80 successful trajectories, including demonstrations, and 10 failed trajectories (§4.1; Appendix B.1). Each failed trajectory requires one onset timestamp; successful trajectories require no manual temporal annotation.
In our rollout collection, operators record candidate onsets via a hotkey (Appendix B.3). A marker denotes the start of a new ineffective segment; repeated attempts within an uninterrupted ineffective segment do not create new onsets. Successful episodes produce no failure-suffix supervision. For failed episodes, we retain the last valid onset in the last autonomous segment, excluding earlier errors from which execution recovered. We clarify this rule in Appendix B.3.
The reported 6.0 seconds per failed trajectory measures offline labeling of existing data, not additional labeling time after online onset recording. In Appendix B.2, three annotators independently labeled the same 80 failed trajectories with collection-time markers hidden; timing included playback, revisiting, and confirmation.
Onset recording can be integrated into rollout collection, eliminating the need for a separate onset-labeling pass even at larger collection scales. Each failed trajectory contributes one retained timestamp, supervising both onset localization and the final ineffective suffix. These sparse markers enable local credit assignment within failed experience without frame-by-frame training annotation.
2. Task-specific policy post-training — W1 (task-specific training) and Q1
Yes, ECR is trained separately for each task in our experiments. We study robot policy post-training: using target-task demonstrations, autonomous rollouts, and human corrections to efficiently improve a pretrained VLA policy’s execution performance on that task. Target-task policy optimization is also used in representative post-training approaches such as π*₀.₆ / RECAP [1] and RL Token [2], while SARM [3] provides a related example of single-task reward modeling. ECR contributes local credit-assignment signals to this setting.
We evaluated ECR’s generalization to unseen trajectories within each task using five-fold cross-validation. Across four simulation tasks and four real-world tasks, ECR achieved macro-average AUROCs of 97.1 ± 0.4% and 93.5 ± 2.5%, respectively (Appendix Table 16). Test trajectories were excluded from both estimator fitting and normalization statistics, and their onset labels were not used during inference (Appendix C.2). These results support reusing the estimator trained for each task to assess local transition effectiveness in new rollouts, extending the benefit of sparse event supervision beyond the annotated training trajectories.
References
[1] Amin et al. π*₀.₆: a VLA That Learns From Experience. In RSS, 2026.
[2] Xu et al. RL Token: Bootstrapping Online RL with Vision-Language-Action Models. In CoRL, 2026.
[3] Chen et al. SARM: Stage-Aware Reward Modeling for Long Horizon Robot Manipulation. In ICLR, 2026.