acceptodds
Under review as a conference paper at ICLR 2027

ARS: Agentic Reward System for Robot Learning

10 views

Abstract

Progress reward modeling is the problem of estimating how a robot's behavior changes task progress over time. Reliable estimation requires distinguishing meaningful state changes from failed attempts and task-irrelevant actions. We introduce the Agentic Reward System (ARS), an inference framework for progress reward modeling with general-purpose vision-language models (VLMs), without additional reward-model training. Given an offline trajectory and a task instruction, ARS uses adaptive visual inspection for both event proposal and verification. A subagent proposes a task-relevant event timeline, which a primary agent verifies and revises before estimating per-frame progress. ARS can incorporate optional terminal outcome labels and visual references to inform its judgments. It can also audit progress estimates from external reward models. We evaluate ARS with a 27B VLM on a controlled semantic-mismatch benchmark and downstream policy learning in simulation and on a real robot. The benchmark reveals that several evaluated reward baselines assign spurious progress to wrong-object manipulation even in simple pick-and-place scenes. ARS better suppresses these errors and outperforms these baselines in simulation policy learning. We further demonstrate that ARS supports long-horizon policy learning from mixed-quality offline experience on real-robot multi-screw fastening in a full-scale laboratory replica of an industrial washing-machine assembly line. These results suggest that structured inference and verification can improve the usefulness of general-purpose VLMs for robot reward modeling. Our code will be publicly available.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Reward Modeling, Robot Learning, Vision-Language Models, Agentic Reasoning, Task Progress Estimation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Sk7kOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper creates a harness for a 27b Qwen model to predict task specific rewards. Their method follows the process of proposing and then revising sub tasks, and then assigning progress scores. They compare to other progress based reward methods (such as robometer) on different reward metrics across a semantic mismatch benchmark and then use their reward for chunk level bc filtering for policy training.

Critical strengths

The authors include multiple strong current progress reward baselines for their simulation results. I also found that the semantic mismatch analysis is strong as for example a robometer assigned progress under wrong object interactions while having high temporal correlation.

Critical weaknesses

I found the pi.5 simulation results to be surprisingly low. Pi 05 fine tuned on the libero datasets as reported on the open pi repository reaches over 92% success rates. Can the authors clarify why these numbers are different, especially for the success filtered BC?

For the real world results, the other reward models are not included. Can you share results with at least success only rollout filtering and if applicable the strongest baselines? These would help establish ARS’s advantage over simple selection methods.

Additionally for each reward to be calculated, it requires a large amount of tokens through a 27b language model. The paper reports token counts, but Including runtime comparisons between robometer per action chunk / trajectory vs ARS would help clarify the practical costs and real world feasibility.

I found the harness description in the main paper hard to follow and unclear on the exact method and tools used by their base model to predict the final progress score. These details are mostly clarified in the appendix but it would be helpful to move them into the main paper.

For their experiments, the rollout collection procedure for the fixed behavior policy is unclearly specified. What are the details of your behavior policy used for the experiments, and how was it trained?

Questions for authors

Included in weaknesses section

5 comments
AuthorsOpenReview ↗

Response to Reviewer Sk7k

Thank you for the helpful questions about practical costs and experimental details.

W1/W5

This is a perceptive observation. OpenPI’s >92% result [1] uses the full LIBERO demonstration dataset. Our behavior policy is obtained by fine-tuning pretrained $ \pi_{0.5} $ on three demonstrations per task from LeRobot [2], then held fixed to collect rollouts. This fixed policy achieves 54.6% success (546/1,000 trials) under the same evaluation protocol as Table 2. The collected dataset contains 53.8% successful trajectories (2,204/4,096). Limited-demonstration SFT initialization also appears in $ \pi_{\mathrm{RL}} $ [3] and RedFlow [4].

All Table 2 policies use the same pretrained π0.5\pi_{0.5} initialization and learn from the shared rollout pool, with method-specific filtering and no demonstration data. We will clarify these distinctions and the behavior-policy setup in the final version.

References:

W2

Please see Point 3 of our Overall response (2/2) for the real-robot results with success-only rollout filtering and our strongest simulation reward baseline, TOPReward-32B (Table R5).

W3

Please see Point 1 of our Overall response (1/2) for the per-trajectory runtime comparison between Robometer and ARS (Table R1).

W4

We agree that the main-text description should be clearer. Due to space constraints, we placed the detailed workflow and tool descriptions in Appendix A. In the final version, we will include a concise summary in the main text, explaining how visual inspection, event verification, and progress editing produce per-frame scores. We have also uploaded the code to help readers understand and reproduce the workflow.

Specifically, we will add the following summary to Section 3:

Given a trajectory and instruction, the subagent uses inspect_visual to examine temporal windows and spatial crops, recording events with frame intervals and supporting evidence. In a separate context, the primary agent gathers additional evidence to revise and confirm these events. It then assigns numerical progress based on achieved subgoals, including partial completion and regressions, and calls edit_progress_segments to interpolate between specified endpoint values into per-frame scores. Using read_progress and summarize_progress, the agent examines the resulting curve and revises unsupported changes. The harness enforces evidence coverage and event confirmation, while validate_progress checks frame assignments and numerical bounds before export.

Area_Chair_ViMNOpenReview ↗

Reminder to respond to the authors’ rebuttal

With one day left in the rebuttal period, please read the authors’ response and leave a comment. It would help the authors to know which of your concerns they have addressed and which questions remain open, so they can make the most of the time left.

Thanks for your time. Your constructive feedback helps the authors improve their work and strengthens the ICLR community.

Best regards,

Area Chair

Thank you for the response

For W1, why do you think the performance is still very low even when you finetune pi on the much larger success filtered dataset? Reading reviewer JqCP's review, I also agree that including a conventional offline RL baseline with binary success reward such as IQL or other similar methods would be an important comparison.

This also raises another question for me regarding comparisons. The behavior policy used to collect the offline data already achieves 54.6% success, compared with 57.9% for ARS and 44.2% for vanilla BC. ARS improves over rollout success filtered BC by 13.7%, but its improvement over the behavior policy is only 3.3%, with a very large rollout budget. Why are the downstream policies not initialized from the behavior checkpoint?

I also thank the authors for including the timing comparisons, inclusion of the code, as well as real world baselines. These address my concerns on these items directly. Even though the time is much slower than other baselines, I think it is fair to include this as a limitation and assess the work without considering the slow inference speed.

Reviewer_29H6OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces the Agentic Reward System (ARS), an offline framework for estimating robot task progress using general-purpose vision-language models without additional reward-model training. A subagent proposes a timeline of task-relevant events, and a primary agent verifies and revises these events in a separate reasoning context before constructing a dense progress curve. Both agents can adaptively inspect temporal windows and spatial crops. The framework also supports terminal outcome labels, visual references, and auditing external reward predictions.

The contributions are the structured inference workflow, a controlled wrong-object semantic-mismatch benchmark, and downstream validation through progress-based action-chunk selection for behavior cloning. Using Qwen3.6-27B, ARS improves LIBERO-Long success from 44.2% for unfiltered BC to 57.9%, compared with 47.6% for the strongest evaluated reward baseline. On real-robot three-screw fastening, success increases from 58.0% for demonstration-only BC to 72.0% with ARS-selected rollouts and 81.5% with additional visual references. Ablations investigate visual inspection, context separation, backbone choice, and thinking mode.

Critical strengths
  1. Meaningful improvements in downstream policy learning. ARS outperforms unfiltered behavior cloning by 13.7 percentage points and the strongest evaluated reward baseline by 10.3 points on LIBERO-Long. ARS and TOPReward-32B retain similar fractions of the data, suggesting that the improvement reflects better selection rather than simply more aggressive filtering. The final-checkpoint results in Appendix E.4 preserve a substantial advantage, strengthening the evidence beyond best-checkpoint reporting. The real-robot results, based on 200 complete-task trials per configuration, provide additional support for the practical value of the approach.

  2. Useful evidence for organizing VLM inference around explicit events and verification. ARS connects progress estimates to an editable event timeline and targeted inspection of visual evidence. The fixed-backbone ablations in Table 4 show benefits from adaptive inspection and further improvement when verification occurs in a separate context. This provides an informative demonstration that the inference workflow can improve a general-purpose VLM's usefulness for offline robot data curation without additional reward-model training.

Critical weaknesses
  1. The scope of the reward-model comparison depends on the selection protocol. The shared rule, pt+10−pt>0p_{t+10}-p_t>0, is clearly motivated in Appendix E.4 and reduces dependence on numerical calibration. Nevertheless, its interaction with temporal resolution substantially affects the selected datasets: Appendix E.5 reports zero increments for 76.7% and 79.5% of chunks for LRM and R²VLM, respectively. These results establish ARS's advantage under the stated local-selection protocol, but leave uncertainty about whether the ranking persists under other reasonable chunk horizons or selection rules. Appendix C reports unsuccessful attempts to improve Robometer through sliding windows and larger frame counts, which partially addresses this concern. A targeted sensitivity analysis would strengthen the broader comparative conclusion. The real-robot study also lacks a competing reward-based selector.

  2. The incremental benefit over existing structured inference and the accuracy of verification remain incompletely characterized. Table 4 provides meaningful evidence that the complete ARS workflow improves downstream performance, including a 6.1-point advantage over same-context verification. However, the paper does not compare against ROVER, a directly relevant training-free structured video-reasoning method. Additionally, Appendix B illustrates an event-boundary correction without reporting aggregate measurements of corrected claims, newly introduced errors, or boundary accuracy. Table 15's near-saturated semantic-mismatch results, which the authors acknowledge, offer limited additional discrimination between workflow variants. A structured inference baseline and aggregate verification analysis would clarify what ARS adds and how reliably its verification stage improves event judgments.

Questions for authors
  1. Requested evidence on selection robustness: How sensitive is the Table 2 ranking to chunk horizon and selection rule, particularly for models with coarse or repeated progress predictions? Do alternative reasonable settings or matched-retention comparisons preserve ARS's advantage?

  2. Clarification on baseline coverage: What motivated excluding ROVER, a directly relevant training-free structured video-reasoning method, from the experiments? Is a matched-backbone comparison available that would clarify ARS's incremental benefit?

  3. Requested evidence on verification accuracy: Beyond the example in Appendix B, are aggregate measurements available showing how often verification corrects event claims or introduces new errors, and whether it improves event-boundary accuracy? Such evidence would help connect the downstream gains to the proposed verification mechanism.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 29H6

We appreciate the reviewer’s careful assessment of selection robustness and verification.

W1/Q1

Please see Point 2 of our Overall response (2/2) for the matched-retention comparison and the additional chunk-horizon evaluation (Tables R3 and R4).

The additional real-robot comparison with TOPReward is reported in Point 3 of our Overall response (2/2) (Table R5).

W2/Q2

With the same Qwen3.6-27B backbone, ARS outperforms ROVER by 14.8 percentage points on LIBERO-Long (57.9% vs. 43.1%). Please see Point 1 of our Overall response (1/2) for detailed comparisons and inference costs.

Q3

Among 500 sampled revised events, verification corrected 63 content errors and introduced 12. Please see Point 4 of our Overall response (2/2) for the full breakdown and boundary-error analysis.

Area_Chair_ViMNOpenReview ↗

Reminder to respond to the authors’ rebuttal

With one day left in the rebuttal period, please read the authors’ response and leave a comment. It would help the authors to know which of your concerns they have addressed and which questions remain open, so they can make the most of the time left.

Thanks for your time. Your constructive feedback helps the authors improve their work and strengthens the ICLR community.

Best regards,

Area Chair

Dear Authors, How did you set up ROVER and GVL on Qwen3.6-27B? Official code, adapted prompts, any tuning? Also, what does random chunk selection get at the same retention? Right now it’s hard to tell if landing below vanilla BC means these methods are just weak here or the port didn’t go well.

Reviewer_JqCPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes ARS, a training-free agentic framework for per frame progress estimation on offline robot trajectories. A subagent proposes an event timeline with adaptive visual inspection. A primary agent verifies it in a separate context and builds the progress curve. A harness enforces evidence coverage. Progress is used to select action chunks for behavior cloning (BC). ARS is evaluated using a general-purpose VLM model. A new semantic-mismatch benchmark in ManiSkill is constructed, which reveals that ARS suppresses spurious progress assignment to semantically incorrect trajectories compared to other reward systems. Additionally, experiments on LIBERO-Long manipulation tasks show a clear gain over unfiltered BC and other baselines. Finally, ARS is evaluated on a real-robot screw fastening task, raising success to 81.5% when visual reference demonstrations are available and to 70.4% based on trajectories alone, compared to demo only BC that achieves a 58% success rate.

Critical strengths
  • The proposed training-free method shows improved performance over the baselines considered, while supporting ablations are provided, spanning base model size, diverse workflows and prompting styles.
  • Besides progress estimation, ARS can also act as an external auditor for other reward models.
  • The results indicate that separate context verification beats a same context control at a comparable budget.
  • The method is evaluated on a real robot, where the gain over BC is confirmed.
  • Training data exposure of baselines, tolerance sensitivity, final checkpoint results and a null outcome conditioning result are all reported and the overhead is acknowledged.
Critical weaknesses
  1. The main baselines considered are designed mostly for online reward generation, online RL or causal prefix scoring, while ARS is offline, non-causal and much more expensive than single-pass models. This raises fairness comparison concerns, while cost is quantified only against TOPReward. This is a major shortcoming.
  2. GVL, the closest offline training-free baseline, has no reported cost and is run with all frames in one prompt instead of its original subsampling.
  3. In addition, WARP-RM, ARM and SARM are cited for weighted BC, but they are not compared and their exclusion is not explained.
  4. The zero-threshold rule for the threshold function gives different retention across methods. Some baselines produce mostly zero increments, partly because of their output form. The rule is justified, but a matched retention check is missing.
  5. Instruction hints are given to all methods. It would be important to provide results in the absence of hints.
  6. The contribution of the reward verification harness is not sufficiently evaluated. A relative ablation would show how much performance drops if the harness is not considered.
  7. The real-robot experiment results reported in Table 5 lack labeling baselines and success-only BC although outcome labels exist, mainly showing that filtering helps rather than ARS being advantageous over other curation signals.
  8. Progress is evaluated quantitatively only on the short mismatch trajectories. On LIBERO in particular, it is evaluated only through chunk selection plus one qualitative example.
Questions for authors
  1. Please elaborate on the fairness concern raised in the Weaknesses section above.
  2. How does GVL with its original protocol perform in Table 2, and what is its cost relative to ARS?
  3. Do the LIBERO conclusions hold under a matched retention of chunks?
  4. How do the baselines perform without the appended instruction hints?
  5. How do labeling baselines perform with the same references?
3 comments
AuthorsOpenReview ↗

Response to Reviewer JqCP

We thank the reviewer for the constructive suggestions on baseline comparisons and evaluation.

W1/Q1

Our comparison evaluates reward systems for offline data curation, where complete trajectories are available. It includes GVL, which also uses future frames. Comparisons between these temporal-access settings also appear in Robometer and ROVER papers. We report actual inference costs in Table R1 of our Overall response (1/2), since causality alone does not determine runtime.

W2/Q2

The requested 30-frame shuffled GVL results, including performance and inference cost relative to ARS, are reported in Point 1 of our Overall response (1/2) (Table R1).

W3

WARP-RM, ARM, and SARM are valuable approaches to reward-guided policy learning, and we cite them as precedents for weighted BC. Their reported pipelines train reward models on task-specific robot data. We did not include them because our comparison focuses on general-purpose reward systems applied without additional reward-model training on the target tasks.

W4/Q3

Please see Point 2 of our Overall response (2/2) for the matched-retention comparison (Table R3).

W5/Q4

Following your suggestion, we evaluate ARS and the two strongest reward baselines in Table 2 without the appended hints, prioritizing these methods due to the limited discussion time. All other settings remain unchanged.

Table R6. LIBERO-Long success rates with and without appended instruction hints.

MethodWith hints (%)Without hints (%)
Robometer46.8 ± 0.946.2 ± 0.7
TOPReward-32B47.6 ± 0.746.9 ± 0.7
ARS57.9 ± 0.456.8 ± 0.5

ARS retains a 9.9-point advantage over TOPReward-32B without hints, compared with 10.3 points with hints.

W6

We address this concern in three parts: event-level verification accuracy (Point 4 of our Overall response (2/2)), downstream workflow ablations (Table 4 of the submission), and the role of mandatory harness checks.

The mandatory harness checks enforce the intended workflow; we do not claim them as a standalone contribution. We disable the mandatory harness checks applied when the primary agent confirms events, while keeping the model, prompts, tools, and workflow unchanged. This yields 57.3 ± 0.5% success on LIBERO-Long with nearly unchanged inference cost. Performance remains close to full ARS (Table 4), suggesting a limited incremental effect of these checks when the verification workflow is retained.

W7/Q5

Point 3 of our Overall response (2/2) adds success-only BC and TOPReward comparisons, including settings with identical outcome labels or visual references (Table R5).

W8

We directly evaluate progress predictions on LIBERO-Long against the privileged progress reference in Appendix E.3. Using H=10H=10, we measure agreement on positive versus nonpositive progress changes within each trajectory and report the average across trajectories.

Table R7. Chunk-level progress-increase agreement on LIBERO-Long (H=10H=10).

MethodAgreement ↑
LRM0.416
R²VLM0.393
SOLE-R10.633
Robometer0.541
GVL, 30 frames, shuffled0.379
GVL, full frames, shuffled0.423
TOPReward-8B0.576
TOPReward-32B0.619
ROVER0.598
ARS0.814

ARS shows the highest agreement with the privileged reference on chunk-level progress-increase decisions.

Area_Chair_ViMNOpenReview ↗

Reminder to respond to the authors’ rebuttal

With one day left in the rebuttal period, please read the authors’ response and leave a comment. It would help the authors to know which of your concerns they have addressed and which questions remain open, so they can make the most of the time left.

Thanks for your time. Your constructive feedback helps the authors improve their work and strengthens the ICLR community.

Best regards,

Area Chair

I thank the authors for their detailed reply. My main concerns have been addressed. I have no further questions.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Overall response (1/2)

We thank the reviewers for their constructive feedback. Below, we address the common concerns raised in the reviews. We will incorporate the clarifications and additional results presented in this discussion into the final version.

To help readers understand and reproduce our method, we have uploaded the code for ARS and semantic-mismatch benchmark generation, together with usage examples and essential output artifacts.

1. Additional baselines and inference cost

Following the reviewers’ requests, we add 30-frame shuffled GVL (JqCP) and ROVER (29H6), and report inference costs (JqCP, Sk7k). GVL, ROVER, and ARS use Qwen3.6-27B, while TOPReward uses Qwen3-VL-32B.

The LIBERO-Long results in Table R1 follow Section 4.2 and Appendix E.4 of the submission. We report per-trajectory model queries, input/output tokens (K), and runtime (s). Inference costs are averaged over all 4,096 trajectories, measured in BF16 on NVIDIA H20 GPUs with one trajectory per GPU and prefix caching enabled. Runtime covers end-to-end annotation, including visual processing.

Table R1. LIBERO-Long performance and inference cost.

MethodFramesSuccess (%)Retained (%)QueriesInput (K)Output (K)Time (s)
Robometer846.8 ± 0.989.510.6800.48
GVL30, shuffled32.2 ± 0.817.612.42.182.6
GVLFull, shuffled34.1 ± 1.114.8131.316.6536.9
ARS w/o verification & inspectionFull45.4 ± 0.378.17.3242.710.1437.2
TOPRewardFull47.6 ± 0.767.9390.32866.301940.7
ROVERFull43.1 ± 0.657.5391.7182.023.0981.3
ARSFull57.9 ± 0.466.924.9980.621.1772.8

Table R2. Semantic-mismatch evaluation with a matched backbone.

MethodVOC ↑FPSR ↑CMPG ↑
ROVER0.9340.8040.535
ARS0.9950.9180.904
  • ROVER comparison. We agree that ROVER is a directly relevant baseline. With the same backbone, ARS achieves higher downstream policy success and scores higher on all three semantic-mismatch metrics (Tables R1 and R2). ROVER’s compact subtask context may omit information relevant to the overall event sequence. ARS maintains an editable event timeline and explicitly verifies and revises event claims against visual evidence.

  • GVL comparison. ARS maintains higher policy success than GVL under both the requested 30-frame shuffled setting and the full-frame setting. For a complementary comparison, ARS w/o verification & inspection and full-frame GVL both rely on the initially supplied trajectory frames without additional visual inspection. This ARS variant achieves higher policy success with shorter runtime (Table R1), and outperforms the GVL-style baseline on all three semantic-mismatch metrics (Tables 3 and 15 of the submission). These results suggest a benefit from organizing progress estimation around an event timeline rather than frame-wise descriptions and progress predictions.

  • Runtime relative to TOPReward and ROVER. Under the full-frame settings in Table R1, ARS achieves higher policy success with shorter end-to-end annotation time than TOPReward and ROVER.

  • Robometer comparison. Robometer is substantially faster, while ARS achieves higher downstream policy success and better suppresses false progress on wrong-object interactions. We also evaluate GVL, ROVER, and TOPReward as reward-training-free baselines in the same offline data curation setting.

  • Annotation throughput. In additional deployment runs on a single GPU, ARS with FP8 quantization and two concurrent trajectories achieved 3.2× the annotation throughput of the BF16, single-trajectory configuration in Table R1.

AuthorsOpenReview ↗

Overall response (2/2)

2. Robustness and fairness of the selection protocol

Following the reviewers’ requests (JqCP, 29H6), we examine both per-task matched retention and sensitivity to the chunk horizon.

Matched retention. For baselines with low retention under the original selection rule, we retain the chunks with the largest Δt=pt+10−pt\Delta_t=p_{t+10}-p_t within each task, matching the number retained by the original ARS selection. Ties at the cutoff are broken by uniform random sampling with a fixed seed. We additionally include uniform random chunk selection within each task, matching the same retained counts. Policy training and evaluation follow Appendix E.4 of the submission.

Table R3. LIBERO-Long performance with per-task retention matched to the original ARS selection.

MethodSuccess (%)Retained (%)
Random40.5 ± 0.666.9
LRM43.3 ± 0.466.9
R²VLM42.1 ± 0.566.9
GVL, 30 frames, shuffled41.5 ± 0.866.9
GVL, full frames, shuffled42.7 ± 0.366.9
ROVER46.0 ± 0.466.9
ARS (original selection)57.9 ± 0.466.9

Matched retention improves the reward baselines’ mean performance, but ARS maintains a clear advantage, suggesting that retained data quantity alone does not explain its gains.

Chunk-horizon sensitivity. We additionally evaluate H=20H=20 using the same offline rollout pool. We set the policy action-chunk horizon to 20 and, for reward-based methods, reselect chunks using pt+20−pt>0p_{t+20}-p_t>0. We include LRM and R²VLM to directly address the concern about coarse or repeated progress predictions.

Table R4. LIBERO-Long performance and retention under different chunk horizons.

MethodSuccess, H=10H=10 (%)Success, H=20H=20 (%)Retained, H=10H=10 (%)Retained, H=20H=20 (%)
Vanilla BC44.2 ± 0.339.7 ± 0.4100.0100.0
LRM38.6 ± 0.535.1 ± 0.713.718.4
R²VLM41.5 ± 0.436.9 ± 0.513.614.0
TOPReward-32B47.6 ± 0.743.5 ± 0.967.970.1
ARS57.9 ± 0.452.1 ± 0.366.968.3

The longer horizon increases retention for all reward-based methods, but absolute success decreases and the ranking remains unchanged. ARS maintains its advantage over all evaluated baselines, including TOPReward at similar retention.

3. Additional real-robot comparisons

Following the reviewers’ requests (JqCP, 29H6, Sk7k), we add success-only BC and TOPReward-32B, the strongest reward baseline in our LIBERO-Long experiments. TOPReward follows the same stage-dependent chunk-selection rule and policy-training protocol as ARS (Section 4.4 and Appendix G). Both methods receive identical outcome labels or visual references in the corresponding conditions.

Table R5. Real-robot comparisons.

ConfigurationAdditional informationSuccess (%)
Demo-only BC—58.0
Demo + success-only BCOutcome labels56.5
TOPReward—63.0
ARS—72.0
TOPReward + outcomeOutcome labels61.5
ARS + outcomeOutcome labels77.0
TOPReward + visual referencesVisual references65.0
ARS + visual referencesVisual references81.5

ARS’s advantage over success-only BC suggests that local progress provides useful selection information beyond terminal outcomes: successful trajectories can contain unproductive actions, while failed trajectories can retain useful segments. Its higher observed success than TOPReward under matched-information settings further supports its utility for real-robot data curation.

4. Event verification accuracy

Following the reviewers’ requests (JqCP, 29H6), we randomly sample 50 events revised by the primary agent from each LIBERO-Long task. We manually assess their content and boundaries using privileged simulator information as the reference, evaluating content correctness separately from boundary accuracy.

Table R8. Event-content correctness before and after verification among 500 sampled revised events.

Content before verificationContent after verificationEvents, n (%)
IncorrectCorrect63 (12.6%)
CorrectIncorrect12 (2.4%)
IncorrectIncorrect18 (3.6%)
CorrectCorrect407 (81.4%)
Total—500 (100%)

Among these sampled revised events, content accuracy increases from 83.8% to 94.0% after verification. For events whose content is correct both before and after verification, mean absolute boundary error decreases from 13.1 to 7.3 frames. Precise localization remains challenging when adjacent frames exhibit only subtle visual changes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.