acceptodds
Withdrawn from ICLR 2027

RoboCompiler: Agentic Experience Compilation for Robot Post-Training

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Successful demonstrations provide correct behavior, but their value for post-training depends on what a robot policy can already do. Long, reliable motions can dominate replay, leaving brief, difficult interactions with little supervision. We study how an agent can use the policy's execution experience to organize further learning. We present RoboCompiler, a framework for agentic experience compilation. A multimodal agent compares policy successes and failures with expert demonstrations, relates them through observable task phases, and maps diagnosed difficulties to relevant expert behavior. It then writes a supervision program that concentrates training around these interactions while preserving the surrounding skill. The program determines both how much exposure each phase receives and which observations provide that supervision. Across ten RoboTwin 2.0 tasks, RoboCompiler improves π₀.₅ mean success by 16.3 percentage points over full-demonstration training under the same learner-update budget. Experiments with LingBot-VLA and three physical robot tasks support its use across learners and in real manipulation. Component studies show the value of policy evidence and phase allocation, with a further, task-dependent benefit from local selection. These findings demonstrate a practical route from an agent's interpretation of robot experience to improved policy learning.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot Learning, Multimodal Agents, Experience Compilation, Robot Post-Training

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_BNYwOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes an approach to leverage an VLM agent's reasoning capability to select data for post-training VLA policies. The agent compares the policy's rollouts and successful expert demonstrations, identifies difficult phases of the task, and allocates more expert training examples around those parts. The authors call this method RoboCompiler.

The authors evaluate their methods against ad-hoc ways to allocate expert training examples, such as uniform, hand-crafted, and Motion, as well as a FrameSkip-style baseline in simulation (RoboTwin) and the real world.

Overall, I find this paper extremely hard to read. It includes many abstract terms without properly explaining their meanings: ground phases, diagnose difficulties, preserve coverage, etc.. Concretely, what do these things entail in the implementation of the algorithm? The method section does not offer a clear explanation. In addition, the experimental setting of the method compared to the baseline is not entirely fair. Policy rollouts are provided to the agentic framework, whereas baselines only use the expert trajectories. Finally, the contribution of the method is marginal and the scope of the experiment is limited. The agentic framework provides a powerful tool to analyze failures and segment phases of execution, but the method uses it to re-weight data for behavior cloning.

For the above reasons, I recommend rejection of the paper.

Critical strengths

The authors propose an agentic framework that leverages powerful models to analyze failures and segemnt task stages. The idea of leveraging foundation models to improve physical robot learning is promising.

The authors evaluate their method across simulation and real world settings.

Critical weaknesses

The presentation of the paper is poor. The method is hard to understand. The implementation of the method is unclear.

The experiment setting is not entirely fair between the method and its baselines. The method assumes access to additional policy rollouts.

Real world results have marginal gains over the baselines. Given limited evaluation samples, statistical significance is unclear.

Questions for authors

What is the actual input / output to the VLM agent?

How is the prompt optimized for the method and how sensitive is the VLM agent's output to different prompts?

How sensitive is the method to the quality of VLM outputs?

For a fixed policy rollout budget, how does this approach compare to an agentic RL approach?

Is there any statistical significance in the real world results between RC and FS, and between RC and HC?

Reviewer_AyenOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper presents RoboCompiler, an LLM-based framework for selecting existing expert supervision for robot policy post-training. It addresses the mismatch between demonstration duration and training value: long, easy phases can dominate replay, while short but important interactions receive insufficient emphasis. Given successful expert demonstrations and diagnostic policy rollouts, a multimodal agent identifies task phases, analyzes policy difficulties, and allocates a fixed number of training starts across phases. It then selects observations within each phase and produces an executable selection program, preserving coverage of the complete task. The selected observations retain their original expert action chunks, and the policy is fine-tuned using its existing objective without LLM involvement during deployment. Experiments cover ten simulated tasks with π0.5, five with LingBot-VLA, and three physical robot tasks. The results show overall improvements over full-demonstration training and fixed data selectors, while ablations examine the contributions of diagnostics, coverage, allocation, and local observation selection.

Critical strengths
  • This paper is easy to follow
  • The idea of using an LLM to select expert data for policy fine-tuning is interesting.
Critical weaknesses

1. Reliability and applicability of the LLM's semantic judgments are insufficiently characterized

The method relies on the LLM for phase segmentation, failure diagnosis, and supervision allocation. Although the downstream improvements support the usefulness of the overall pipeline, they do not independently establish the correctness of these intermediate judgments. The program checks verify selection constraints rather than semantic accuracy. Controlled evaluations that vary task horizon, contact complexity, and ambiguity between different failure mechanisms would help establish when the proposed diagnosis is reliable and how incorrect judgments affect subsequent learning. These capability boundaries remain insufficiently examined in the current evaluation.

2. The contribution of explicit phase structure is not isolated

The Direct Agent Selection ablation is relevant, but it simultaneously removes the requirements for phase graphs, diagnosis cards, and quota programs. Moreover, Appendix G indicates different amounts of reasoning and inspection between the two procedures. This comparison therefore does not isolate the contribution of explicit phase structure beyond failure diagnosis itself. A more targeted baseline would retain explicit failure diagnosis and directly map diagnosed weaknesses to relevant expert segments, without requiring phase graphs or phase quotas. Matching the LLM, evidence, selection budget, learner settings, and actual inference budget would clarify whether the proposed structured allocation offers benefits over this simpler alternative.

3. Insufficient justification for the overall collection and post-training pipeline

The experiments support the proposed data selector, but provide limited justification for the broader pipeline of expert collection, phase-based selection, and post-training. Two alternative strategies deserve consideration:

  • Failure-guided expert collection. Although expert demonstrations are collected from the diagnostic initial scenes, their collection is not guided by the subsequent failure diagnosis. A natural alternative is to collect targeted demonstrations in failure-prone scenes or from states near observed policy deviations. Comparing this strategy with collecting complete demonstrations and then selecting from them, under matched expert-supervision and environment-interaction budgets, would clarify whether diagnosis is more useful when it also informs what data to collect.

  • Alternative uses of the LLM and environment access. The same LLM could instead be evaluated through Code-as-Policies, online policy assistance, or reward generation followed by RL fine-tuning. These alternatives may incur different interaction, training, and deployment costs, so equal cost should not be assumed. However, comparisons reporting both performance and resource requirements would help establish when the proposed collection–selection–post-training pipeline offers a preferable trade-off.

Questions for authors

See weaknesses

Reviewer_23AfOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes RoboCompiler, which uses an off-the-shelf LLM agent to decide how much each part of the successful expert demonstrations should be trained on during VLA post-training. Before training, the agent compares the policy's own rollouts (successes and failures) with the expert demos, segments the task into observable phases, and diagnoses where the policy fails. On ten RoboTwin 2.0 tasks with π0.5, mean success improves from 54.8% (training on full demos) to 71.2%.

Critical strengths
  • Well-motivated and easy to follow.
  • The experiments are carefully controlled and the ablation is informative.
Critical weaknesses
  1. Limited novelty, and some potentially missing related work. My understanding: at high level, the method prompts an off-the-shelf frontier model to author a static, phase-level sampling distribution once before training. Sveral closely related lines of work are not discussed:
    • Upweighting where the learner fails: IWR (Mandlekar et al., 2020) and Sirius (Liu et al., 2023) reweight demonstration segments around human interventions.
    • VLM-based failure reasoning for robots: REFLECT (Liu et al., CoRL 2023) and AHA (Duan et al., ICLR 2025).
  2. The FrameSkip-style baseline seems weakened. FS is below Full on average (47.5 vs. 54.8), and on Alarm it is even below the untrained policy (25.0 vs. 55.0). This is at odds with FrameSkip's own results.
Questions for authors

Please see the weaknesses above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.