- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes an approach to leverage an VLM agent's reasoning capability to select data for post-training VLA policies. The agent compares the policy's rollouts and successful expert demonstrations, identifies difficult phases of the task, and allocates more expert training examples around those parts. The authors call this method RoboCompiler.
The authors evaluate their methods against ad-hoc ways to allocate expert training examples, such as uniform, hand-crafted, and Motion, as well as a FrameSkip-style baseline in simulation (RoboTwin) and the real world.
Overall, I find this paper extremely hard to read. It includes many abstract terms without properly explaining their meanings: ground phases, diagnose difficulties, preserve coverage, etc.. Concretely, what do these things entail in the implementation of the algorithm? The method section does not offer a clear explanation. In addition, the experimental setting of the method compared to the baseline is not entirely fair. Policy rollouts are provided to the agentic framework, whereas baselines only use the expert trajectories. Finally, the contribution of the method is marginal and the scope of the experiment is limited. The agentic framework provides a powerful tool to analyze failures and segment phases of execution, but the method uses it to re-weight data for behavior cloning.
For the above reasons, I recommend rejection of the paper.
Critical strengths
The authors propose an agentic framework that leverages powerful models to analyze failures and segemnt task stages. The idea of leveraging foundation models to improve physical robot learning is promising.
The authors evaluate their method across simulation and real world settings.
Critical weaknesses
The presentation of the paper is poor. The method is hard to understand. The implementation of the method is unclear.
The experiment setting is not entirely fair between the method and its baselines. The method assumes access to additional policy rollouts.
Real world results have marginal gains over the baselines. Given limited evaluation samples, statistical significance is unclear.
Questions for authors
What is the actual input / output to the VLM agent?
How is the prompt optimized for the method and how sensitive is the VLM agent's output to different prompts?
How sensitive is the method to the quality of VLM outputs?
For a fixed policy rollout budget, how does this approach compare to an agentic RL approach?
Is there any statistical significance in the real world results between RC and FS, and between RC and HC?