- Rating
- 4
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes RoboBranch, a framework that improves task success on robotic benchmarks by 1. allowing each state in the robot simulation environment to be snapshotted and hence can be used to execute >1 actions for exploration; 2. prompting the backbone model to revise the task guidance (i.e., hints) using these explored branches; and 3. after N iterations, the final task guidance is used to solve the task in one go. The authors show that RoboBranch outperforms strong baselines such as equipping Qwen with Harness VLA, and also brings open-source agent performance near proprietary agents built on Claude Code and Codex.
Critical strengths
- The writing is clear and easy to follow. The proposed method (RoboBranch) is generally intuitive and sound.
- The experiment and baseline setups are sound. Since RoboBranch uses "more compute" as it additionally branches per state, the authors performed additional comparisons with independent rollouts matching the compute (Section 5.4) as well as efficiency analysis (Section 5.5) to ensure a fair comparison.
- The overall positive results on multiple benchmarks such as LIBERO-Pro and RoboCasa365 shows the effectiveness of the proposed methods compared to strong baselines such as Harness VLA and proprietary agents build with Claude Code or Codex.
Critical weaknesses
-
The idea of using branching to improve exploration or subsequently used for reflection (e.g., revising the task guidance) is established in prior work such as [1] and [2]. The authors should provide a more detailed comparison against these methods in the related work.
-
The idea of branching requires snapshots (for backtracking and exploring alternatives). This is applicable only to simulated environments such as the ones used in this paper. How could the author extend this to scenarios where the environment cannot be easily snapshot/restored (e.g., robot rollouts in physical world)?
References
[1] Zhang, Kai, et al. "Agent learning via early experience." arXiv preprint arXiv:2510.08558 (2025).
[2] Yu, Xiao, et al. "Exact: Teaching ai agents to explore with reflective-mcts and exploratory learning." International Conference on Learning Representations. Vol. 2025. 2025.
Questions for authors
- Generally speaking, search like methods requires a decently strong model (as errors in certain components such as reflection can easily compound and bottleneck the whole system). Have the authors tried RoboBranch with other smaller models, and I wonder what is the minimum model size for RoboBranch to be effective?
Response to Reviewer zogt
We thank the reviewer for the valuable feedback and acknowledgments of:
W1: Comparison with Early Experience and ExACT.
RoboBranch's guidance update advances over Early Experience and ExACT in three ways: (i) Branching from the agent's own intermediate states. RoboBranch restores any state the agent reaches during exploration and tries N actions from it, judging each by its task progress and success; Early Experience branches only from states in expert demonstrations and records only the immediate next state of each alternative action. (ii) Learning from every outcome. RoboBranch revises task guidance from the complete branch set, grouped by its shared state; Early Experience explains why the expert action beats each alternative, and ExACT reflects only on its most erroneous actions. (iii) No expert data, no weight updates, and branching only at learning time. RoboBranch keeps the agent, learner, and VLA frozen and evaluates with a single trajectory under the learned guidance, so deployment needs no state restoration and no extra search cost; Early Experience requires expert demonstrations and fine-tunes the policy, and ExACT runs tree search on every test task and further fine-tunes the model.
We will cite Early Experience and add this comparison to the related work.
W2: Environments that cannot be snapshotted, such as physical robots.
RoboBranch needs snapshots only while learning guidance. The learned guidance is plain text that a frozen agent follows during single-trajectory execution, so it applies wherever that agent runs; RoboTwin C2R already tests this separation, since guidance is learned in the clean setting and evaluated in the randomized setting. We see three routes to physical robots:
(i) Learn in simulation, execute on hardware: learn guidance in a simulator or a digital twin of the target scene, then run the frozen agent on the robot.
(ii) Approximate restoration: for tasks with resettable scenes, re-create a decision state by resetting the scene and replaying the selected prefix.
(iii) Learned world models: branch inside a learned world model, with progress estimated from observations rather than read from the simulator (Section 6).
We will expand this discussion in Section 6.
Q1: Smaller models, and the minimum model size.
We appreciate this insightful question. We ran RoboBranch with two smaller models as both the agent and the learner on all 50 RoboTwin C2R tasks, learning from scratch with the same setting as the paper (10 candidates), and evaluated each learned guide on the same 250 held-out episodes against the same model without guidance learning:
Minimum model size: RoboBranch remains effective down to 9B, the smallest model we tested. Guidance learning improves every model in every run: by 5.2 points on average for Qwen3.5-9B (39.2% to 44.4%) and by 7.8 points for Qwen3.8-27B (50.4% to 58.2%). At 9B, whose model fits on a single GPU, the gain is still positive, so we have not yet found a size below which RoboBranch stops helping.