acceptodds
Under review as a conference paper at ICLR 2027

BARM: Bayesian Assistance Routing for Embodied Robot Manipulation over VLM and Human Experts

Abstract

Embodied agents deployed in open-world environments must decide, for each incoming task, whether to execute it autonomously with a lightweight policy, invoke a vision-language model planner, or request human assistance. These execution modes differ in capability and cost, and the routing controller is typically trained on atomic tasks but deployed on compositional, multi-step instructions that lie outside its training distribution. We cast embodied assistance routing as a contextual bandit problem with heterogeneous arm costs and identify two desiderata for sample-efficient routing under distribution shift: distance-aware uncertainty, so that unseen tasks are recognized as uncertain, and probability-matching exploration, so that uncertainty translates into cost-aware action. We propose BARM (Bayesian Assistance Routing for embodied robot Manipulation), a unified Bayesian framework whose single belief state simultaneously satisfies both desiderata: its epistemic variance grows with distance from the training distribution, and Thompson Sampling on this belief realizes probability-matching exploration over net rewards. Reward modeling, exploration, and online adaptation all emerge from this one belief rather than from separately engineered heuristics. We also establish a sublinear Bayesian regret bound for BARM under the well-specified regime (linear reward). On CALVIN and LIBERO benchmarks, BARM achieves the highest cumulative reward against 22 baselines spanning failure detection, reinforcement learning, and contextual bandit methods.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied AI, Assistance routing, Contextual bandits, Bayesian uncertainty, Thompson sampling, Distribution shift

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_eYYBOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces BARM, aiming to serve as a router to distinguish OOD situations in robot manipulation that need VLM or human help.

Critical strengths

The paper aims to solve a problem which is important nowadays as agent-based robot manipulation is a trend, which makes simpler ways to distinguish OOD cases during rollout crucial. The method in this paper is lightweight.

Critical weaknesses
  1. The experiments are fully conducted in simulated environments, while real-world environments are lacked. Real-world scenes are more diverse and uncontrolled.
  2. The method relies on SNGP to detect OOD cases where VLM or human helps are needed. However, in broader settings and open world with various scenes and different tasks, SNGP might be unreliable for such detection In the paper, the authors use only hundreds of samples for training and testing, which is a very small one compared pretraining-scale or multitask datasets used in actual practice. Thus, while the authors aim to solve a practical problem, the method itself might be not practical enough.

Update Rating increased.

Questions for authors

I suggest the authors should conduct experiments under broader settings and with a much larger dataset and see if the performance will degrade, which is probable.

2 comments
AuthorsOpenReview ↗

Thank You for Raising the Score to 4

We thank the reviewer for reading our response and for raising the score to clear acceptance.

We will incorporate the results and clarifications from our response into the revised paper.

AuthorsOpenReview ↗

Follow-up to eYYB (W1): Real-Robot Evaluation

We thank the reviewer again for suggesting a real-world evaluation. As promised in our response to W1, we report the results on a real robot. The table reports net reward, the task reward minus the cost of the selected mode, as mean ± standard deviation over five seeds.

Setup. We deploy BARM on a real myPalletizer 260 arm with a single third-person camera. The three execution modes are those of Section 5.1, with π0.5\pi_{0.5} in place of the diffusion policy. We fine-tune π0.5\pi_{0.5} only on demonstrations of simple tasks collected on this robot. Simple tasks ask the arm to reach one object on the table, and composite tasks ask it to reach multiple objects in a given order. We collect 244 real-robot rollouts spanning simple and composite tasks. The router is trained offline on simple tasks only, and online evaluation uses the same costs as on CALVIN and the same number of episodes, task mixture, seeds, and 22 baselines as in Section 5.1.

Results.

MethodReal robot
Oracle0.923 ± 0.002
BARM0.857 ± 0.010
Bandit methods
Neural-LinUCB0.840 ± 0.009
NeuralTS0.735 ± 0.005
EE-Net0.684 ± 0.050
Detection methods
SNGP+Conf0.746 ± 0.012
Sentinel0.743 ± 0.011
FIPER0.740 ± 0.015
DeeR-VLA0.718 ± 0.009
KnowNo0.711 ± 0.014
CogVLA0.651 ± 0.028
IntroPlan0.648 ± 0.007
RL & learned routing
DQN0.790 ± 0.013
LLM Routing0.736 ± 0.054
MCDrop0.726 ± 0.040
PPO0.701 ± 0.011
REINFORCE0.688 ± 0.006
FiS-VLA0.646 ± 0.018
VADER0.644 ± 0.007
AutoVLA0.548 ± 0.018
Fixed policies
Always Arm20.770 ± 0.013
Random0.682 ± 0.012
Always Arm30.648 ± 0.004
Always Arm10.544 ± 0.018

With the execution modes running on a physical robot, BARM achieves the highest net reward, outperforming all 22 baselines.

We will add these results to the main text of the revised paper, with the full setup and photos of the platform in the appendix and a demo video in the supplementary material.

Reviewer_P7RtOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies instruction-level assistance routing for robot manipulation: choosing between an autonomous policy, VLM-assisted task decomposition, and human assistance, with the objective of maximizing task reward minus assistance cost. The proposed BARM framework combines SNGP-based features, per-arm variational Bayesian last-layer reward models, Thompson sampling, and closed-form online posterior updates. The router is trained on atomic tasks and evaluated through online adaptation on a mixture of atomic and previously unseen compositional tasks. Experiments on CALVIN and LIBERO report improvements over routing, detection, reinforcement learning, and contextual bandit baselines. The paper also provides a Bayesian regret bound under a well-specified linear reward model.

Critical strengths
  • Well motivation. Choosing among execution modes with different capabilities and costs is a useful problem for embodied systems. Explicitly modeling per-mode rewards provides a principled way to distinguish whether assistance is needed from which assistance is worth its cost.

  • Coherent and computationally efficient design. Maintaining reward uncertainty, using it for exploration, and updating the selected arm through conjugate Bayesian updates is a sensible approach when deployment feedback is limited.

  • Positive empirical results. BARM improves average net reward over the strongest reported external baseline, Neural-LinUCB, by 0.034 on CALVIN and 0.038 on LIBERO. The component ablations and additional robustness experiments provide useful evidence about the behavior of the proposed combination.

Critical weaknesses
  • The methodological and theoretical contributions are limited. BARM combines established components, while its regret analysis relies on linear realizability and a matched prior without establishing these conditions under deployment shift. The ablations demonstrate component benefits, but do not establish their claimed joint necessity.

  • The experimental protocol lacks essential details. The manuscript does not clearly specify compositional reward computation, whether online evaluation uses fresh executions or cached outcomes, or how the human-assistance arm is implemented. These omissions limit reproducibility and assessment of the reported assistance trade-offs.

  • The Oracle definition is ambiguous. It is unclear whether the Oracle selects the best realized reward or the highest estimated expected reward. Under stochastic execution, this distinction directly affects arm-selection accuracy, the Oracle gap, and empirical regret.

Questions for authors
  • Could the authors provide a complete account of the evaluation pipeline, including compositional reward computation, the concrete implementation of the human-assistance arm, and whether online episodes use fresh executions or cached outcomes?

  • Does the Oracle maximize realized rewards or estimated expected rewards? How are the reference arm labels constructed, and is the empirical regret comparator consistent with the comparator used in the theoretical analysis?

2 comments
AuthorsOpenReview ↗

Results on a Real Robot

In addition to the simulation benchmarks, we have evaluated BARM on a real robot, where it also achieves the highest net reward, outperforming all 22 baselines. The setup and the full comparison are in our follow-up to Reviewer eYYB, and we will add them to the revised paper.

AuthorsOpenReview ↗

Gentle Follow-up to Reviewer P7Rt on Our Rebuttal

Dear Reviewer P7Rt,

Thank you again for your review. As the discussion period closes on November 18, we would like to briefly summarize our response and ask whether it addresses your concerns.

  • Contribution and assumptions. BARM provides distance-aware uncertainty and probability-matching exploration through a single posterior, and replacing its closed-form update with gradient updates degrades performance on both benchmarks. The assumptions of Theorem 1 concern reward parameters shared across contexts, and linear realizability is verified on both simple and composite tasks (Appendix F.3). BARM also corrects its routing online when the prior favors the wrong arms.
  • Joint necessity. The completed two-by-two ablation shows that removing either SNGP features or Thompson sampling degrades BARM, and removing both degrades it further.
  • Evaluation protocol. Online episodes use cached outcomes, composite rewards follow each benchmark's built-in success checks, and the human arm uses ground-truth subtask decomposition executed by the diffusion policy. We will add these details to Section 5.1.
  • Oracle. The Oracle selects the arm with the highest realized net reward in hindsight, and reference arm labels are constructed from the same Oracle for all methods. The empirical regret against this Oracle upper-bounds, in expectation, the regret against the expected-reward comparator used in Theorem 1.
  • Real-robot evaluation. BARM also achieves the highest net reward among all 22 baselines on a physical robot. The complete experimental setup and comparison table are provided in our November 13 comment, "Follow-up to eYYB (W1): Real-Robot Evaluation," under Reviewer eYYB.

We hope the additional evidence on the theoretical assumptions and joint ablation, together with the clarifications of the evaluation protocol and Oracle, addresses your concerns. We would greatly appreciate any remaining questions or feedback before the discussion period ends. If these responses have resolved your concerns, we would be grateful if you could consider updating your rating.

Reviewer_jm7BOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents BARM, a framework that formulates selection of different execution modes as a contextual bandit problem. The authors target the compositional tasks that can be accomplished from the atomic tasks trained offline. However, there is a distribution shift when combining tasks. To address this, the paper introduces distance-aware uncertainty and employs Thompson sampling for online exploration and adaptation. The experiments use three arms, diffusion policy, VLM, and human operator, on CALVIN and LIBERO benchmarks and show the improved cumulative rewards over 22 baselines.

Critical strengths
  • The idea to formulate the executor selection as a contextual bandit problem is interesting and a sound solution. The integration also handles the common failure cases when combining atomic tasks which is a plus.
  • The evaluation spans a wide range of setups/baselines and shows improved empirical results.
Critical weaknesses
  • The experiments include human intervention as one of the arms. This will lead to a much higher success rate, and the cost is not that much compared with the VLM arm, the method can just call for human help to improve the task completion. So, it is not a fair comparison for other baselines which do not involve humans.
  • If removing the arm with human intervention, it becomes choosing between VLM subtask planner and diffusion policy. It is very similar to a hierarchical policy. In this case, the paper misses a critical baseline for hierarchical VLM+diffusion policy baseline.
Questions for authors
  • How often does BARM select different arms across simple vs. composite tasks? It would be helpful to clarify the gain from VLM routing vs. human intervention.
  • How does BARM compare with a simple hierarchical baseline?

Post-discussion update: The authors' response has largely address the above questions. The inclusion of the real-robot experiment is great but the setup is unclear in the rebuttal, and will be helpful to include more details in the updated manuscript.

4 comments
AuthorsOpenReview ↗

Additional Real-Robot Evaluation

We have also evaluated BARM on a real robot, where it achieves the highest net reward, outperforming all 22 baselines. The setup and the full comparison are in our follow-up to Reviewer eYYB, and we will add these results to the main text of the revised paper.

AuthorsOpenReview ↗

Gentle Follow-up for Reviewer jm7B on Our Rebuttal

Dear Reviewer jm7B,

Thank you again for your review. As the discussion period closes on November 18, we would like to briefly summarize our response and ask whether it addresses your concerns.

  • Human arm and fairness. Every baseline in Table 1 is evaluated with the same three execution modes as BARM, including human assistance. Always calling the human ranks well below BARM, showing that the gain comes from deciding when to request assistance. The arm-selection statistics further show that BARM almost never calls the human on simple tasks and concentrates assistance on composite tasks.
  • Hierarchical baseline. Always Arm 2 in Table 1 is exactly the hierarchical VLM + diffusion-policy baseline. BARM outperforms it on both benchmarks. In our additional two-arm experiments without human assistance, BARM still outperforms both direct execution and the hierarchical baseline on CALVIN and LIBERO.
  • Real-robot evaluation. BARM also achieves the highest net reward among all 22 baselines on a physical robot. The complete experimental setup and comparison table are provided in our November 13 comment, "Follow-up to eYYB (W1): Real-Robot Evaluation," under Reviewer eYYB.

We hope the equal-access comparison, the hierarchical baseline results, and the additional two-arm experiments address your concerns about fairness and the source of BARM's improvements. We would greatly appreciate your feedback on whether these results resolve your concerns. If so, we would be grateful if you could reconsider your rating before the discussion period ends.

I thank the authors to address my questions. I have raised my rating and added a post-discussion update.

AuthorsOpenReview ↗

Many Thanks for Updating the Rating to Weak Accept

Dear Reviewer jm7B,

We sincerely appreciate your updated rating of Weak Accept and your constructive feedback!

The clarifications and additional results from this discussion, together with the complete real-robot setup, will be incorporated into the revised manuscript.

Comments on the paper

3 comments

I appreciate the authors' efforts in addressing my comments. My concerns have been largely resolved. Therefore, I have raised my score to Weak Accept.

AuthorsOpenReview ↗

Thank You for Raising the Rating to Weak Accept

Dear Reviewer P7Rt,

Thank you for raising your rating to Weak Accept and for your constructive feedback!

We will incorporate the clarifications and additional results from this discussion into the revised paper.

AuthorsOpenReview ↗

AC Rebuttal Summary: All Three Reviewers Raised Their Ratings to 4 (Clear acceptance), 3 (Weak acceptance), and 3 (Weak acceptance)

Dear Area Chair,

We thank you and the reviewers for the constructive discussion. All three reviewers raised their ratings during the discussion. We summarize the discussion with each reviewer below.

Reviewer eYYB (Rating 4: Clear acceptance)

Reviewer eYYB recognizes that the paper addresses "a problem which is important nowadays as agent-based robot manipulation is a trend" and that "the method in this paper is lightweight."

After reading our response, the reviewer raised the rating to "4: Clear acceptance."

Real-robot evaluation (W1). Following the reviewer's suggestion, we evaluated BARM on a real robot. With the execution modes running on a physical robot, BARM achieves the highest net reward and outperforms all 22 baselines. See our comment "Follow-up to eYYB (W1): Real-Robot Evaluation".

SNGP in broader settings and at larger scale (W2 + Q1). BARM routes each task by sampling from per-arm reward posteriors that are updated online from observed outcomes. SNGP features keep the predictive uncertainty calibrated on unseen composite tasks, BARM routes tasks held out from offline training to direct execution as often as seen tasks, and BARM keeps improving with longer deployment. See our comment "Rebuttal to eYYB", W2 + Q1.

Reviewer jm7B (Rating 3: Weak acceptance)

Reviewer jm7B finds that "the idea to formulate the executor selection as a contextual bandit problem is interesting and a sound solution", that "the integration also handles the common failure cases when combining atomic tasks which is a plus", and that "the evaluation spans a wide range of setups/baselines and shows improved empirical results."

After the discussion, the reviewer raised the rating to "3: Weak acceptance" and wrote that "the authors' response has largely address the above questions" and that "the inclusion of the real-robot experiment is great".

Human arm and fairness of the comparison (W1 + Q1). Every baseline routes over the same three execution modes as BARM, including the human arm, under identical settings. The policy that always calls the human ranks well below BARM on both benchmarks, and BARM almost never calls the human on simple tasks, so its gain comes from deciding when to call the human. See our comment "Rebuttal to jm7B", W1 + Q1.

Hierarchical VLM + diffusion-policy baseline (W2 + Q2). This baseline is already in the paper as Always Arm2, and BARM outperforms it by a large margin on both benchmarks. With the human arm removed, BARM still outperforms both direct execution and the hierarchical baseline on both benchmarks. See our comment "Rebuttal to jm7B", W2 + Q2.

Real-robot setup (post-discussion update). Following the reviewer's suggestion to include more details in the updated manuscript, we have added the complete real-robot setup to the revised PDF (Section 5.1 and Appendix H).

Reviewer P7Rt (Rating 3: Weak acceptance)

Reviewer P7Rt finds the paper well motivated, noting that "choosing among execution modes with different capabilities and costs is a useful problem for embodied systems" and that "explicitly modeling per-mode rewards provides a principled way to distinguish whether assistance is needed from which assistance is worth its cost." The reviewer describes a "coherent and computationally efficient design" and "positive empirical results", with ablations and robustness experiments that "provide useful evidence about the behavior of the proposed combination".

After the discussion, the reviewer wrote: "My concerns have been largely resolved. Therefore, I have raised my score to Weak Accept."

Contribution and assumptions (W1). Theorem 1 establishes a sublinear Bayesian regret bound for the heterogeneous-cost routing problem. Assumptions 4 and 5 concern the reward parameters of each arm, which are shared by all contexts, so the shift from simple to composite tasks changes which contexts are observed, not these parameters. We also extended the ablation to a two-by-two design, in which each of SNGP features and Thompson sampling helps more when the other is present. See our comment "Rebuttal to P7Rt", W1.

Evaluation protocol and Oracle (W2 + Q1, W3 + Q2). We detailed how online episodes return outcomes executed in advance, how composite rewards are computed, and how the human arm is implemented, and defined the Oracle as the arm with the largest realized net reward in each episode. See our comment "Rebuttal to P7Rt", W2 + Q1 and W3 + Q2.

We have revised the PDF accordingly, with all changes highlighted in blue, and added a demo video of the real-robot experiment to the supplementary material. To keep all table, figure, and appendix numbers cited in the discussion unchanged, the new real-robot table in the main text is numbered Table 1 (Real robot).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.