acceptodds
Under review as a conference paper at ICLR 2027

Train Together or Merge Later? Unifying VLA Experts via a Shared Action Interface

41 views

Abstract

Co-training offers a straightforward way to build a multi-task vision-language-action (VLA) policy, but can fall short of the performance achieved by training each task independently. The challenge is to retain these task-specific gains in a multi-task policy without joint post-training. Combining independently trained experts through model merging is a natural approach, yet strong individual experts do not necessarily yield a strong merged policy. We identify one source of this incompatibility: task-specific changes to the action interface, comprising action normalization and the action encoder and decoder. We propose POLICYWEAVE, combining merge-compatible post-training with context-guided sparse merging. During post-training, all experts retain the common base policy’s action interface, while task adaptation is restricted to LoRA updates in the hidden layers of the action model. This makes the experts more compatible with existing model merging methods. However, merging all experts can still introduce interference from unrelated tasks at deployment. POLICYWEAVE scores each expert’s LoRA updates using the initial visual-language context, determines the expert set through leave-one-layer-out ranking stability, and forms a sparse weighted merge of the selected updates that remains fixed for current task. We evaluate POLICYWEAVE with GR00T N1.5 on 18 RoboCasa365 tasks, using only 10% of the target-task demonstrations for supervised fine-tuning (SFT). Preserving the shared action interface raises the average success rate across four static merging methods from 17.0% to 52.8%. POLICYWEAVE achieves 64.7% success with these SFT experts and 74.1% after task-specific reinforcement learning (RL), compared with 60.7% for joint RL. Further evaluations on LIBERO-10 and an AgileX Piper arm support the deployment of independently learned skills in long-horizon and real-world manipulation. These results show that independently learned robot policies can be merged for multi-task deployment without joint post-training across tasks. Source code and real robot demos are available in the supplementary material.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-language-action model, model merging, multi-task robot learning, VLA post training

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_SFKjOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PolicyWeave, which aims to automatically unify VLA models trained for different specific tasks into a single deploy policy model. It freezes action normalization and projectors during adaptation, then selects or sparsely merges experts using initial task context. Experiments on RoboCasa365 and LIBERO-10 shows improved ability to merge different models and the reserve of the independently learned skills.

Critical strengths
  1. The paper shows taht preserving the shared action interface improves static merging: SR increases from 17.0% to 52.8%, while individual expert performance remains comparable. This is a useful practical finding for merging different specifically trained VLA experts. With the recent large deployment of Physical AI APIs, this would be a practical and useful direction.

  2. The deployment method preserves much of the improvement from task-specific RL and is evaluated across simulation and real-world manipulation. This shows the closed-loop training for VLA models can improve even when they got merged; which could provide useful insights for the readers.

Critical weaknesses
  1. The contribution beyond expert routing remains unclear. Almost 99% of RoboCasa episodes and 93.4% of LIBERO episodes execute a single expert. Therefore, most of the reported advantage may come from independent specialization and accurate selection. The same-scorer Top-1 comparison is reported only on LIBERO, where adaptive merging improves success from 73.2% to 75.4%.

  2. The action-interface findings are not fully disentangled. The main comparison changes normalization and encoder/decoder adaptation together. Post-training component replacement in Table 3 demonstrates sensitivity to interface mismatch, but does not ablate the effect of normalization and projectors.

Questions for authors
  1. I hope the authors could explain more on how does the method compare with a policy selector on a fixed set of task-specific experts? Why is this particularly helpful?

  2. Will the proposed method work when the deployment task is a combination of multiple skills, and can not be achieved by any single expert policy? Will this better show the usefulness of the proposed method?

Reviewer_yDMxOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to use separately trained VLA experts in one multi-task system. PolicyWeave trains task-specific LoRA adapters while keeping action normalization and the action encoder and decoder fixed. It ranks experts using the initial observation and instruction. It picks one expert if removing any single layer’s score does not change the top choice. Otherwise, it merges a small set of experts. The selected parameters stay fixed throughout the episode, while the policy continues to use new observations. PolicyWeave outperforms the tested joint-training baselines on RoboCasa365, LIBERO-10, and a real robot. It also beats static merging methods in simulation.

Critical strengths
  1. S1: Sharing the action interface works well. On RoboCasa365, it raises average success across four static merging methods from 17.0% to 52.8%, without reducing the success rates of the individual experts. The normalization analysis and examples of unwanted base movement help explain what goes wrong when experts use different interfaces. Other merging methods could benefit from this finding too.

  2. S2: Most of the improvement from training each expert with RL carries over to PolicyWeave. On RoboCasa365, individual experts improve by 10.6 percentage points, and PolicyWeave improves by 9.4 points. SSR-Merge has the best final success rate among the static baselines, but improves by only 3.9 points. PolicyWeave also avoids the extra step of training a router to choose experts.

Critical weaknesses
  1. W1: I am not sure how much the merging step helps. Table 14 shows that PolicyWeave picks just one expert in over 98% of RoboCasa episodes and 93.4% of LIBERO episodes. On LIBERO, merging improves success by 2.2 percentage points over Top-1 selection using the same scorer. The improvement is larger on the episodes that use multiple experts, but no uncertainty estimates are reported. For the two RoboCasa expert banks and the real robot, results for Top-1 selection with the same scorer are missing. These comparisons would help show whether merging is worth adding.

  2. W2: The simulation comparison needs better control of training conditions. Each task expert gets 10,000 SFT steps. The joint-training baselines use official checkpoints, but the paper does not show whether their SFT budgets and settings are comparable. The RL interaction budgets are similar, but training starts from different SFT policies. Therefore, some of the final performance gap may come from differences in SFT budgets or settings. The real-robot comparison with GR00T is better controlled, so this concern mainly applies to simulation.

Questions for authors
  1. Q1: How well does Top-1 selection work with the same scorer on both RoboCasa expert banks and the real robot? For episodes that use multiple experts, how many are there, and how much does merging improve success over Top-1 on those same episodes? Please include confidence intervals for the difference.

  2. Q2: Did joint training and individual expert training use comparable SFT compute, data exposure, and trainable layers in simulation? If not, what differed? Does PolicyWeave still do better when these conditions are matched?

  3. Q3: Would sharing normalization alone give most of the improvement? Table 3 changes components after training. Could you instead train experts with shared normalization, then separately test freezing the encoder, the decoder, and both?

  4. Q4: Tables 4 and 14 suggest that merging adds nine successes on the 33 multi-expert episodes, but eleven overall. Where do the other two come from? For the remaining 467 episodes, did both methods use the same expert parameters, initial states, and policy-noise seeds?

1 comment

Dear Reviewer yDMx,

With the discussion period nearing its end, we would like to follow up to see if our response has resolved your questions. We would be glad to provide any further clarification if needed.

Best regards,

The Authors

Reviewer_Af1eOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2(−1)
Soundness
2
Read review
Paper summary

This paper proposes PolicyWeave, a framework for deploying independently post-trained VLA experts through a shared action interface and context-guided sparse merging. PolicyWeave combines merge-compatible VLA post-training with context-guided expert selection and sparse merging. Its central insight is to preserve a shared action interface, which includes normalization statistics and an action encoder/decoder, while adapting the hidden layers of the action model using LoRA.

Critical strengths
  1. The shared-interface intervention produces a substantial and consistent improvement.
  2. The evaluation covers atomic tasks, long-horizon manipulation, real-world execution, and experts improved through task-specific RL.
  3. The framework preserves the original VLA architecture and separates merge-compatible post-training from the choice of merging algorithm, making the recipe practically useful.
Critical weaknesses
  1. The headline performance primarily reflects expert routing. Table 14 shows that only 1.22–1.44% of RoboCasa episodes and 6.60% of LIBERO-10 episodes combine multiple experts. On LIBERO-10, adaptive merging improves success from 73.2% for the same scorer’s Top-1 selection to 75.4%. This supports a modest incremental benefit, but does not establish that multi-expert merging is the main source of the overall gains.
  2. The distinction from prior work needs stronger experimental support. MergeVLA[1] already addresses VLA non-mergeability and uses training-free initial-context response-based routing. CORAL[2] and MoIRA[3] establish independent VLA LoRA expert banks with expert selection. The paper discusses related approaches and reproduces MergeVLA, but different backbones and training pipelines limit causal interpretation. Reporting merged-to-individual retention ratios does not eliminate these confounds.

[1] Yuxia Fu, Zhizhen Zhang, Yuqi Zhang, Zijian Wang, Zi Huang, and Yadan Luo. MergeVLA: Cross-Skill Model Merging Toward a Generalist Vision-Language-Action Agent. CVPR, 2026.

[2] Yuankai Luo, Woping Chen, Tong Liang, and Zhenguo Li. CORAL: Scalable Multi-Task Robot Learning via LoRA Experts. arXiv preprint arXiv:2603.09298, 2026.

[3] Dmytro Kuzmenko and Nadiya Shvai. MoIRA: Modular Instruction Routing Architecture for Multi-Task Robotics. Neurocomputing, 674:132962, 2026.

Questions for authors

see weaknesses

Reviewer_FopgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to consolidate task-specific VLA experts independently adapted from a common pretrained model while retaining their specialized capabilities. The proposed method, PolicyWeave preserves a shared action interface during expert training: action normalization statistics, the action encoder and decoder, and the VLM remain fixed, while task adaptation is restricted to LoRA updates in the hidden action-model layers. After that, it scores experts using the normalized responses of their key-projection updates to the initial visual-language context. A leave-one-layer-out ranking-stability test determines whether to select one expert or merge a small subset. The resulting parameters remain fixed throughout the episode. Experiments use GR00T N1.5 as the base backbone on 18 atomic RoboCasa365 tasks, 10 long-horizon LIBERO tasks, and four real-world Piper manipulation tasks. The results demonstrate substantial improvements in static merging when experts share an action interface, as well as retention of task-specific reinforcement-learning gains.

Critical strengths

1. The shared-action-interface finding has meaningful empirical value.

The paper shows that individual expert performance remains approximately comparable with task-specific versus shared interfaces—64.1% versus 65.6%—while the average success rate of four static merging methods increases from 17.0% to 52.8%. Similar improvements occur on LIBERO. These results support the distinction between individual policy competence and compatibility for parameter merging.

2. The method requires limited architectural changes.

PolicyWeave retains the original VLA architecture and uses a shared backbone with task-specific LoRA updates. So it only trains a few parameters, which is an efficient method and can retain as more the pretrained VLA's ability as possible. Expert scoring requires neither a separate trained router nor a full forward pass through every expert.

3. Retaining independently acquired RL gains is practically relevant.

On RoboCasa, task-specific RL improves individual experts by 10.6 percentage points, and PolicyWeave retains a 9.4-point improvement. The interaction budgets are reasonably controlled: independent RL uses approximately 23.74 million environment steps, compared with 26.27 million for joint RL. Thus, the reported advantage cannot simply be attributed to more RL interaction.

Critical weaknesses

1. Expert selection and multi-expert merging are not sufficiently disentangled.

This is my primary concern. Table 14 shows that multiple experts are merged in only 1.44% of RoboCasa SFT episodes, 1.22% of RoboCasa SFT+RL episodes, and 6.60% of LIBERO episodes. According to Equation (2), the remaining episodes execute exactly one selected expert. If this lose the sense of your method?

The LIBERO comparison is informative: Top-1 selection with the same scorer achieves 73.2%, whereas adaptive merging achieves 75.4%. However, corresponding matched Top-1 closed-loop results are missing for the main RoboCasa SFT, RL, and real-world experiments. Consequently, the contributions of independent specialization, expert identification, and actual multi-expert merging remain insufficiently separated.

In particular, the paper does not report the performance of the true-task-ID-selected expert on the subset where merging is triggered. Merging might exploit complementary capabilities, or it might primarily compensate for errors made by the proposed scorer. These explanations imply different contributions and should be distinguished experimentally.

2. The co-training comparisons do not adequately establish the practical advantage of the proposed deployment strategy.

The LIBERO comparison omits representative strong fine-tuning baselines. OpenVLA-OFT reports 94.5% on LIBERO-Long, and its single policy jointly trained across all four suites achieves 95.3% on the Long suite. The official openpi implementation reports 92.4% for fine-tuned π0.5. In comparison, this paper reports 72.4% for co-trained GR00T, 80.8% for individual experts, and 75.4% for PolicyWeave.

There is also insufficient information about the simulated co-training SFT budgets. Each expert receives 10,000 training steps, totaling 180,000 steps for RoboCasa and 100,000 for LIBERO, while the co-training baselines use official checkpoints without a sufficiently explicit account of their training budgets and adaptation configurations. Matching the demonstration fraction does not establish computational comparability.

These issues weaken the argument for independently training and consolidating experts over a well-optimized joint policy. They do not invalidate the interface result, but the practical benefit of maintaining an expert bank remains insufficiently demonstrated.

3. The action-interface experiments do not sufficiently isolate the contributions of individual components.

The principal comparison simultaneously changes whether normalization, the encoder, and the decoder are task-specific. Table 3 replaces components after experts have already been trained. This demonstrates sensitivity to incompatible interfaces, but it does not isolate the contribution of each constraint during training.

A controlled training-time ablation—such as shared versus task-specific normalization crossed with frozen versus trainable input-output projectors—would clarify which components drive the improvement. Both individual and merged performance should be reported to determine whether freezing the complete interface is necessary and whether it imposes an adaptation cost.

4. Generality is supported only within one backbone and task-covered expert banks.

All PolicyWeave experiments use GR00T N1.5. Including π0.5 as a co-training baseline or reproducing MergeVLA on its own backbone does not validate PolicyWeave on another architecture. The benefits of freezing the action interface and the usefulness of key-response scoring may depend on the backbone and its pretraining procedure.

The task scope is also limited. RoboCasa evaluation covers 18 Atomic-Seen tasks, each evaluated over 50 episodes. These tasks already appear in pretraining. Although target kitchens differ from pretraining kitchens, target adaptation itself uses target-domain demonstrations. The paper does not evaluate the Composite-Seen or Composite-Unseen task groups. Official RoboCasa dataset definitions.

5. Scalability is only partially characterized.

At the beginning of each episode, the method still scores every expert. A shared base pass and low-rank computations reduce the cost, but scoring and adapter storage grow linearly with the number of experts. Sparse execution does not eliminate the full-bank search. Table 18 provides useful measurements, but only up to 18 experts, using 250 recorded contexts from five tasks. Latency is measured with a batch size of five and the entire model and expert bank already resident on the GPU. This does not establish single-request end-to-end latency or selection reliability for substantially larger banks.

The evaluation should distinguish computational scaling from interference caused by additional candidates. Adding increasingly similar and unrelated experts while measuring closed-loop success, routing accuracy, selected-set size, latency, and memory would be informative. Because the implementation applies updates in low-rank form, the paper should also distinguish mathematical equivalence to merged weights from empirically demonstrated inference overhead.

Questions for authors

1. Can you provide a complete contribution decomposition?

On the same expert bank and evaluation episodes, compare true-task-ID selection, a well-trained single-expert router, Top-1 selection using the proposed scorer, and adaptive merging. Within the multi-expert subset, how does merging perform relative to the task-matched expert, and does it help differently when Top-1 identification is correct versus incorrect?

2. Are the co-training baselines sufficiently optimized and computationally comparable?

Please identify the exact simulated SFT checkpoints, training budgets, sampling strategies, and interface configurations. Can stronger joint policies be evaluated under aligned protocols, and does PolicyWeave retain its benefits when applied to a stronger expert bank?

3. Can the routing rule transfer without redesign to newly introduced tasks and another backbone?

Please clarify whether scorer design and layer selection used results from all evaluated tasks. A task-disjoint development/evaluation split and a bounded experiment on another VLA would help establish the scope of the method.

1 comment

Dear Reviewer Fopg,

With the discussion period nearing its end, we would like to follow up to see if our response has resolved your questions. We would be glad to provide any further clarification if needed.

Best regards, The Authors

Comments on the paper

2 comments
Area_Chair_QfVUOpenReview ↗

Please check author's repsonse

Dear Reviewers,

Please read the authors’ response and let us know whether it addresses your concerns. If any concerns remain, please briefly indicate which ones.

Thank you for your time and for your thoughtful reviews.

Best, AC

AuthorsOpenReview ↗

Summary of rebuttal

Dear Area Chair and reviewers,

Thank you for your time. Before the discussion period ends, we would like to briefly summarize the review status and our main contributions.

Review Status

  • Reviewers Af1e, yDMx, and SFKj all confirmed that our rebuttal addressed their main concerns and gave their Weak Accept ratings.
  • For Reviewer Fopg, we answered all questions and added the requested experiments and analyses. Unfortunately, Reviewer Fopg did not join the discussion despite our follow-up.

Main Contributions

  1. Shared action interface for merge-compatible post-training: Keeping the base action interface frozen during LoRA post-training makes independently trained VLA experts compatible for parameter merging without changing the model architecture. Across two different backbones (GR00T N1.5 and RLDX-1) and both SFT and RL, this simple practice greatly improves existing static merging methods and even lets fixed weight averaging transfer to unseen tasks. We think this provides a useful insight for VLA and model merging community.
  2. Dynamic sparse merging as an alternative to co-training: Built on this shared interface, PolicyWeave dynamically merges experts without training a router and outperforms joint co-training in both SFT and RL, with the clearest advantage in RL. This shows a different way to obtain a generalist multi-task policy: training task-specific experts independently (especially with RL) and merging them at deployment.

Thank you again for your time and consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.