acceptodds
Under review as a conference paper at ICLR 2027

RoboFL: Federated Expert Assembly for World-Action Models

Abstract

Vision-language-action and world-action models are increasingly popular, yet remain bottlenecked by physical interaction data that is scarce, institutionally siloed, and task-heterogeneous. A natural federated solution is to let each client adapt a shared foundation model through parameter-efficient fine-tuning, avoiding the exchange of full-model updates. However, federating these adapters is nontrivial, as naive aggregation can entangle incompatible updates, while incorporating MoE-style routing into federated aggregation may dilute specialization and destabilize expert selection. We present RoboFL, which instantiates MoSAIC (Mixture of Slotted Adapters) for federated world-action learning. MoSAIC directly installs locally trained LoRA adapters as the expert branches of a server MoE. Server-side routers learn token assignments over these prior-informed branches while jointly refining routing and expert parameters. Foresight-to-Action Routing Distillation (FARD) aligns routing across the model's three paths, while Path-Consensus Expert Aggregation (PCEA) converts complete expert updates into a compact global adapter for personalized redistribution. Experiments on RoboTwin 2.0, RLBench, and a real-world Franka robot arm show the superiority of RoboFL with structured expert assembly, as it outperforms centralized PEFT InternVLA-A1 by 12.23% on the Franka arm, while reducing per-round client communication by up to 86.81% relative to MoE-based federated VLA baselines.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Federated learning, World-Action Models, Parameter-efficient Finetuning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 5 reviews, above 85% of the others.
Reviewer_9V1KOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces RoboFL, a method for collaboratively training VLA and WAM models on distributed robotic data using Federated Learning. To this end, clients locally train LoRA adapters for a pretrained model and transmit these adaptations to a server. On the server, the received adapters are assembled into experts of a Mixture-of-Experts model using MoSAIC and further trained together with the routers on additional server data. With FARD, action routing is additionally trained based on the routing distributions of the understanding and generation paths. Subsequently, the server-refined experts are combined into a global LoRA adapter using PCEA, which is then combined with the respective expert of each client and sent back for the next FL round. Evaluations on RoboTwin 2.0, RLBench, and a real-world Franka robot show overall better results than the evaluated FL baselines, although RoboFL performs worse than centralized approaches on RLBench.

Critical strengths
  • Separating local LoRA training from server-side expert combination is a reasonable approach for highly heterogeneous task distributions. In particular, I find it convincing that local specializations are not immediately lost through parameter averaging.
  • The evaluation on six different tasks using a real-world Franka robot is a positive aspect. This extends the investigation beyond simulation and shows that the method can also work under realistic conditions.
  • The authors show that RoboFL is comparable to FedAvg in terms of client-side communication and memory requirements while achieving better results on RoboTwin (Table 3).
Critical weaknesses
  • The authors assume that the server has its own training data. How does this training data influence the selection of MoE experts? A quantitative investigation in this context should be performed.

  • The relationship between tasks on the server that are very similar to client tasks, and vice versa, should be investigated. Currently, it is not entirely clear how the server's training data influence the selection or decision-making within the MoE.

  • Some design decisions of the method, particularly the weighting of experts within PCEA and the equally weighted combination of global and local adapters, are not sufficiently justified through experimental investigations. Therefore, it remains unclear what additional advantages these decisions offer compared to simpler alternatives.

  • The experiments were conducted using only one seed each for training and evaluation. Multiple seeds would be necessary to assess the robustness of the results.

  • The authors extensively investigate client-side communication overhead and memory requirements (Table 3). However, RoboFL requires additional computations and a larger MoE model on the server. A corresponding investigation of server-side computational costs and the overall training effort is missing.

Minor Weaknesses:

  • Stronger, controlled comparisons with simpler LoRA aggregation or personalization approaches are missing.

  • Although the methods are generally very promising and the results are also very convincing, the overall method is sometimes very difficult to follow, and the connections between individual steps or methods are difficult to understand. Parts of the method section are described too implicitly. MoSAIC could be explained in more detail, particularly how its individual methods are connected and executed.

Questions for authors
  • The method assumes that the routing distributions across the three model paths can be meaningfully compared. What justifies this assumption?

  • Where does the server's training data come from in a real-world setting?

  • Why was no mixing ratio other than 50/50 between the local expert and the global adapter investigated?

  • How is the reliability of the constructed teacher distribution validated, and to what extent does agreement in routing across different paths correspond to the selection of experts that improve action prediction?

Reviewer_DF4fOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RoboFL addresses federated learning of world-action models under heterogeneous, task-siloed robot data and limited client resources. Its MoSAIC framework installs locally trained LoRA adapters into fixed server-side expert slots, then jointly refines the experts and routers using the server's own task data. FARD uses agreement between understanding and generation routing distributions to guide action routing. PCEA combines server-refined expert updates into a compact global adapter for personalized redistribution. Experiments on RoboTwin 2.0, RLBench, and a physical Franka robot report improvements over the evaluated federated baselines

Critical strengths
  1. The problem setting is meaningful for a practical seperation of local specialization and server-side corrdination for robotics. Making client adaptors as distinct experts avoids immediately averaging heterogeneous updates.
  2. The reported results on both simulation benchmarks and real world Franka tasks show improvements across simulation and physical manipulation.
Critical weaknesses
  1. There is insufficient validation of knowledge transfer from the server mixture to individual client adapters. For example, Equation 4 combines the server-refined expert, not the original local adapter, with a compressed global update and applies another rank trunctation. The paper does not quantify how client-specific capabilities are affected by different training stages.
  2. There is limited evidence for the specific benefits of cross-path consensus. Table 10 reports only 0.48 and 0.7 improvements from FARD and PCEA.
Questions for authors
  1. Could you report client-task performance for the original uploaded adapter, the server-refined expert used independently, and the final redistributed adapter?
  2. How do you assess whether the refinement from expert parameter replacement, router retaining and joint server-side update restores router-expert compatibility on client-task distributions? As server trains only on its own residual task group.
Reviewer_ziEcOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes RoboFL, a federated learning framework for world-action models under a task-silo setting, where each client owns a disjoint family of manipulation tasks. Each client trains one rank-16 LoRA adapter on InternVLA-A1-3B and uploads it. The server installs the uploaded adapters as the expert branches of a layer-wise LoRA-MoE (MoSAIC) and trains routers and experts on its own held-out task group. FARD distills understanding-generation routing consensus into the action router. PCEA weights the complete expert updates by three-path routing agreement and compresses them into a rank-16 global adapter, which is blended 1:1 with each client's server-refined expert before the next round. Experiments on RoboTwin 2.0, RLBench, and a real-world Franka robot show improvements over several federated learning baselines.

Critical strengths
  1. The paper studies a practical problem in federated robot learning, especially when clients have different task distributions.

  2. Combining locally trained LoRA adapters as server-side experts is a nice idea, while keeping the communication cost low.

  3. The experiments are conducted on both simulation benchmarks and real-world robots, which makes the evaluation more convincing.

Critical weaknesses
  1. Unclear evaluation setup. Figure 1 suggests inference with MoE, while Appendix A.1 states that the deployed model is a single redistributed LoRA adapter. It is unclear which model is used to obtain the reported results.

  2. Potentially unfair comparison. The server uses its own task data (e.g., 7 out of 50 RoboTwin tasks) to train routers and refine experts. However, it is unclear whether the federated baselines have access to the same data and server-side computation, making it difficult to isolate the contribution of expert assembly.

  3. Missing baselines. Several relevant federated LoRA methods, including FLoRA, FRLoRA, FedSA-LoRA, and LoRA-A2, are discussed in related work but not included in the experiments. These would provide useful comparisons for evaluating the proposed aggregation strategy.

  4. Limited gains from FARD and PCEA. Table 10 shows improvements of only 0.48 and 0.70 percentage points from FARD and PCEA, respectively. Since the results are based on a single seed, it is difficult to assess whether these gains are consistent.

  5. Minor presentation issues. There are several typos and inconsistencies, including “FogreVLA” (Figure 1), “ROBOFLperforms” (p. 7), “RLBench MoLe single-view” (Figure 9), “fedforesight_category” (Appendix A.3), and inconsistent use of “Vanilla RoboFL” and “vanilla MoSAIC.”

Questions for authors
  1. Which model produces the reported success rates: the server MoE, the global adapter, or a specific client's personalized adapter?
  2. How much access did each FL baseline have to the server-held tasks, and what does FedAvg reach with the same server-side fine-tuning?
  3. What training budget (steps, epochs, batch size) did the centralized InternVLA-A1 and Motus references use on each benchmark?
  4. How does a router trained only on server-held tasks select experts for client tasks it never sees during server training?
  5. What success rate does RoboFL reach on RoboTwin when the action router is uniformized or scrambled?
  6. How many demonstrations per task do the clients and the server hold in each benchmark?
  7. How large are the FARD and PCEA gains across several training seeds?
Reviewer_7EwROpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes RoboFL, a federated learning framework for world-action models (WAMs). It is built on InternVLA-A1-3B, a three-path (understanding, generation, action) mixture-of-transformers. Each client trains a single LoRA adapter on its own disjointed task silo. The server (MoSAIC) places each uploaded adapter into its own expert slot in a mixture-of-experts layer, with one expert per client. It then learns a router and fine-tunes the experts on data the server holds from a separate group of tasks. Foresight-to-Action Routing Distillation (FARD) adds a JS-divergence loss that pulls the action-path router toward the geometric mean of the understanding and generation routers, with each sample's loss weighted by a reliability score (U–G agreement and teacher concentration). Path-Consensus Expert Aggregation (PCEA) weights the refined expert updates by Hellinger-barycenter routing evidence. It compresses them by truncated SVD into one global adapter of rank at most 16. The global adapter is averaged 50/50 with each client's server-refined expert and sent back. On RoboTwin 2.0 with 50 tasks and 8 clients plus a server group, the method reaches 83.12%, against 78.92% for FedAvg and 81.60% for centralized InternVLA-A1. On an 8-task RLBench suite with 4 clients plus a server group, it reaches 41.25%, below the 52.75% success rate of the centralized. On six Franka tasks it reaches 59.17%, against 46.94% for centralized. Per-round client communication drops by up to 86.81% relative to client-side MoE federated baselines.

Critical strengths
  • A practical recipe that keeps client cost at single-adapter level and still beats the centralized baselines on RoboTwin 2.0 and the federated baselines on both RoboTwin 2.0 and RLBench. In RoboFL each client trains and sends only one rank-16 LoRA adapter (40.85M parameters, 311.63 MiB per round, 8.33 GiB peak memory), the same as FedAvg and about 7–8× lower than FedVLA in parameters and communication. The multi-expert part runs only on the server, so clients never hold, train or send more than one adapter, whereas FedMoE and FedVLA send up to 8 experts per round. Despite this low cost, RoboFL reaches 83.12% success on RoboTwin 2.0, above FedAvg (78.92%), FedVLA (79.32%) and FedMoE (75.12%). The 86.81% communication saving follows from the Table 3 numbers. Why it is consequential: client cost is a real barrier to federating large robot policies, and the paper shows that a design with no extra client overhead can still outperform heavier client-side MoE methods.

  • A broad and transparent empirical evaluation. The paper evaluates on all 50 RoboTwin 2.0 tasks with 100 trials each, an 8-task RLBench suite, and six real Franka tasks (3 rounds of 20 trials each, reported as mean ± SD). Every federated baseline uses the same task silos and the same federated optimization settings. The authors also report weaknesses openly, including the 11.5-point gap to centralized InternVLA on RLBench (41.25% vs 52.75%), a failure analysis with failed rollouts for all three settings (App. C.4), a limitations section, and the statement that single-seed differences of a few points are descriptive. The real-robot gain is large: 59.17% success against 46.94% for centralized InternVLA, 35.56% for ForgeVLA and 33.61% for FedVLA (+12.2, +23.6 and +25.6 points), and larger than the reported SD (about 7.6 points).

  • A sound motivation for PCEA's aggregation step. Eq. 8 correctly notes that averaging LoRA factors separately is not the same as averaging whole updates, since (∑ωB)(∑ωA)≠∑ωBA(\sum \omega B)(\sum \omega A) \neq \sum \omega B A. Weighting complete updates and reconstructing a low-rank adapter avoids that error. Whether the specific weights help is a separate question, which I raise in the weaknesses.

  • Clear exposition of the pipeline. The paper separates what clients do (single-adapter PEFT) from what the server does (routing and expert refinement). It also states the notation clearly, including the difference between the pre-top-k routing probabilities qq and the renormalized top-k weights π\pi.

Critical weaknesses
  • The gains from FARD and PCEA are very small. On RoboTwin 2.0, they add only +0.48 and +0.70 points (Table 10, App. C.1) from a single training run (App. D), which a repeat run could easily change. Fig. 3 also shows the gains are mixed, since FARD lowers the number of tasks above 90% success from 19 to 17 (Sec. 4.3).

  • On RLBench, the comparison with baselines may be unfair. RoboFL's server trains on 2 extra tasks that the clients don't have (put rubbish in bin and phone on base), and the paper doesn't say if the baselines get this data too. FedAvg scores 0% on both tasks while RoboFL scores 26% and 50%, which accounts for most of RoboFL's lead.

Questions for authors
  1. Server data. RoboFL's server trains on its own tasks (for example, 2 of the 8 RLBench tasks), but the paper doesn't say whether the baselines get the same data. This matters because on those 2 tasks FedAvg scores 0% while RoboFL scores 26% and 50%. Could you (a) say whether the baselines were trained on the server's data and (b) add a FedAvg baseline that gets the same server training? It would also help to show how RoboFL performs when the server has less data, or none.

  2. Router vs. experts. The paper says most of the gain comes from "retaining task-trained experts and learning their composition on the server" (App. C.1). But the server also trains the adapters themselves on its own data, so the gain could come from that extra training instead. To check, could you run a version where the adapters are kept fixed, exactly as the clients sent them, and only the router is trained? If it still performs well, the routing idea is what helps. If not, the extra server training is doing most of the work.

  3. RLBench task choice. The 8 RLBench tasks appear to be a subset of the 10 used by MoLe-VLA (Zhang et al., 2026), specifically Fig. 9 calls the setup "RLBench MoLe single-view". The tasks frame off hanger and change clock are left out, and the paper doesn't say why. Could you explain this choice and report results on all 10 tasks, or on a larger standard set such as the 18 tasks used by RVT (Goyal et al., 2023) and RVT-2 (Goyal et al., 2024)?

Reviewer_1mjWOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on practical issues in Vision-language-action and world-action models utilizing a federated solution. To address these issues, it introduces ROBOFL for federated world-action learning. Specifically, two components, i.e., Foresight-to-Action Routing Distillation (FARD) and Path-Consensus Expert Aggregation (PCEA), are deliberately proposed. Extensive experiments on RoboTwin 2.0, RLBench and a real-world Franka robot arm demonstrate the superior performance of the proposed method, compared with FL-based VLA methods and centralized PEFT WAMs .

Critical strengths

(+) This paper tackles practical issues in deploying vision-language-action and world-action models utilizing a federated solution, which has real-world impacts. (+) This paper conducts extensive experiments and demonstrates superior performance compared to FL-based methods and centralized methods in certain scenarios.

Critical weaknesses

(-) The effectiveness of some claimed advantages, e.g., “The fixed client-to-slot mapping preserves parameter correspondence across rounds, while still allowing flexible token-to-expert assignments”, lacks justification to support these claims.

Questions for authors
  • Thanks for including more demonstration videos in the supplementary materials. For failure cases, could you please provide more detailed analysis?
1 comment

Thanks to authors’ detailed rebuttal, especially the clear analysis of failure cases. I keep my original score to support this paper.

Comments on the paper

1 comment

We sincerely thank all reviewers for carefully reading our paper and for their constructive comments and suggestions, which have been invaluable in helping us improve our work. We have now completed our responses to all reviewers in the first round, and in light of the reviews and the discussion, we have revised the manuscript accordingly and uploaded the updated version. With three days remaining in the rebuttal period, we would be truly delighted to continue the discussion with the reviewers and warmly welcome any further comments or replies. Thank you again for your time, effort, and valuable feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.