acceptodds
Under review as a conference paper at ICLR 2027

FedWAM: Federating Predictive World Knowledge for Personalized Robot Control

Abstract

World-action models (WAMs) can benefit from experience collected across sites, but data ownership and heterogeneous model architectures complicate joint training. We introduce FedWAM, a framework that shares predictive world knowledge while retaining local demonstrations and private action experts. Clients upload conditional chunk-end predictions in a common visual feature space. A client-balanced server teacher integrates these contributions and is broadcast to all participants. Each recipient calibrates teacher supervision against local future observations, distills it into its native world pathway, and adapts its action expert to the updated context. Repeated exchange combines shared predictive learning with personalized control. The interface accommodates both LaWAM's future features and ImageWAM's internal control representations. In the reported comparisons, FedWAM reaches 78.4% LIBERO-Plus success versus 76.3% for local training, with most gains outside the recipient's training domain, while reaching 97.9% on original LIBERO. Mixed-architecture comparisons improve both recipient families, with direction-dependent donor transfer; three Piper tasks provide a complementary physical comparison. These findings motivate assessing shared predictors through both feature accuracy and the behavior of locally adapted controllers. Project page: https://submit-anonymous.github.io/FedWAM.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Federated Learning, World Action Models, Robot Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_MfGjOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Paper introduces federated learning for WAMs without sharing demos or weights. Clients upload DINOv3 features, a server fits a teacher to them, and each client distills the teacher back into its own world model and retrains its action expert. Claims 2 pp in sim, mostly on unseen domains, and +5 pp on every real robot task.

Critical strengths
  • The design makes sense, sharing chunk features instead of weights means you don't need matching architectures, and the self-distillation baseline (fit a teacher only from your own predictions) is the right control for whether the gain comes from other clients or just from the distillation step.
  • The authors are upfront about what's missing.
Critical weaknesses
  • The paper says its own results might not be real. Section 5 calls them "author-supplied point estimates" with a "source-provenance conflict". Logs, configs, checkpoints, trial counts, seeds are all listed as outstanding in appendix. That covers every table and figure in the paper. The only experiments that do have records (Appendix K) don't test the actual method.
  • The robot numbers look made up. In Table 5, every Pen-to-Basket cell is exactly Corn-to-Plate minus 5.0, and every Cube-to-Bowl cell is exactly minus 10.0. Fig 15's gain matrices are identical across all three tasks. Real robot evals usually differ significantly across different pick-and-place tasks.
  • no seeds or error bars anywhere in the main study
Questions for authors
  1. Can you clarify the provenance situation in Appendix H? Specifically, which of the reported results come from completed runs of the method in Section 4?
  2. For the Piper experiments, how many trials per cell were run, and is there an explanation for the constant 5 pp offset between the three tasks in Table 5?
  3. How many seeds were used for the sim experiments, and could you report per-seed results?
Reviewer_4tSJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes FedWAM, a framework for federated knowledge sharing among World-Action Models (WAMs) with heterogeneous architectures and private action experts. Instead of aggregating model parameters, FedWAM maps client predictions of future visual features into a shared DINOv3 feature space. A server-side conditional Transformer is trained on client-balanced prediction packets and subsequently provides teacher supervision to local models. Each client calibrates the distillation strength using prediction errors on locally observed futures and adapts its private action expert to the updated world representation. The evaluation includes LIBERO, LIBERO-Plus, mixed LaWAM/ImageWAM populations, and three physical robot tasks. The reported results show an improvement from 76.3% to 78.4% over local training on LIBERO-Plus, with most gains occurring outside each client's training domain. The paper also investigates asymmetric cross-architecture transfer and the relationship between predictive accuracy and downstream control performance.

Critical strengths
  1. A practical predictive interface for heterogeneous WAMs. The proposed feature-space sharing mechanism is a meaningful design choice for federated robotic learning. By communicating conditional future predictions instead of model parameters, FedWAM can accommodate different internal world representations while keeping native action experts local. The implementations for LaWAM and ImageWAM illustrate how this interface can be connected to distinct model architectures.
  2. A useful evaluation perspective on knowledge transfer and control. The paper distinguishes between in-domain and cross-domain performance, investigates directional transfer between different architectures, and examines the relationship between predictive feature error and closed-loop success. In particular, the observation that lower prediction MSE does not necessarily translate into higher task success is important for evaluating world-model knowledge sharing.
Critical weaknesses
  1. The main experimental results have unresolved provenance and insufficient verification. Appendix H explicitly states that the plotted comparison arrays originated from a synthetic layout-preview artifact and that their status as measured results has not been reconciled with new raw experimental records. Complete run-to-configuration and checkpoint mappings, executed training records, physical trial denominators, and paired uncertainty estimates also remain outstanding. This is a substantial concern because the main empirical claims rely on these comparison arrays. Although the authors transparently acknowledge the issue, numerical reproducibility of the figures is not equivalent to independent verification of the underlying experiments. The historical measurements in Appendix K do not independently validate the complete proposed FedWAM procedure.
  2. The evidence for the incremental benefits of federated and heterogeneous knowledge transfer remains limited. On LIBERO-Plus, FedWAM improves average success by 2.1 percentage points over Local but only 0.9 points over Self-distillation. Without uncertainty estimates or repeated training runs for the main comparisons, the robustness of this incremental benefit is unclear. Furthermore, the matched-donor experiments show asymmetric cross-architecture transfer: ImageWAM donors improve LaWAM recipients by 0.5 points, whereas LaWAM donors reduce ImageWAM performance by 0.2 points. Thus, mixed-pool improvements do not conclusively establish consistently beneficial heterogeneous knowledge transfer. The evaluation is also restricted primarily to visual-domain shifts with a shared embodiment and task set, limiting the generality of the conclusions.
Questions for authors

Q1. Experimental provenance and verification. Could the authors clarify the discrepancy documented in Appendix H between the synthetic layout-preview artifact and the subsequently authorized experimental results? Specifically, can the authors provide original run logs, checkpoint identifiers, executed configurations, evaluation manifests, and trial-level records corresponding to the main results in Tables 1–5? Which results can currently be independently verified from raw experimental evidence? Q2. Statistical reliability of the reported improvements. Can the authors provide repeated-training results and uncertainty estimates for the main comparisons, especially the 0.9 percentage-point improvement over Self-distillation on LIBERO-Plus? For the physical experiments, can the authors clarify the number of trials, sessions, and independently trained models underlying the reported success rates? These details are necessary to assess whether the reported improvements are robust. Q3. Contribution of heterogeneous knowledge sharing. The matched-donor comparison reports negative transfer from LaWAM donors to ImageWAM recipients. Can the authors clarify how much of the mixed-pool improvement is attributable specifically to cross-architecture knowledge, beyond self-distillation and same-architecture sharing? A controlled comparison that adds cross-architecture donors to an otherwise identical same-architecture pool would help establish the contribution of heterogeneous federation.

Reviewer_CXnzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FedWAM proposes sharing future-feature predictions across robot clients while keeping their demonstrations and action experts local. A shared teacher and local calibration are used to transfer this information into personalized control.

Critical strengths
  • Sharing predictions rather than model parameters is a reasonable approach to collaboration between heterogeneous world-action models without pooling raw demonstrations.
  • The paper includes useful controls such as self-distillation, same-architecture sharing, and calibration ablations, which help examine the contributions of different components.
Critical weaknesses
  • The motivation relative to stronger use of public data could be clearer. Although public datasets cannot necessarily replace private, domain-specific experience, it is unclear whether broader public-data pretraining followed by local fine-tuning could achieve comparable improvements with less complexity.
  • The gains may not justify the added training and communication costs: LIBERO-Plus improves by only 2.1 points over Local and 0.9 over self-distillation, while some physical-task results decline in-domain. Reporting statistical variability and total costs would clarify the practical value.
Questions for authors
  • What does federation add beyond stronger public-data pretraining and local fine-tuning, when adaptation and total computational budgets are matched?

  • How much computation and communication are required to obtain the reported gains, and how stable are these gains across independent training runs?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.