acceptodds
Withdrawn from ICLR 2027

WAM-Adapter: Efficiently Adapting Video-DiT World Representations to Robot Actions

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Pretrained Video Diffusion Transformers encode rich spatiotemporal priors from large-scale videos, making them promising backbones for robot control. However, efficiently adapting them into World Action Models remains challenging. Existing approaches reduce video-adaptation or action-prediction costs, but it remains unclear whether representations produced during future-video adaptation can directly support lightweight action prediction. To address this gap, we propose WAM-Adapter, a parameter- and training-efficient framework that freezes the Video-DiT backbone, adapts it with LoRA, and reuses multi-layer representations for action prediction. A lightweight readout organizes multi-layer K/V representations from current-and-future processing into Current, Early-Future, and Late-Future groups, applying group-specific cross-layer fusion for action-chunk prediction. Inference requires a single Video-DiT forward over the current observation and noisy future slots, without future-video generation or iterative action denoising. Without embodied pretraining, WAM-Adapter uses only 121M trainable parameters with a compact Wan-1.3B backbone, achieving 89.0% success across 50 RoboTwin 2.0 tasks, 98.5% on LIBERO, and 75.17% on LIBERO-Plus. To our knowledge, this is the smallest reported trainable-parameter budget for directly trained WAMs without embodied pretraining. Under the same setting, WAM-Adapter improves training throughput by 9.66 while using 98% fewer trainable parameters than Fast-WAM. These results show that Video-DiT representations formed during world-model adaptation can be efficiently reused for strong robot control.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Action Models, Video Diffusion Models, Robot Manipulation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_kfeQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces WAM-Adapter, an efficient framework for adapting pretrained Video-DiT models to robot control. The method combines LoRA adaptation, temporal-group-specific cross-layer K/V fusion, and lightweight action decoding to enable single-pass action prediction. Experiments on RoboTwin 2.0, LIBERO, and LIBERO-Plus demonstrate competitive performance with only 121M trainable parameters.

Critical strengths
  1. Strong efficiency-performance trade-off: WAM-Adapter achieves competitive robot manipulation performance with relatively few trainable parameters and reduced computational costs.

  2. Solid experimental results: The method performs well across multiple benchmarks, and the ablation studies demonstrate the effectiveness of its main architectural components.

Critical weaknesses
  1. Limited methodological novelty: The main components of WAM-Adapter, including LoRA adaptation, multi-layer feature reuse, and lightweight action decoding, are closely related to existing approaches such as Light-WAM (Li et al., 2026) and Faster-WAM (Zhao et al., 2026). The primary new design appears to be the Current/Early-Future/Late-Future grouping strategy, which is a relatively incremental modification of existing cross-layer fusion mechanisms. The paper does not sufficiently establish a fundamentally new modeling principle or capability beyond these architectural refinements.

  2. Limited evidence for the unique contribution of the proposed fusion mechanism: Although the ablations show that temporal grouping improves performance, they mainly compare the proposed design with simplified variants. The paper lacks a direct comparison with existing cross-layer fusion approaches under matched settings. Consequently, it remains unclear whether the improvements arise from the proposed temporal grouping itself or from the overall combination of established techniques.

[1] Light-WAM: Li, Z., et al. (2026). Light-WAM: Efficient World Action Models with State-Fusion Action Decoding. arXiv preprint, arXiv:2606.08242.

[2] Faster-WAM: Zhao, W., et al. (2026). Faster-WAM: Efficient Inference-Time Future Conditioning for Robust World Action Models. arXiv preprint, arXiv:2608.04404.

Questions for authors

1: What is the fundamental methodological contribution of temporal-group-specific K/V fusion beyond existing cross-layer fusion approaches such as Faster-WAM? Could the authors clarify what new modeling capability it introduces?

2: Could the authors provide a controlled comparison between the proposed temporal grouping and existing fusion mechanisms under the same backbone, decoder, and training budget? This would help establish whether the improvement comes specifically from the proposed design.

Reviewer_mZC7OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces WAM-Adapter a method that agregates LoRA training instead of full fine-tuning, KV temporal grouping, and a direct regression action expert, claiming cheaper and faster training, while still reaching good performance on multiple benchmarks. The authors validate their model on three simulated benchmarks and some real robot tasks.

Critical strengths

I see two main strenghts in this work:

  1. The idea of temporal grouping is simple but seemingly effective in improving long horizon tasks.

  2. Most current learned policies use a diffusion / flow matching action expert, this work uses a simple direct regression and still gets competitive results, by using the flow matching model's KVs. This is a very interesting result that the authors could easily analyze further.

Critical weaknesses

The paper in its current state has several critical weaknesses:

  1. In the figures and beginning of the text, training efficiency and number of parameters are clearly made to be the main selling points of the paper. However, the training throughput comparison needs detailing, and the impact of LoRA needs to be made clearer efficiency-wise, see the questions section.

  2. Notations can be confusing and writing is unclear, making the method which is decently simple much harder to understand. A third of the paper is spent giving substantially more details than necessary on the method, details that could have been one pseudocode and one or two sentences, with the rest in the appendix or simply seen in the provided code. Streamlining the method section would leave room for more extensive analyses into why your model works, or aditional ablations.

  3. You cite FasterWAM (Zhao et al) and use it as a baselines but omit FasterWAM (Ma et al), which seems to match your method on libero, and outperform WAM-Adapter on LIBERO-Plus and Robotwin, with almost half your latency.

  4. This paper mainly compares to FastWAM, and isn't close to competing with SOTA on the benchmarks that are used. This makes the comparison more important than the absolute number, but this comparison is obscured by the difference in generative backbone. The method might be faster, and maybe even better than shown here if it were to use Wan2.2 just like FastWAM. All comparisons would then be a lot more meaningful, and

Questions for authors

Q1: Training efficiency is one of the main contributions of this method. You claim a ~10x throughput improvement over fast wam, but don't give a lot of details of how you do the training itself : are the models compiled, do you precompute the VAE latents (the VAE used by FastWAM is rather slow), and could you provide more details in general?


Q2: Could you run an ablation showing the impact of your action expert compared to a classic flow matching expert? This would give useful insight into what part of your model actually improves performance, and if it could be pushed even further.


Q3: You compare the final performance of different LoRA configurations to full finetuning, could you also provide the efficiency statistics of these trainings? Showing the tradeoff between training efficiency and loss in performance should be a major point in this work. Given that the error rate on libero long almost doubles when going from finetuning to LoRA-64, some crucial results to understand the performance of your method are missing.


Q4: Could you provide a latency breakdown of your measurements, what part is backbone KV prefill, action expert, encoders etc? These numbers would be useful here, especially if also provided with the FastWAM backbone which is much bigger.


Q5: Minor issue : You need to add moe detail in some of the figure titles. "Figure 4: Backbone adaptaion" for example is not descriptive enough at all and forces the reader to go back to the text when it should be clear from the description.

Reviewer_sSzjOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces WAM-Adapter, a method to efficiently adapt a video diffusion model into a world action model. The key idea is to read out actions from the temporally-grouped KV of a single video DiT forward pass. The number of trainable parameters is small: the DiT is LoRA finetuned, and the action head is a lightweight transformer adapter. This allows the WAM to inherit the pretraining knowledge of the video DiT for action prediction while only training a small number of parameters.

Concretely, the finetunine pipeline consists of the following steps: (1) take a frozen video-DiT, e.g. WAN 2.1, (2) sample a batch of frame-action trajectories from the robot dataset, (3) sample a random flow timestep τv\tau_v, (4) corrupt future frames and query the video-DiT for future noise prediction, (5) group the intermediate KV across all layers into three groups by time: current, early-future, and late-future, (6) aggregate KV in each group using group-specific projection layers and weights, (7) pass these aggregated KV to a transformer action readout head, (8) LoRA-finetune the DiT with flow-matching loss and train the action head with L2 regression loss. During inference, the model (1) queries the LoRA-finteuned video DiT with flow timestep τv=1\tau_v = 1 (full noise) for its KV representations, and (2) reads out an action from those KV representations.

Experimentally, the paper evaluates WAM-Adapter on three simulation benchmarks (LIBERO, LIBERO-Plus, RoboTwin 2.0) and a real-world Agile-X benchmark. Across these benchmarks, WAM-Adapter matches or outperforms the best WAM baselines with significantly fewer number of trainable parameters and faster inference speed. These results are further corroborated by comprehensive ablations (removing future KV, removing temporal grouping, reducing the number of transformer layers to aggregate over, and varying LoRA rank) showing the contribution of each individual design choice.

Critical strengths
  1. The paper introduces an efficient method for finetuning WAMs from a video diffusion model. This removes a significant hurdle of using WAMs, namely the heavy cost of finetuning. In general, it is desirable to make WAMs accessible to the community.
  2. The simulation results are promising, matching or outperfoming WAM baselines with much lower costs.
  3. The ablations are extensive and verify the contribution of each design choice. I find Figure 5 and Figure 6 particularly intuitive for understanding the role of timestep grouping.
Critical weaknesses
  1. My main reservation for this paper is the underwhelming real-world results. Across the three real-world tasks, WAM-adaper achieves a maximum of 35% success rate. I wouldn't consider these tasks solved, nor would I consider 35% over 25% as carrying any real evidence that the method is better than the LightWAM baseline. I encourage the authors revise their real-world results by choosing more representative tasks and adding more baselines.
  2. On a similar note, the simulation success rates are only matching or marginally better than baselines. I understand that the main claim of the paper is efficiency not raw performance, but this claim is a bit weak on its own. If all I get with efficiency is the same success, then there are much simpler ways to do so, e.g. quantization, compilation. One can even argue that as compute gets cheaper we wouldn't even need these optimizations. However, the impact would be much bigger if the authors show that efficient inference results in a step-change of behavior, e.g. for dynamic tasks (dynamic handover) the slow inference speed of a standard WAMs makes it impossible to solve the task, whereas WAM-Adapter can with its fast reactivity. Having these results would not only lift the impact of this paper, but the "efficient WAM" line of work as a whole.
Questions for authors
  1. During LoRA finetuning of the video diffusion model, you randomly sample the timestep τv\tau_v. But during inference, you use τv=1\tau_v = 1. Why not fix τv=1\tau_v = 1 during finetuning?
  2. The action readout model is a deterministic transformer. As a result the whole policy is deterministic. Do you notice any degredation by dropping the stochasticity and multimodality?
  3. How are the early and late groups selected? From Appendix A it seems to come with the WAN model. From my understand of WAN, each latent frame represents multiple real frames. Can you describe the exact time window of early and late future?
  4. In the experiment section, can you add brief descriptions of the baseline methods? This will make the paper more self-contained.
  5. Table 10 does not bold columns where WAM-adapter is not the best method.
  6. Algorithm 1 does not have τv\tau_v as input to the flow model.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.