acceptodds
Under review as a conference paper at ICLR 2027

Faster-WAM: Do World Action Models Need Deep Action Modules?

42 views

Abstract

World Action Models (WAMs) build on pretrained video models, whose representations are grounded in physical dynamics and provide a natural basis for action prediction. Despite this natural foundation, many WAMs still rely on deep, parameter-heavy action-prediction modules that incur high inference latency and may overfit to limited robot demonstrations, restricting their real-world applicability. In this paper, we advocate a world-model-centric principle that concentrates capacity and computation in the video world model, while a lightweight action expert translates the backbone's representations into executable robot actions. We realize this principle through three key choices: Dock of Transformers (DoT) with Lite KV-Fusion to give the shallow, lightweight action expert access to representations from all video layers; world-model-only conditioning of the action expert; and retracted 1D-RoPE for positional alignment between video keys and action queries. We test this principle using Faster-WAM, a world-model-centric WAM with only a single-layer action expert. Despite this restriction on action-specific computation, Faster-WAM achieves competitive control performance on LIBERO and RoboTwin 2.0 without additional embodied pretraining. It provides approximately and inference speedups over Fast-WAM and , respectively. Consistent with its world-model-centric design, Faster-WAM demonstrates stronger generalizability under distribution shifts: the same LIBERO-trained policy achieves success on LIBERO-Plus, exceeding Fast-WAM and LingBot-VA by and percentage points, respectively. Finally, real-robot experiments demonstrate success rates comparable to Fast-WAM, with substantially lower inference latency and shorter task-completion times.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Action Models, Robot Learning, Efficient Action Prediction

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_iLt3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Faster-WAM connects a pretrained video backbone to a single-layer action expert through learned fusion of cached keys and values, with world-model-only conditioning and aligned positional encodings. It retains competitive manipulation performance while reporting approximately 3.7–3.8× faster inference than Fast-WAM and improved LIBERO-Plus generalization.

Critical strengths
  1. Lite KV-Fusion exposes all video layers to a shallow action expert using only 720 learned mixing logits.
  2. Matched 32-action, ten-denoising-step comparisons show approximately 3.7–3.8× speedups over Fast-WAM while retaining competitive control.
  3. LIBERO-Plus gains, LTX-backbone transfer, and four physical tasks provide complementary evidence beyond standard LIBERO success.
Critical weaknesses
  1. No depth sweep holds fusion, conditioning, and RoPE fixed, limiting attribution of the gains specifically to a shallower action expert.
  2. Learned fusion is compared with final-layer readout, but not uniform averaging; Figure 4’s weights remain close to uniform.
  3. Reported deviations cover evaluation runs, while the training configuration uses one seed; small ablation gains lack training-level uncertainty.
Questions for authors
  1. Can you compare 1-, 2-, 4-, and deeper-layer experts with identical fusion, conditioning, and RoPE, reporting success, OOD performance, and latency?
  2. Can you compare learned per-head fusion against fixed uniform averaging and shared-across-head mixing to isolate the benefit of learned weights?
  3. Can you repeat the key conditioning and RoPE ablations across independent training seeds and report confidence intervals for their performance differences?
3 comments

W1 + Q1. Action-depth comparison

To better understand the effect of action depth, we trained additional 2-, 4-, and 8-layer action experts while retaining learned KV fusion, world-model-only conditioning, and retracted 1D-RoPE.

Action-expert depthLIBERO (%)LIBERO-Plus (%)
1 layer (original)98.32 ± 0.1478.29 ± 0.03
2 layers98.30 ± 0.2575.02 ± 0.03
4 layers98.25 ± 0.1575.50 ± 0.12
8 layers98.22 ± 0.0672.17 ± 0.27

On LIBERO-Plus, increasing depth does not consistently improve performance. In particular, the 8-layer expert achieves lower mean success than the 2- and 4-layer experts. These results are consistent with a possible OOD generalization benefit from a lightweight action expert.

It is worth mentioning that all variants achieve near-ceiling success on standard LIBERO, leaving limited room for further improvement. This may partly explain the small differences in in-domain performance across depths. On more challenging tasks, a modest increase in action depth might improve in-domain performance.

We use a single-layer action expert as the main configuration to demonstrate that DoT enables WAMs to perform well even with an extremely lightweight action module. This choice is not intended to establish one layer as the optimal depth.

We also measured the 2-, 4-, and 8-layer variants on the same consumer-grade GPU using the original timing protocol. The single-layer Faster-WAM and Fast-WAM rows report the original paper benchmarks.

ModelAction depthAction generation (ms)Phase sum (ms)
Faster-WAM (original)15.84 ± 0.1052.04 ± 0.18
Faster-WAM213.65 ± 0.5362.23 ± 0.82
Faster-WAM422.86 ± 0.9572.24 ± 1.19
Faster-WAM840.33 ± 1.5691.00 ± 1.71
Fast-WAM (original)30150.12 ± 1.65196.03 ± 1.67

Among the Faster-WAM variants, additional action depth increases inference cost without a consistent improvement in OOD performance.


W2 + Q2. Uniform and shared-across-head fusion

We trained two additional variants of Lite KV-Fusion: fixed uniform fusion, which assigns a weight of 1/30 to each video layer, and learned head-shared fusion, which learns one set of layer weights shared across all attention heads. Faster-WAM instead learns a separate set of layer weights for each head. All other architectural choices follow the final Faster-WAM model.

FusionLIBERO (%)LIBERO-Plus (%)
Fixed uniform fusion98.20 ± 0.1573.07 ± 0.15
Learned head-shared fusion98.43 ± 0.1674.62 ± 0.26
Learned per-head fusion (Faster-WAM)98.32 ± 0.1478.29 ± 0.03

Fixed uniform fusion yields lower mean success rates on both benchmarks, with a small difference on LIBERO and a larger gap on LIBERO-Plus. Although the learned weights are close to uniform, their numerical proximity alone does not determine their effect on the fused representations. Differences in key norms and amplification through subsequent computation may contribute to this effect.

Learned head-shared fusion achieves similar standard LIBERO performance to per-head fusion, while its LIBERO-Plus success rate lies between fixed uniform and learned per-head fusion. This ordering is consistent with OOD benefits from both learning the layer weights and allowing them to vary across attention heads. As noted above, all three fusion variants achieve near-ceiling success on standard LIBERO, leaving limited room for further improvement.

Learning head-specific fusion weights allows the readout to adapt to the video backbone and dataset. Both uniform and head-shared fusion are special cases of this parameterization, which retains these options without imposing either in advance.

W3 + Q3. Training-level uncertainty

We agree that evaluations across multiple seeds and independent training runs capture different sources of uncertainty. Our reported means and standard deviations are computed over three action-noise seeds for each fixed checkpoint. These quantify evaluation variability, while independent training runs are needed to assess training variability.

Obtaining confidence intervals that account for training randomness requires independently training and evaluating each ablation configuration multiple times, which entails substantial time and computation for large WAM/VLA models. Given the limited time and computational resources available during the rebuttal period, we may not be able to complete a full study of the conditioning and RoPE ablations across multiple independent training seeds. We will make every effort to complete additional training runs and their evaluations, and aim to include the results and confidence intervals for the corresponding performance differences in a subsequent revision.

For context, the LIBERO results in Fast-WAM (Table 2) and the LIBERO-Long results in Motus (Table 9) are reported as point estimates without accompanying standard deviations or confidence intervals. By additionally reporting means and standard deviations across repeated evaluations, we make evaluation variability explicit and provide a more informative basis for comparison than point estimates alone. The relatively small deviations observed in our evaluations provide useful evidence of the repeatability of the evaluated checkpoints.

Thanks for the authors' response. I will keep my positive score. Please be sure to include the discussion into your final version (if accepted).

Reviewer_76ajOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether World Action Models need deep action modules and proposes Faster-WAM, which pairs a single-layer action expert with a jointly fine-tuned Wan2.2-5B video backbone. Lite KV-Fusion learns head-specific weights to combine cached keys and values across video layers, allowing the shallow expert to use multi-level representations without additional dense KV reprojection. Language and proprioception reach the action expert through the video backbone, while retracted 1D-RoPE aligns the positional encodings used in video–action attention. Without additional embodied pretraining, the method achieves competitive results on LIBERO and RoboTwin 2.0, with approximately 3.7–3.8× speedups in profiled model computation over Fast-WAM. The same LIBERO-trained policy achieves 78.3% success on LIBERO-Plus, compared with 51.5% for Fast-WAM. The authors also provide component ablations, experiments with LTX-Video, and evaluations on four real-robot Franka tasks.

Critical strengths
  1. Strong OOD performance with a shallow action expert. Using its LIBERO-trained policy without further training, Faster-WAM achieves 78.3% overall success on LIBERO-Plus, compared with the reported 51.5% for Fast-WAM, with improvements in all seven perturbation categories. It also maintains strong LIBERO performance (98.3% vs. 97.6%). The same trend appears with LTX-Video, where LIBERO-Plus success improves from 43.0% to 53.6%. These results provide useful evidence that strong control and robustness can be achieved with a shallow action expert.

  2. Substantial inference speedup with a lightweight interface. Lite KV-Fusion gives the single-layer action expert access to representations across the video backbone’s layers using only 720 additional mixing parameters. The resulting architecture achieves a 3.7–3.8× speedup over Fast-WAM in profiled model computation, with both methods using 32-action chunks and 10 denoising steps. A comparable speedup is reported on the real-robot workstation, where per-call inference latency drops from 502 ms to 129 ms. The phase-wise timings show that most of the saving comes from action generation, making this a practically relevant improvement for frequent replanning.

Critical weaknesses
  1. Incomplete attribution of the OOD gains. Figure 3 provides useful sequential ablations, but it does not isolate action depth while keeping fusion, conditioning, and positional encoding fixed. The main Fast-WAM success rates are also taken from prior work, and the comparability of the training recipes and checkpoints is not fully established. The results support the overall design, but provide less direct evidence that reduced depth itself better preserves pretrained generalization. A controlled depth comparison and clarification of the baseline settings would strengthen this interpretation.

  2. Missing baseline for the learned fusion weights. The comparison with final-layer readout demonstrates the benefit of accessing multiple layers, but does not isolate the benefit of learning head-specific mixing weights. Figure 4 shows that these weights remain close to the uniform value of 1/30, making fixed averaging a natural baseline. Training an otherwise identical model with fixed uniform fusion would clarify how much of the improvement comes from learned weighting rather than multi-layer aggregation, and help establish the added value of this central component.

Questions for authors
  1. How closely does the reported Fast-WAM baseline match your training and evaluation settings? Also, the 51.5% → 60.3% step in Fig. 3 changes both action-expert depth and which video features the expert can access. Is there any evidence separating these two effects?

  2. Have you compared learned Lite KV-Fusion against fixed uniform averaging under the same training setup? Since the weights in Fig. 4 stay close to 1/30, how much of the gain comes from learning the mixing weights rather than from aggregating multiple layers?

  3. Do the real-robot evaluations include any held-out changes in background, lighting, or object placement beyond the variation in the demonstrations? If not, a small controlled comparison with Fast-WAM under such changes would show whether the simulation OOD gains carry over to physical settings (a small suggestion for future work).

2 comments

W1 + Q1. Action depth and comparability with Fast-WAM

We agree that the 51.5% to 60.3% transition in Figure 3 changes both depth and access to video features. It represents a progression of architectural choices.

To better understand the effect of action depth, we trained additional 2-, 4-, and 8-layer action experts while retaining learned KV fusion, world-model-only conditioning, and retracted 1D-RoPE.

Action-expert depthLIBERO (%)LIBERO-Plus (%)
1 layer (original)98.32 ± 0.1478.29 ± 0.03
2 layers98.30 ± 0.2575.02 ± 0.03
4 layers98.25 ± 0.1575.50 ± 0.12
8 layers98.22 ± 0.0672.17 ± 0.27

On LIBERO-Plus, increasing depth does not consistently improve performance. In particular, the 8-layer expert achieves lower mean success than the 2- and 4-layer experts. These results are consistent with a possible OOD generalization benefit from a lightweight action expert.

It is worth mentioning that all variants achieve near-ceiling success on standard LIBERO, leaving limited room for further improvement. This may partly explain the small differences in in-domain performance across depths. On more challenging tasks, a modest increase in action depth might improve in-domain performance.

We use a single-layer action expert as the main configuration to demonstrate that DoT enables WAMs to perform well even with an extremely lightweight action module. This choice is not intended to establish one layer as the optimal depth.

We also measured the 2-, 4-, and 8-layer variants on the same consumer-grade GPU using the original timing protocol. The single-layer Faster-WAM and Fast-WAM rows report the original paper benchmarks.

ModelAction depthAction generation (ms)Phase sum (ms)
Faster-WAM (original)15.84 ± 0.1052.04 ± 0.18
Faster-WAM213.65 ± 0.5362.23 ± 0.82
Faster-WAM422.86 ± 0.9572.24 ± 1.19
Faster-WAM840.33 ± 1.5691.00 ± 1.71
Fast-WAM (original)30150.12 ± 1.65196.03 ± 1.67

Among the Faster-WAM variants, additional action depth increases inference cost without a consistent improvement in OOD performance.

Training and Evaluation Setting

For LIBERO, Faster-WAM follows Fast-WAM's training and evaluation protocols. On LIBERO-Plus, we evaluate Fast-WAM (with the LIBERO checkpoint released by the Fast-WAM authors) and Faster-WAM using the same evaluation settings.

W2 + Q2. Learned fusion versus fixed uniform fusion

We trained an additional variant that replaces the learned per-head layer weights in Lite KV-Fusion with a fixed weight of 1/30 for each video layer. All other architectural choices follow the final Faster-WAM model.

FusionLIBERO (%)LIBERO-Plus (%)
Fixed uniform fusion98.20 ± 0.1573.07 ± 0.15
Learned per-head fusion (Faster-WAM)98.32 ± 0.1478.29 ± 0.03

Fixed uniform fusion yields lower mean success rates on both benchmarks, with a small difference on LIBERO and a larger gap on LIBERO-Plus. Although the learned weights are close to uniform, their numerical proximity alone does not determine their effect on the fused representations. Differences in key norms and amplification through subsequent computation may contribute to this effect.

Learning the fusion weights allows the readout to adapt to the video backbone and dataset. Uniform fusion is a special case of this parameterization; learning therefore retains this option without imposing it in advance.


Q3. Real-world generalization

We additionally conducted a small controlled comparison on the Franka Panda platform using two tasks, place cube and fruit pick, to assess robustness under controlled visual perturbations during real-robot execution. We evaluated Faster-WAM and Fast-WAM with ten trials per model for each task and perturbation condition, reporting the success rate below.

To test the perception robustness (corresponding to sensor noises in LIBERO-Plus), we applied Gaussian blur with a radius of 2 pixels to both camera observations. To test the viewpoint generalization (corresponding to camera perturbation in LIBERO-Plus), we applied 10-pixel shifts to the agent-view and wrist-view images for place cube and fruit pick. These perturbations affect the visual observations supplied to the policies during robot execution.

Place cube: success rate (%)

PerturbationFaster-WAMFast-WAM
Gaussian blur, radius 2 px100.070.0
Image shifts, 10 px70.040.0

Fruit pick: success rate (%)

PerturbationFaster-WAMFast-WAM
Gaussian blur, radius 2 px20.020.0
Image shifts, 10 px100.020.0

For place cube, Faster-WAM achieves higher success rates under both blur and image shifts. For fruit pick, the improvement is observed under image shifts, while both models are significantly impacted by the blur. Thus, the gains depend on the task as well as the strength and type of perturbation, with improvements in three of the four tested conditions. These results provide additional evidence that the robustness benefits extend to real-robot execution under the tested image perturbations.

The updated supplementary materials include figures illustrating these perturbation experiments, together with rollout videos from the original real-robot evaluations.

Reviewer_fmubOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether world action models require deep action experts, and thus proposes Faster-WAM. It connects a single-layer action expert to a pretrained video backbone through learned multi-layer KV fusion, world-model-only conditioning, and RoPE alignment. It achieves competitive performance on LIBERO, RoboTwin, and real-robot tasks, with approximately 3.7–3.8× lower inference latency than Fast-WAM and improved generalization on LIBERO-Plus.

Critical strengths
  1. The paper addresses an important question: how much action-specific computation is needed to turn pretrained video representations into effective robot policies? Reducing this cost matters for responsive robot control, and examining whether deep action experts are necessary is a useful direction.

  2. The speedup over Fast-WAM is substantial under matched inference settings: both predict 32-action chunks with ten denoising steps, but action-generation latency drops from roughly 150 ms to 6 ms. This yields a 3.7–3.8× reduction in total inference latency, with better LIBERO performance and a small decrease on RoboTwin. The real-robot experiments show a similar speedup while maintaining comparable success rates.

  3. The finding that a single-layer action expert can achieve strong control performance is useful. The proposed interface gives it access to all video-backbone layers with little additional computation. Results on LIBERO-Plus and a second video backbone provide further evidence that this design works beyond the main benchmark setting.

Critical weaknesses
  1. The benefit of learned KV fusion is not isolated. Figure 3 shows that multi-layer fusion outperforms final-layer readout, but does not compare learned fusion with fixed uniform fusion. Given that the weights in Figure 4 remain close to 1/30, this baseline would help establish whether learning head-specific weights improves performance.

  2. The conditioning ablation changes several factors at once. Removing action-side text cross-attention and direct proprioceptive conditioning increases LIBERO-Plus success from 68.7% to 77.6%, but also reduces the action expert from approximately 46M to 29M parameters. The experiment does not distinguish the effects of removing each input or separate the proposed conditioning-pathway benefit from the possible regularization effect of reduced capacity.

  3. The real-world evaluation is limited. The experiments cover four tabletop tasks on a single Franka Panda platform, with 20 rollouts per task for most comparisons. They support the practical latency reduction, but provide limited evidence for real-world generalization: the distribution-shift evaluation is conducted in simulation, without a corresponding controlled test of changes in objects, camera viewpoints, or lighting on the physical robot. This leaves the transfer of the reported robustness gains to real deployment uncertain.

Questions for authors
  1. How does learned Lite KV-Fusion compare with fixed uniform fusion under the same training settings on LIBERO and LIBERO-Plus?

  2. What are the individual effects of removing action-side text cross-attention and direct proprioceptive conditioning? Can a comparison with similar parameter counts clarify whether the OOD gain comes from the conditioning pathway or reduced action-expert capacity?

  3. Have you evaluated the real-robot policies under controlled changes in object appearance, camera viewpoint, or lighting? If so, do the robustness gains over Fast-WAM observed on LIBERO-Plus also hold in these settings?

The reviewer also suggests adding multimedia materials, such as real-robot rollout videos, to make this submission stronger.

3 comments

We thank the reviewer for the constructive suggestions. We have added additional experiments to address the concerns.


W1 + Q1. Learned fusion versus fixed uniform fusion

We trained an additional variant that replaces the learned per-head layer weights in Lite KV-Fusion with a fixed weight of 1/30 for each video layer. All other architectural choices follow the final Faster-WAM model.

FusionLIBERO (%)LIBERO-Plus (%)
Fixed uniform fusion, newly trained98.20 ± 0.1573.07 ± 0.15
Learned per-head fusion (Faster-WAM)98.32 ± 0.1478.29 ± 0.03

Fixed uniform fusion yields lower mean success rates on both benchmarks, with a small difference on LIBERO and a larger gap on LIBERO-Plus. Although the learned weights are close to uniform, their numerical proximity alone does not determine their effect on the fused representations. Differences in key norms and amplification through subsequent computation may contribute to this effect.

Learning the fusion weights allows the readout to adapt to the video backbone and dataset. Uniform fusion is a special case of this parameterization; learning therefore retains this option without imposing it in advance.


W2 + Q2. Conditioning pathway versus action-expert capacity

We agree that the original ablation changes both the conditioning pathway and parameter count. We consider the conditioning pathway and reduced action-expert capacity to be two possible contributors to the OOD gain.

We therefore trained three variants that retain the action-side cross-attention (CA) module and its parameter count while changing its direct conditioning inputs:

  • Text only removes the direct proprioception token from action-side Text CA and retains the text tokens.
  • Proprioception only removes the direct text tokens and retains the proprioception token in Text CA.
  • Zero context sets both text and proprioception embeddings to zero before the action-side context projection, preserving the context length and attention mask.

In all three variants, the video backbone continues to receive the original text and proprioception inputs.

Inputs to action Text CALIBERO (%)LIBERO-Plus (%)# Action Params
Text only98.17 ± 0.3269.71 ± 0.1946M
Proprioception only98.13 ± 0.4374.54 ± 0.1546M
Zero context98.30 ± 0.4175.49 ± 0.1846 M
No Text CA (original)98.32 ± 0.1478.29 ± 0.0329M

The three additional variants retain the CA module and achieve similar standard LIBERO performance, but differ substantially on LIBERO-Plus (OOD). The lower OOD performance of text only is consistent with possible over-reliance on direct text conditioning. Proprioception only, with a single proprioception token as direct context, performs better but still falls short of Faster-WAM.

Zero context performs slightly better than proprioception only, but also remains below Faster-WAM. It supplies no direct text or proprioception inputs to action Text CA, yet the trainable projection and CA biases allow this branch to produce a learned constant residual. The retained conditioning branch accounts for approximately 37% of the action-module parameters. The remaining gap is consistent with a possible benefit from making the action module small.

These results suggest that the direct conditioning pathway matters beyond nominal parameter count, while leaving open a contribution from reduced action-expert capacity. They are consistent with our broader design principle: concentrating capacity and computation in the world model can support generalization with a lightweight action expert.

W3 + Q3. Real-world generalization

We additionally conducted a small controlled comparison experiment on the Franka Panda platform using two tasks, place cube and fruit pick, to assess robustness under controlled visual perturbations during real-robot execution. We evaluated Faster-WAM and Fast-WAM with ten trials per model for each task and perturbation condition, reporting the success rate below.

To test the perception robustness (corresponding to sensor noises in LIBERO-Plus), we applied Gaussian blur with a radius of 2 pixels to both camera observations. To test the viewpoint generalization (corresponding to camera perturbation in LIBERO-Plus), we applied 10-pixel shifts to the agent-view and wrist-view images for place cube and fruit pick. These perturbations affect the visual observations supplied to the policies during robot execution.

Place cube: success rate (%)

PerturbationFaster-WAMFast-WAM
Gaussian blur, radius 2 px100.070.0
Image shifts, 10 px70.040.0

Fruit pick: success rate (%)

PerturbationFaster-WAMFast-WAM
Gaussian blur, radius 2 px20.020.0
Image shifts, 10 px100.020.0

For place cube, Faster-WAM achieves higher success rates under both blur and image shifts. For fruit pick, the improvement is observed under image shifts, while both models are significantly impacted by the blur. Thus, the gains depend on the task as well as the strength and type of perturbation, with improvements in three of the four tested conditions. These results provide additional evidence that the robustness benefits extend to real-robot execution under the tested image perturbations.

Illustrative figures for these experiments are provided in the updated supplementary materials.


Additional suggestion. Multimedia materials

Appendix G provides timestamped rollout sequences illustrating policy execution. We additionally provide real-robot rollout videos in the updated supplementary materials.

We will also provide a project webpage for Faster-WAM and include its link in the later version.

Reviewer_fmubOpenReview ↗

Reply to review

The reviewer thanks the efforts of authors during rebuttal, and thus will keep the positive rating while increase the confidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.