acceptodds
Under review as a conference paper at ICLR 2027

Z-WAM: Learning World-Action Models from Image-Editing Priors

Abstract

Video-based world–action models (WAMs) connect robot action learning with dense future-video prediction, incurring the computational cost of synthesizing intermediate frames. We argue that this expensive visual prediction is unnecessary for learning effective actions: a policy can model the intended scene change and the actions that realize it without reconstructing every intermediate visual state. Motivated by this observation, we present Z-WAM, a vision–language–action framework that transfers pretrained image-editing priors to robot control. Conditioned on current observations and a language instruction, Z-WAM jointly predicts a single future multi-view observation and the corresponding action chunk. This formulation uses image editing to represent the outcome of a manipulation sequence while retaining the full sequence of executable actions. Sequential bidirectional attention couples the editing and action streams, allowing action hypotheses to refine visual features that subsequently guide action prediction. Progressive pretraining first adapts the editing model to robot observation transitions without action labels, then connects visual changes to controls through joint world–action learning. During task adaptation, grasp-aware objectives supervise cumulative arm motion and gripper transitions to encourage coordinated execution. Preliminary experiments on six real-world manipulation tasks support image-editing priors as a foundation for robot action learning without dense future-video prediction.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World–Action Modeling, Image Editing, Event-Aligned Supervision

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jPd4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Z-WAM adapts an image-editing backbone to jointly denoise a single future multi-view observation and a 48-step robot action chunk. Sequential action-to-visual and visual-to-action attention couples the two streams. Training proceeds through robot visual adaptation, joint world-action pretraining, and task-specific fine-tuning. Additional objectives supervise cumulative joint displacement around gripper closure and emphasize gripper state transitions. Preliminary evaluation on six real-robot tasks reports 18 successes in 30 trials and 82.5% average partial completion, compared with lower scores for several available policy baselines.

Critical strengths
  1. The endpoint formulation is practically motivated. Predicting one future visual state while retaining a full executable action sequence is a reasonable way to reduce visual prediction requirements. The manuscript correctly states that the editing stream remains active at inference even when RGB decoding is omitted.

  2. The action supervision is described concretely. The cumulative joint-increment representation, closure-window weighting, detached selector, underreach penalty, and gripper BCE are explicitly defined. The distinction between commanded joint configuration and measured Cartesian outcome is useful.

  3. The preliminary real-robot results are encouraging. The tasks include bimanual transport and handover, and Table 1 reports a substantial observed completion-rate advantage. The paper includes pose/gripper and cumulative-pretraining comparisons rather than only a full-system result.

  4. The manuscript is candid about unfinished evidence. It identifies reused physical trials, missing provenance, pending generalization, and limits of the five-trial comparisons. This transparency is valuable, although the missing evidence still affects the recommendation.

Critical weaknesses
  1. The main evaluation is not yet a finalized experimental record. Each method/task has only five trials, and Appendix E says terminal success, assistance/safety conditions, scoring versions, resets, and method ordering remain to be reconciled with the reported records. Appendix A also leaves final demonstration counts and split manifests unresolved. These are necessary details for interpreting the reported 18/30 success rate and baseline gap, not merely optional documentation.

  2. The central architectural contribution lacks its decisive control. Image-editing-based robot control is already established by ImageWAM. Z-WAM's distinctive sequential bidirectional coupling therefore needs a controlled comparison, but that experiment is explicitly pending. The planned removal of feedback only after bidirectional pretraining would test retention during adaptation, not the full effect of coupling throughout training.

  3. The component results do not isolate the proposed mechanisms. Table 2 lacks a gripper-BCE-only condition and a uniformly weighted BCE control; PredU is not separated from uniform closure-window supervision or ordinary accumulated-command loss. Table 3 adds data and compute together with training stages and omits joint pretraining without robot-visual adaptation. The observed gains cannot yet be attributed specifically to transition weighting, prediction-weighted closure alignment, or progressive staging.

  4. The efficiency motivation is unmeasured. A single future target reduces token requirements in principle, but the editing backbone still participates in ten joint denoising steps. Appendix D leaves measured inference costs and deployment settings unfinished, including the executed prefix. Without observation-to-action latency, memory/compute, and matched endpoint-versus-video comparisons, the practical efficiency advantage remains unestablished.

  5. Generalization and baseline comparability remain unresolved. Held-out object, layout, and background tests are planned rather than reported. Run-specific baseline checkpoint, training-exposure, normalization, and execution records also remain incomplete. Consequently, the present six-task comparison supports preliminary feasibility but cannot separate architectural benefits from differences in initialization, adaptation, or deployment.

Questions for authors
  1. Can you provide finalized per-trial outcomes under one success protocol, including deadlines, interventions, matched resets, randomized ordering, and uncertainty estimates?
  2. What is the effect of bidirectional coupling when pretraining and adaptation are both controlled, using the same initialization, data, and compute?
  3. Do the benefits persist with gripper-only, uniformly weighted BCE, fixed/uniform closure-window, and equal-exposure pretraining controls?
  4. What are the complete inference latency, executed action prefix, and baseline deployment settings? Are any held-out generalization results available in the completed experiment set?
5 comments
AuthorsOpenReview ↗

Response to Reviewer jPd4

We thank the reviewer for their careful review and constructive feedback. To address the key concerns, we expand real-world evaluations to 15 trials per task and method and clarify the evaluation protocol (W1), provide a controlled bidirectional ablation across pretraining and fine-tuning (W2), add ablations on gripper supervision, action supervision, and progressive pretraining (W3), report measured inference latency and deployment settings (W4), and present quantitative generalization results alongside clarified baseline configurations (W5). We detail these findings and address the related questions below.

Response to W1: Trial Records and Success Criteria

Thank you for pointing this out. We have finalized the real-world evaluation protocol and expanded the experiments from 5 to 15 independent trials per task and method. The updated results are reported in Table R2 and have been incorporated into Table 1 of the revised manuscript.

We have also revised the Appendix to clarify the task-specific success and completion criteria, intervention and safety conditions, reset procedures, and evaluation ordering. In addition, we have finalized the demonstration counts and data split manifests to ensure consistency between the experimental records and reported results. These revisions provide a more complete and reproducible evaluation of Z-WAM and its baselines.

Response to W2: Bidirectional Controls Across Training

Thanks for pointing this out. We conducted an additional controlled ablation comparing the full Z-WAM with a unidirectional variant that allows only visual-to-action information flow. Specifically, the unidirectional variant removes action-to-visual feedback throughout both joint world–action pretraining and task-specific fine-tuning, rather than only during fine-tuning. Both variants use the same image-editing backbone, training data, objectives, and training schedule, ensuring a controlled comparison.

We evaluated both variants on two representative real-world manipulation tasks, Place Three Objects and Hand Over the Object. As shown in Table R1, bidirectional interaction improves success rates by 40.0 and 66.7 percentage points on the two tasks, respectively, while also increasing completion rates from 87.8% to 97.8% and from 40.0% to 90.0%. These results demonstrate the effectiveness of bidirectional visual–action interaction in improving manipulation performance.

Table R1: Ablation study on bidirectional interaction. Each entry reports success rate (SR, %) followed by mean completion rate (CR, %), separated by a slash. Each variant is evaluated over 15 independent trials per task. The full-model trials are shared with the corresponding rows in the other tables and are not additional independent trials.

InteractionPlace Three ObjectsHand Over the Object
Unidirectional (visual → action)46.7 / 87.820.0 / 40.0
Bidirectional (Z-WAM)86.7 / 97.886.7 / 90.0
AuthorsOpenReview ↗

Response to Reviewer jPd4

Response to W3: Loss Functions and Pretraining Attribution

Thank you for pointing this out. We conducted additional controlled ablations on gripper supervision, action supervision, and progressive pretraining. All variants were evaluated on Place Three Objects and Hand Over the Object, with 15 independent trials per task under the same protocol.

1. Gripper Supervision Ablation. We compare BCE-only, uniform BCE, and transition-weighted BCE configurations, with and without PredU, to isolate the contribution of gripper supervision and transition weighting. As shown in Table R6, with PredU fixed, transition-weighted BCE achieves 86.7% SR and 93.9% CR, outperforming uniform BCE (56.7% SR / 73.1% CR). Notably, uniform BCE does not consistently outperform the no-BCE variant, highlighting the importance of transition-aware weighting.

Table R6: Ablation study on gripper supervision. Each entry reports success rate (SR, %) followed by mean completion rate (CR, %).

PredUGripper SupervisionPlace Three ObjectsHand Over the ObjectAverage
×None40.0 / 63.320.0 / 41.730.0 / 52.5
×BCE-only46.7 / 91.120.0 / 33.333.3 / 62.2
✓None53.3 / 87.860.0 / 70.056.7 / 78.9
✓Uniform BCE73.3 / 91.140.0 / 55.056.7 / 73.1
✓Transition-weighted BCE (Z-WAM)86.7 / 97.886.7 / 90.086.7 / 93.9

2. Action Supervision Ablation. Without auxiliary gripper BCE, we compare ordinary accumulated-command loss, fixed reference closure, uniform and prediction-weighted closure windows, and the complete PredU objective. As shown in Table R7, PredU achieves the highest average SR (56.7%) and CR (78.9%). Although ordinary accumulated-command supervision performs better on Place Three Objects, PredU achieves substantially stronger performance on Hand Over the Object, suggesting improved robustness for grasp-sensitive manipulation. The weaker results without directional underreach further indicate the importance of combining the two components.

Table R7: Ablation study on action supervision without auxiliary gripper BCE. Each entry reports SR / CR (%).

Action SupervisionPlace Three ObjectsHand Over the ObjectAverage
None (base loss only; NoPose)40.0 / 63.320.0 / 41.730.0 / 52.5
Ordinary accumulated-command loss80.0 / 96.726.7 / 45.053.3 / 70.8
Fixed reference closure26.7 / 67.70.0 / 0.013.3 / 33.9
Uniform closure window40.0 / 86.76.7 / 25.023.3 / 55.8
Prediction-weighted closure window0.0 / 41.126.7 / 45.013.3 / 43.0
PredU (full, without auxiliary BCE)53.3 / 87.860.0 / 70.056.7 / 78.9

3. Progressive Pretraining Ablation. We compare joint world–action pretraining with and without preceding robot-domain visual adaptation. To control for additional training exposure, we include an extended-pretraining variant that reallocates the visual adaptation budget to joint pretraining, matching the total training budget. As shown in Table R8, the full progressive pipeline achieves 86.7% SR and 93.9% CR, substantially outperforming all alternatives. In particular, extended joint pretraining fails to recover the performance of the full pipeline, supporting the benefit of progressive staging beyond additional computation.

Table R8: Ablation study on progressive pretraining. Each entry reports SR / CR (%). All variants retain general editing initialization and task-specific fine-tuning; extended pretraining matches the total training budget of the full pipeline.

Robot-domain Visual AdaptationJoint World–Action PretrainingPlace Three ObjectsHand Over the ObjectAverage
××0.0 / 21.10.0 / 8.30.0 / 14.7
✓×0.0 / 11.10.0 / 5.00.0 / 8.1
×✓53.3 / 87.80.0 / 25.026.7 / 56.4
×✓ (extended pretraining)0.0 / 73.313.3 / 35.06.7 / 54.2
✓✓ (Z-WAM)86.7 / 97.886.7 / 90.086.7 / 93.9

Overall, these controlled experiments provide further evidence for the effectiveness of transition-weighted gripper supervision, the complete PredU objective, and progressive pretraining. The expanded results will be incorporated into the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer jPd4

Response to W4: Efficiency and Execution Settings

To evaluate the practical inference efficiency of Z-WAM, we conducted additional latency benchmarks against representative video-based WAMs, including LingBot-VA, Motus, and Fast-WAM-Joint.

The measurements cover model-side observation-to-action inference, including visual preprocessing and encoding, all denoising required for action prediction, and action postprocessing. For Z-WAM, this includes all ten joint denoising steps. Camera capture, communication, and physical execution are excluded. For each configuration, we collected 150 timed measurements across three independent model loads after warm-up, reporting mean, P50, and P95 latency.

As shown in Table R4, Z-WAM achieves a mean inference latency of 672.05 ms, lower than LingBot-VA (4,535.49 ms), Motus (1,464.98 ms), and Fast-WAM-Joint (809.81 ms). Z-WAM also achieves the lowest P50 and P95 latency among these configurations.

We have clarified the deployment settings in Appendix B.2: at each inference call, Z-WAM predicts a 48-step action chunk (H = 48) and executes the first 24 steps (K = 24) before replanning based on the latest observation. The measurement protocol and complete latency results are reported in Appendix F. These results demonstrate a model-side latency advantage under the evaluated configurations.

Table R4: Model-side inference latency of representative world–action models.

MethodInference ModeDenoising StepsMean (ms) ↓P50 (ms) ↓P95 (ms) ↓
LingBot-VAVideo + Action (successive stages)10 + 104,535.494,527.024,583.00
MotusJoint Video–Action101,464.981,483.591,539.54
Fast-WAM-JointJoint Video–Action10809.81796.95856.09
Z-WAM (Ours)Joint Image–Action10672.05672.67680.01

Response to W5: Generalization and Baseline Comparability

Thank you for pointing this out. We have conducted additional generalization experiments and clarified the baseline configurations to address these concerns.

Generalization. Building on the preliminary demonstrations in our supplementary video, we conducted quantitative experiments under three progressively accumulated variations involving object types, basket shapes, spatial arrangements, and backgrounds. Z-WAM and ImageWAM were evaluated under the same conditions without additional fine-tuning, with 15 independent trials per setting. As shown in Table R3, Z-WAM consistently achieves higher completion rates across all three settings, reaching 53.3% SR under G1 and G2. Although performance decreases under the more challenging G3 setting, Z-WAM still outperforms ImageWAM in completion rate (58.3% vs. 16.7%). These results provide further evidence of generalization under varying task conditions.

Baseline Comparability. We have clarified the baseline configurations, including model initialization, training data and exposure, normalization, deployed checkpoints, and execution settings, in Section B.1 and B.2 of the revised Appendix. Furthermore, the controlled ablations in Tables R1 and R6–R8 use matched initialization, training data, and training budgets to isolate the contributions of bidirectional interaction, action supervision, and progressive pretraining. Together, these results provide additional evidence that Z-WAM's performance improvements are supported by its architectural and training designs, rather than solely by differences in experimental configurations.

Reviewer_9GiqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper propose to use image editting prior from the pre-trained Z-Image for down-stream robot manipulation. The paper presents a model architecture to adapt Z-Image for action generation, an action objective that separates reaching and grasping, and a progressive training strategy. The effectiveness of the method is supported with real-world evaluations.

Critical strengths
  1. The separate supervision of approaching and grasping for action generation is interesting and effective.
  2. The idea of using image generation model for action generation is somewhat interesting.
Critical weaknesses
  1. There may risk over-design of the overall architecture. A simpler alternative would be to project the action tokens into the same hidden space and concatenate them with the existing token sequence, thereby enabling bidirectional vision–action interaction without introducing reciprocal cross-attention modules. The paper neither compares against such a baseline, leaving it unclear whether the additional parameters and architectural modifications are necessary.
  2. Regarding the motivation: predicting image is less expensive than video. However, recent video VAE (i.e., WAN) adopt a 1+4n1 + 4n compression rate, saying that every four future frames are compressed into one latent frame. The prediction effort of 4 continuous future frames is somewhat equal to predicting one image in image editing model.
Questions for authors

A general weakness of the so-called world-action-model is that it does not predict the world state, but rather only the observation state. The workspace is often occluded by the robot arms and grippers. Therefore, the generation of future observations will always require the model to generate something from nothing, i.e., generating concepts that are absent from both observation and instruction. Will it lead to more severe overfitting to the robot data?

6 comments
AuthorsOpenReview ↗

Response to Reviewer 9Giq

We thank the reviewer for their careful review and constructive feedback. To address the key concerns, we provide a controlled token-concatenation ablation (W1), report measured inference latency and clarify the implications of video VAE compression (W2), and explain the role of future-observation prediction and pretrained visual priors in world modeling and handling occlusion (Q1). We detail these findings and clarifications below.

Response to W1: Token Concatenation Baseline

Thank you for this valuable suggestion. Compared with direct token concatenation, Z-WAM employs bidirectional cross-attention to enable more structured information exchange between visual and action streams. This design allows action features to refine visual representations, which subsequently guide action prediction, facilitating better visual–action alignment and coordinated action learning.

To validate this design, we conducted an additional ablation study comparing Z-WAM with a token-concatenation baseline using shared self-attention. Both variants used the same editing backbone, training data, objectives, and training schedule throughout joint pretraining and downstream fine-tuning. As shown in Table R5, Z-WAM substantially outperforms the token-concatenation baseline, improving success rates from 20.0% to 86.7% on Place Three Objects and from 0.0% to 86.7% on Hand Over the Object. The corresponding completion rates also improve from 58.3% to 97.8% and from 25.0% to 90.0%, respectively. These results demonstrate the effectiveness of the proposed bidirectional cross-attention design in facilitating visual–action interaction and improving manipulation performance.

Table R5: Ablation study on visual–action interaction architectures. Each entry reports success rate (SR, %) followed by mean completion rate (CR, %), separated by a slash. Each variant is evaluated over 15 trials per task.

InteractionPlace Three ObjectsHand Over the Object
Token concatenation (shared self-attention)20.0 / 58.30.0 / 25.0
Bidirectional cross-attention (Z-WAM)86.7 / 97.886.7 / 90.0

Response to W2: Video VAE Compression and Efficiency

Thank you for this important observation. Modern video VAEs, such as Wan-VAE, reduce temporal redundancy through frame compression. However, inference cost also depends on latent token dimensions, backbone architecture, and denoising steps. Z-WAM focuses on predicting a single task-relevant future endpoint jointly with actions, avoiding explicit generation of intermediate visual states.

To evaluate practical efficiency, we conducted additional observation-to-action latency benchmarks against representative video-based WAMs. The measurements include visual encoding, joint denoising, and action postprocessing. As shown in Table R4, Z-WAM achieves a mean latency of 672.05 ms, lower than LingBot-VA (4,535.49 ms), Motus (1,464.98 ms), and Fast-WAM-Joint (809.81 ms).

These results support the practical inference efficiency of Z-WAM under the evaluated configurations. We have also clarified in the revised manuscript that endpoint prediction simplifies the visual prediction objective without necessarily guaranteeing lower computational cost across different architectures.

Table R4: Inference latency of representative joint world–action models.

MethodInference ModeDenoising StepsMean (ms) ↓P50 (ms) ↓P95 (ms) ↓
LingBot-VAVideo + Action10 + 104,535.494,527.024,583.00
MotusJoint Video–Action101,464.981,483.591,539.54
Fast-WAM-JointJoint Video–Action10809.81796.95856.09
Z-WAM (Ours)Joint Image–Action10672.05672.67680.01
AuthorsOpenReview ↗

Response to Reviewer 9Giq

Response to Q1: Observations, World State, and Occlusion

We would like to clarify that future-observation prediction serves as more than an image-reconstruction objective: it provides a learning signal connecting robot actions with their observable consequences. By jointly predicting future observations and actions, Z-WAM is encouraged to capture object interactions, spatial relationships, and the correspondence between actions and scene changes. This is the motivation for incorporating future-observation prediction into policy learning.

Regarding occlusion, Z-WAM is initialized from a model pretrained on large-scale image-editing data, so its visual and semantic priors are not learned solely from the limited robot demonstrations. Predictions of occluded regions are informed by these pretrained priors and the available multi-view context, rather than being generated without prior knowledge.

Furthermore, our generalization evaluation goes beyond the original training-task configuration: we introduce changes to the objects and basket, spatial arrangements, and background, and evaluate the policies without additional fine-tuning. Z-WAM achieves higher completion rates than ImageWAM across all three settings. These results provide evidence that the learned policy can transfer beyond the original task configuration, supporting the practical value of the pretrained priors and joint visual–action modeling.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer 9Giq,

Thank you again for your valuable time and constructive feedback.

As the rebuttal period is nearing its end, we would like to kindly check whether our previous response has addressed your main concerns. If there are any aspects that you feel need further clarification or improvement, please let us know, and we would be more than happy to provide additional explanations or revisions.

We sincerely appreciate your thoughtful review and consideration.

Best regards, Submission 58724 Authors

Reviewer_gYACOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Z-WAM, an image-editing-based world–action model that jointly predicts a future endpoint observation and an action chunk. Its main components include sequential bidirectional interaction between visual and action streams, progressive robot-domain pretraining, and grasp-aware auxiliary supervision. Evaluation covers six real-world manipulation tasks.

Critical strengths

Using endpoint image prediction instead of dense video prediction is a reasonable direction for robot world–action modeling. The paper includes real-robot comparisons and preliminary analyses of pretraining and grasp-aware supervision.

Critical weaknesses

The central bidirectional interaction mechanism lacks a completed ablation against one-way conditioning. Its contribution beyond the image-editing backbone and training pipeline therefore remains unclear. Evaluation uses only five trials per task and configuration. Generalization experiments remain pending, limiting the strength and scope of the conclusions. The efficiency motivation is not supported by measured end-to-end inference costs or a controlled endpoint-versus-video comparison.

Questions for authors

Can the authors provide a controlled bidirectional-versus-unidirectional comparison and measured inference latency? Can the authors finalize the evaluation protocol and provide additional independent trials and generalization results?

4 comments
AuthorsOpenReview ↗

Response to Reviewer gYAC

We thank the reviewer for their careful review and constructive feedback. To address the key concerns, we provide a controlled bidirectional ablation (W1), expand real-world evaluations to 15 trials per task and method (W2), add quantitative generalization results (W3), and present measured inference latency and speed–performance trade-offs (W4). We detail these new findings below.

Response to W1: Contribution of Bidirectional Interaction

To isolate the contribution of bidirectional interaction, we conducted an additional ablation comparing the full Z-WAM with a unidirectional variant that allows only visual-to-action information flow. Specifically, we removed action-to-visual feedback throughout both joint world–action pretraining and task-specific fine-tuning, while keeping the image-editing backbone, training data, objectives, and training schedule unchanged. This controlled setting isolates the effect of bidirectional interaction from the shared backbone and training pipeline.

We evaluated both variants on two representative real-world manipulation tasks, Place Three Objects and Hand Over the Object, with 15 independent trials per task and variant. As shown in Table R1, bidirectional Z-WAM achieves an average success rate (SR) of 86.7% and completion rate (CR) of 93.9%, compared with 33.3% SR and 63.9% CR for the unidirectional variant. These results demonstrate the effectiveness of bidirectional interaction in improving manipulation performance beyond the shared image-editing backbone and training pipeline.

Table R1: Ablation study on bidirectional interaction. Each entry reports success rate (SR, %) followed by mean completion rate (CR, %), with 15 trials per task and variant.

InteractionPlace Three ObjectsHand Over the ObjectAverage
Unidirectional (visual → action)46.7 / 87.820.0 / 40.033.3 / 63.9
Bidirectional (Z-WAM)86.7 / 97.886.7 / 90.086.7 / 93.9

Response to W2: Expanded Real-World Evaluation

Thank you for pointing this out. We have expanded the real-world evaluation from 5 to 15 independent trials per task and method, following a consistent evaluation protocol. This results in 90 trials per method across six manipulation tasks.

As shown in Table R2, Z-WAM achieves the highest overall success rate (SR) of 50.0% and mean completion rate (CR) of 78.2%, compared with 25.6% SR and 56.7% CR for the strongest baseline in aggregate performance, π₀.₅. Z-WAM also achieves the highest mean completion rate across all six tasks. These expanded results provide additional evidence of Z-WAM's real-world manipulation performance beyond the original five-trial evaluation. We have accordingly updated Table 1 in the revised manuscript to reflect the expanded evaluation with consistent evaluation protocol.

Table R2: Expanded real-world evaluation. Each entry reports success rate (SR, %) followed by mean completion rate (CR, %), separated by a slash. Each method is evaluated over 15 independent trials per task.

MethodArrange fruitsReturn the bookPlace three objectsArrange slippersLoad the trayHand over the objectAverage
ImageWAM0.0/16.033.3/46.760.0/93.320.0/31.70.0/38.30.0/0.018.9/37.7
Fast-WAM0.0/8.713.3/25.053.3/80.00.0/31.70.0/42.56.7/6.712.2/32.4
Motus0.0/30.00.0/11.70.0/48.90.0/23.36.7/62.56.7/23.32.2/33.3
LingBot-VA33.3/67.36.7/16.713.3/73.30.0/28.36.7/55.020.0/35.013.3/45.9
π₀.₅6.7/29.36.7/43.346.7/83.326.7/55.013.3/64.253.3/65.025.6/56.7
Z-WAM33.3/75.333.3/58.386.7/97.840.0/75.020.0/72.586.7/90.050.0/78.2
AuthorsOpenReview ↗

Response to Reviewer gYAC

Response to W3: Generalization Evidence

Beyond the preliminary generalization demonstrations in our supplementary video, we conducted additional quantitative experiments on the fruit-arrangement task under three progressively accumulated variations: (G1) changing object types and basket shape, (G2) additionally modifying object positions, and (G3) further changing the scene background. Both Z-WAM and ImageWAM were evaluated over 15 independent trials per setting without additional fine-tuning.

As shown in Table R3, Z-WAM consistently outperforms ImageWAM in completion rate across all three settings, achieving 53.3% success rates under G1 and G2. Although performance decreases under the more challenging G3 setting, Z-WAM still achieves higher completion (58.3% vs. 16.7%). These results demonstrate Z-WAM's generalization ability under varying task conditions. Additional experimental details and visualizations are provided in Section 4.4 and Appendix E of the revised manuscript.

Table R3: Generalization under progressively accumulated distribution shifts. Each entry reports success rate (SR, %) followed by mean completion rate (CR, %), over 15 trials per setting.

Evaluation SettingZ-WAMImageWAM
G1: Object and basket changes53.3 / 85.00.0 / 30.0
G2: G1 + spatial rearrangement53.3 / 88.30.0 / 18.3
G3: G2 + background change6.7 / 58.30.0 / 16.7

Response to Q1: Directional Controls and Latency

Thank you for this suggestion. We conducted additional experiments to evaluate bidirectional coupling and inference efficiency.

For bidirectional coupling, we compared Z-WAM with a unidirectional variant under matched initialization, training data, and computational budgets. As shown in Table R1, Z-WAM achieves an average SR of 86.7% and CR of 93.9%, outperforming the unidirectional variant (33.3% SR / 63.9% CR), supporting the effectiveness of bidirectional interaction.

For inference efficiency, Table R4 reports measured observation-to-action latency against representative video-based WAMs. Z-WAM achieves a mean latency of 672.05 ms, lower than LingBot-VA (4,535.49 ms), Motus (1,464.98 ms), and Fast-WAM-Joint (809.81 ms), demonstrating its practical inference efficiency.

Response to Q2: Evaluation Protocol and Generalization

Thank you for this valuable suggestion. We have established a unified evaluation protocol with consistent task completion criteria, success and completion scoring rules, and trial execution procedures, as detailed in Section A of the Appendix. Under this protocol, we expanded our real-world evaluation from 5 to 15 independent trials per task and method. The updated results are reported in Table R2 and have been incorporated into Table 1 of the revised manuscript.

To further assess generalization, we conducted additional experiments under three progressively accumulated variations involving object types, basket shapes, spatial arrangements, and backgrounds. Each setting includes 15 independent trials per method, with the results presented in Table R3.

AuthorsOpenReview ↗

Response to Reviewer gYAC

Response to W4: Measured Efficiency

Z-WAM achieves a mean model-side inference latency of 672.05 ms with ten denoising steps, lower than LingBot-VA, Motus, and Fast-WAM-Joint, which also predict future visual states and actions. Although the ten-step action-only configurations of ImageWAM and Fast-WAM are faster, reducing Z-WAM to one denoising step lowers its latency to 103.95 ms, below ImageWAM (171.31 ms) and Fast-WAM (383.29 ms), while retaining higher average success and completion rates across Place Three Objects and Hand Over the Object.

Inference latency. We measure model-side observation-to-action latency for generating a 48-step action chunk, including visual preprocessing and encoding, all denoising required for action prediction, and action postprocessing. RGB decoding used solely for visualization, model initialization, camera capture, communication, and physical execution are excluded. Each configuration is evaluated over 150 timed requests following warm-up across three independent model loads. We report the mean, P50, and P95 latency. Each method retains its native input representation, precision, and inference implementation.

Table R4: Model-side inference latency. All configurations produce a 48-step action chunk. Latency is reported in milliseconds; lower is faster.

MethodInference ModeDenoising StepsMean ↓P50 ↓P95 ↓
LingBot-VAVideo + Action (successive stages)10 + 104,535.494,527.024,583.00
MotusJoint Video–Action101,464.981,483.591,539.54
ImageWAMAction-only10171.31166.67187.13
Fast-WAMAction-only10383.29381.25403.00
Fast-WAM-JointJoint Video–Action10809.81796.95856.09
Z-WAM (Ours)Joint Image–Action10672.05672.67680.01

As shown in Table R4, ten-step Z-WAM achieves the lowest mean, P50, and P95 latency among the configurations that predict both future visual states and actions, while the action-only configurations of ImageWAM and Fast-WAM are faster. This comparison reflects system-level efficiency under each method’s inference configuration.

Speed–performance trade-off. We further evaluate Z-WAM with one, five, and ten joint denoising steps, keeping the checkpoint, input resolution, precision, and implementation optimizations fixed and varying only the denoising budget. The latency benchmarks and real-robot evaluations use the same Z-WAM checkpoint, with 15 trials per task and configuration.

Table R4b: Inference latency and task performance across denoising budgets. Task results are reported as SR / CR (%), with the average weighting the two tasks equally. Task results for ImageWAM, Fast-WAM, and ten-step Z-WAM reuse the corresponding main-evaluation trials.

MethodStepsMean (ms) ↓P50 (ms) ↓P95 (ms) ↓Place Three Objects · SR / CR ↑Hand Over the Object · SR / CR ↑Average · SR / CR ↑
ImageWAM10171.31166.67187.1360.0 / 93.30.0 / 0.030.0 / 46.6
Fast-WAM10383.29381.25403.0053.3 / 80.06.7 / 6.730.0 / 43.3
Z-WAM (Ours)1103.95103.69106.3173.3 / 94.426.7 / 45.050.0 / 69.7
Z-WAM (Ours)5357.06356.98363.1846.7 / 87.746.7 / 60.046.7 / 73.8
Z-WAM (Ours)10672.05672.67680.0186.7 / 97.886.7 / 90.086.7 / 93.9

As shown in Table R4b, one-step Z-WAM achieves an average SR of 50.0% and CR of 69.7%, exceeding ImageWAM’s 30.0% / 46.6% and Fast-WAM’s 30.0% / 43.3%. These results demonstrate a speed–performance trade-off: ten-step Z-WAM achieves the highest task performance among the listed configurations, while one-step Z-WAM provides the lowest latency and higher average performance on these two tasks than the two action-only references. Full measurement details and results are provided in Appendix F.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer gYAC,

Thank you again for your valuable time and constructive feedback.

As the rebuttal period is nearing its end, we would like to kindly check whether our previous response has addressed your main concerns. If there are any aspects that you feel need further clarification or improvement, please let us know, and we would be more than happy to provide additional explanations or revisions.

We sincerely appreciate your thoughtful review and consideration.

Best regards, Submission 58724 Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary Response to ACs

Dear Area Chair and Reviewers,

We sincerely thank the reviewers for their constructive feedback. Below, we summarize our work, the reviewers’ recognition, and our responses to their concerns.

Summary of Our Work

We propose Z-WAM, which transfers image-editing priors to robot control by jointly predicting one future multi-view observation and a full action chunk, without reconstructing intermediate video frames. Our framework combines:

  • Sequential bidirectional visual–action interaction.
  • Progressive robot-domain visual adaptation and joint world–action pretraining.
  • Grasp-aware supervision: PredU constrains cumulative joint motion near gripper closure, while transition-weighted gripper supervision encourages coordinated execution.

Reviewers’ Positive Feedback

The original reviews recognized the endpoint formulation (gYAC, jPd4), the interest in image-generation priors and grasp-aware supervision (9Giq), and the concrete objectives, encouraging real-robot results, and transparent reporting (jPd4). During discussion, Reviewer 9Giq confirmed: “My concerns about the architecture design are addressed.”

Responses to Reviewers’ Concerns

  • Evaluation (gYAC, jPd4): We expanded evaluation to 15 trials per task and method across six tasks. Z-WAM achieves 50.0% success rate (SR) / 78.2% completion rate (CR), versus 25.6% / 56.7% for π₀.₅ (Table R2). The fixed round-robin protocol and per-trial records are documented in Appendices A and H.
  • Architecture (gYAC, 9Giq, jPd4): We added unidirectional and token-concatenation controls throughout joint pretraining and fine-tuning. Across placement and handover, Z-WAM achieves 86.7% average SR, versus 33.3% and 10.0%, respectively (Tables R1 and R5).
  • Supervision and pretraining (jPd4): We added BCE-weighting, PredU-component, visual-adaptation, and extended-joint-pretraining controls. These comparisons support the complete supervision and training pipeline under the evaluated settings (Tables R6–R8).
  • Generalization (gYAC, jPd4): Without further fine-tuning, Z-WAM completes 8/15 trials under both object/basket changes and additional spatial rearrangement, versus 0/15 for ImageWAM. With background changes added, Z-WAM achieves 58.3% CR but 1/15 full-task success (Table R3).
  • Efficiency (gYAC, 9Giq, jPd4): Ten-step and one-step Z-WAM take 672.05 ms and 103.95 ms, respectively (Tables R4 and R4b). Our additional Wan2.2-TI2V-5B comparison matches hardware, action horizon, denoising budget, and timing protocol: ten-step Z-WAM has 31.7% lower latency (672.05 vs. 983.57 ms). These are configuration-specific measurements.
  • Occlusion (9Giq): We clarified supervision using recorded future observations, which retain persistent occlusions, and closed-loop execution using three camera views with replanning after 1.6 seconds of action execution. In future work, we would like to explore tactile feedback to improve control under persistent occlusion.

The expanded experiments are documented in the revised manuscript; the latest Wan comparison and occlusion clarification appear in our November 19 follow-up responses.

Unfortunately, we have not yet received follow-up responses from most reviewers. We hope this summary helps the Area Chair assess the updated evidence and our responses. Thank you again for your consideration; we remain happy to clarify any questions.

Best regards,
The Authors of Submission 58724

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.