acceptodds
Under review as a conference paper at ICLR 2027

PrismWAM: Decoupling State Representations in World Action Models

Abstract

World Action Models (WAMs) typically represent the world through latent embeddings from pretrained video models, which entangle heterogeneous aspects of the world, from how it looks (e.g., texture and color) to how it is physically configured (e.g., occupancy and pose). This raises an underexplored question: what should constitute the state of a WAM? We argue that the world state should separate how the world appears from how the world evolves under action: the former captures semantic context, while the latter is governed by physical state. To this end, we propose PrismWAM, which decouples physics from semantics while jointly modeling state and action. PrismWAM factorizes an embodied task into a pipeline: given only the initial RGB observation, semantic planning learns to predict the final goal image, while physical rollout predicts the fine-grained evolution toward this goal in depth space; executable actions are then generated from the depth rollout. Without any embodied action pretraining or additional data augmentation, PrismWAM achieves state-of-the-art manipulation performance in both simulation and the real world. Notably, PrismWAM even surpasses a broad range of models with large-scale embodied action pretraining (e.g., OpenWAM-). More broadly, we reframe World Action Models from modeling how the world looks to how it responds to action, paving the way for action-centric world models grounded in occupancy, flow, tactile sensing, proprioception, and beyond.

Primary area:
foundation or frontier models, including LLMs
Keywords:
World Action Models, Robotics, Manipulation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_N8UGOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

I use "human" to denote my hand-written opinions in the following review and I use "Opus" to denote Opus 5.5.

Human:

This paper presents a model called PrismWAM to separately model semantic apperance and physical properties. This paper demonstrate that the resulting model has better manipulation pereformance.

Critical strengths

Human:

  1. This idea of disentangeld modeling could be very useful for WAM.

  2. This paper is well written, especially the methodology section. It's very easy to see what authors propose to do.

  3. The experimental tables look sounds, and the empirical gain looks huge from the graph (although w/ some concerns).

Opus appreciates the ablation study and thorough appendix.

Critical weaknesses
  1. About the headline results:

Opus: '''Headline claims rest on an unspecified larger model, and stronger baselines are left out of the main figures. The abstract, the intro's three bullet claims, and Fig. 1(c) all use the larger model (79.00, 95.30, 25.22). This model is called "PrismWAM-15B" in Fig. 3 and Table 1 but "PrismWAM-14B" in Appendix A.1/A.2.1. Its architecture, initialization, and training recipe are never described. A.2.1 gives only the evaluation protocol, which uses 20 episodes per task, versus 100 for the 5B model and the baselines. More seriously, Fig. 3(c) and Table 4 present PrismWAM-15B as the top RoboDojo method, ahead of DM0.5 at 24.9. But Table 28 in the paper's own appendix lists Liber-0 Preview (30.74), Liber-0 Lite (29.24), and GPT-6-Astra (28.97), all above 25.22. These rows never appear in the main-text figures or tables.'''

Human: these conflicts from Opus are true. As a researcher in this field, I don't believe baselines are so bad. For example, in RoboDojo of Figure 1, the baselines are below 10%, but the authors report a score at 25.22%. I don't think this reflects the truth, and stronger baselines are not incorporated. See https://robodojo-benchmark.com/LeaderBoard.

  1. About the claimed physical modeling:

Human: the proposed physical modeling is only depth modeling, which may not be useful. We need to rely on RGB to infer physical properties. Also, depth might not be enough to reflect many useful informations, such as contact. The naming of physical modeling is misleading. Please also refer to conflicts raised by Opus. These points also point to concerns in experiments.

Opus: "Predicting physical rollouts" is confounded with "having a depth sensor." Almost every baseline is RGB-only, while PrismWAM gets ground-truth simulator depth as input. The "w/o depth stream" ablation removes depth as both input and prediction target, so the drop from 54.1 to 2.4 cannot separate the two. The paper's central claim is about what the WAM state should be. The essential missing control is therefore: depth given as an observed input (and optionally to the action tower), but future prediction kept in RGB, or no future prediction at all (FastWAM-style). There is a "Depth-only" column in Tables 24/25 (71.38 / 24.76), but it is never described. A.5.2 refers to "the earlier depth-only system in ??", which is a broken reference. The RoboTwin C2R pattern also points toward the simpler explanation. Every RGB-only model collapses under texture and lighting randomization (starVLA 3.16, FastWAM 1.90, the no-depth ablation 2.40). That is exactly the "appearance robustness" story RQ1 tries to move beyond.

  1. Human: it's not sure why the authors measure the final task performance. The goals of this model are not necessarily related to the success rate. And more focus should be whether the physical properties are really modeled through the depth.

Human: Opus give a 2, I agree. These concerns are hard to address in the rebuttal period, especially the headline results and core contribution framing.

Questions for authors

The questions are raised by Opus.

  1. What is the full training recipe for PrismWAM-14B/15B? Why is it evaluated with 20 episodes per task while the baselines use 100?
  2. What happens with depth given as input but no depth rollout prediction (and RGB future prediction kept, or none)? What exactly is the "Depth-only" column in Tables 24/25?
  3. How does C2R Random performance change if the depth VAE excludes RoboTwin randomized episodes?
  4. Why does PrismWAM do so poorly on LIBERO-Pro Position (6.55%) and LIBERO-Plus camera shifts, given the physical-state motivation?
  5. Why are Liber-0 and GPT-6-Astra omitted from Fig. 3(c) and Table 4?
  6. What are the inference latency and achievable control frequency?
  7. Does using separate (unshared) visual blocks for RGB and depth change the results?
4 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1/2)

We sincerely thank you for the detailed and constructive evaluation, and are encouraged by your recognition of the potential of disentangled modelling for WAMs, the clarity of the methodology, and the thorough ablations and appendix. We address your concerns below.

W1/Q1/Q5. Headline results.

The scaled model. We clarify that PrismWAM-14B and PrismWAM-15B refer to the same scaled variant, initialized from Wan2.1-T2V-14B. Its training recipe is given below and will be added to the appendix.

SettingPrismWAM-14B
Backbone initializationWan2.1-T2V-14B
Depth VAEOurs, same recipe as Appendix A.1.3, with 16 latent channels to match the Wan2.1 VAE
Training stages and updatesSame two stages as the 5B model (depth pretraining / joint training): 15k / 15k on RoboTwin, 20k / 60k on RoboDojo
Global batch size256
Learning rate scheduleSame as the 5B model (Tables 7 and 8)
Training dataSame as the 5B model for each benchmark; no embodied action pretraining

Its RoboTwin results in the submission came from 20 episodes per task. Re-evaluated with 100 episodes per task, matching the baselines, RoboTwin-Full gives 95.26 Clean / 95.66 Random / 95.46 Avg. (previously 94.90 / 95.70 / 95.30), and clean-to-random gives 91.90 / 66.10 / 79.00 (previously 93.00 / 65.00 / 79.00). We will update the main table and the appendix.

Baselines. Each benchmark has many entries, so in the main comparisons we include open-source VLA and WAM policies, the family our method belongs to. We will state this scope explicitly, keep following the official leaderboard, and add newer and stronger baselines in the revision including Liber-0 and GPT-6-Astra. The RoboDojo baseline results are taken from the official leaderboard, where they are verified by the benchmark team.

W2. The claimed physical modelling.

Depth as input versus depth rollout. Following your suggestion, we added the requested control, giving depth as an observed input while keeping the future rollout in RGB (RoboTwin clean-to-random, 50 tasks × 20 episodes per condition). As shown below, with the same depth input an RGB rollout does not transfer to randomized scenes, which indicates that the gain comes from modelling the physical rollout in depth rather than from having a depth sensor, and supports depth as a suitable modality for the physical state. "Depth-only" refers to a legacy ablation variant without the RGB stream and language inputs. We will fix the reference and define this column in the revision.

InputRolloutCleanRandomAvg.
RGB + current depthDepth (paper)87.4054.1070.75
RGB + current depthRGB (new)87.5013.6050.55

On the naming. Our proposal is to separate the physical state from the semantic context; depth is how we instantiate the physical state in this work. Other physical modalities such as occupancy and contact are future extensions to explore of the same principle, as noted in our limitations and future works part. We will refine the wording to make clear that this work is an exploratory step on the geometric side, not a complete physical model.

W3. Why measure final task performance.

We agree that task success alone does not show how well depth is predicted. Following your suggestion, we add an evaluation of the depth rollout against ground-truth future depth on held-out episodes (50 tasks × 10 episodes × 4 timesteps per split, head camera, 8 future frames). Entries are MAE in mm (lower is better); "moving" pixels are those whose depth changes by more than 10 mm. As shown below, PrismWAM achieves lower MAE across both splits and pixel subsets, indicating better preservation of geometric features from a geometric reconstruction perspective.

MethodDepth predictionClean: all MAE (mm) ↓Clean: moving MAE (mm) ↓Random: all MAE (mm) ↓Random: moving MAE (mm) ↓
PrismWAMDirect depth rollout7.835.518.057.1
PrismWAM variant (RGB-D input, RGB rollout)RGB rollout → Depth Anything 3; GT scale aligned37.766.842.1110.1
OpenWAM-α (official C2R)RGB rollout → Depth Anything 3; GT scale aligned41.983.141.8109.5
AuthorsOpenReview ↗

Rebuttal by Authors (2/2)

Q2. Depth given as input but no depth rollout prediction

Please see the table in W2 above. "Depth-only" in Tables 24/25 is the ablation variant without the RGB stream and language inputs.

Q3. Depth VAE without RoboTwin randomized episodes.

Following your suggestion, we retrained the depth VAE without any randomized RoboTwin episodes and retrained the policy on top of it, with all other settings unchanged.

Depth VAE training dataCleanRandomAvg.
AgiBot + RoboTwin clean + randomized (paper)87.4054.1070.75
AgiBot + RoboTwin clean only (new)88.5053.7071.10

Performance on randomized scenes is essentially unchanged, within the evaluation noise of this protocol. The clean-to-random result does not rely on the VAE having seen randomized scenes.

Q4. LIBERO-Pro Position and LIBERO-Plus camera shifts.

LIBERO-Pro Position. This suite includes switching look-alike objects, which requires semantic identification that comes largely from pretraining; even the strongest models stay below 30%. PrismWAM reaches 6.55%, above starVLA, FastWAM, Spatial-Forcing and 4D-WAM, which are all below 4%.

LIBERO-Plus camera shifts. Viewpoint change is hard for most world action models, because the policy depends heavily on the visual stream. FastWAM reaches 16.01% and the pretrained OpenWAM-α 33.21%; PrismWAM reaches 60.60%, nearly twice the latter. There is still a gap to π0.5 (71.23%), and we will discuss both cases further in the revision.

Q6. Inference latency.

Each inference denoises the future depth latents and the action chunk together in 10 Euler steps, with RGB features cached and no RGB or depth frames decoded. With infrastructure optimizations similar to those of OpenWAM, PrismWAM takes 183 ms per inference on a single RTX 5090, on par with OpenWAM-α at 187 ms on the same hardware.

Q7. Separate (unshared) visual blocks.

To assess the effect of parameter sharing, we trained a variant with two separate backbones, one for RGB (semantic planning) and one for depth (physical rollout), keeping the same causal attention order and all other training settings. As shown below, the two are comparable, with differences within the evaluation noise of this protocol, so the results do not depend on sharing the visual blocks. Detailed results will be added to the appendix.

Visual blocksCleanRandomAvg.
Shared (paper)87.4054.1070.75
Unshared (new)89.2051.7070.45

Thanks for the rebuttal. I think it address most of my concerns. I'll increase my score.

Dear Reviewer N8UG,

We sincerely appreciate your time and effort in reviewing our work and providing insightful feedback. We’re glad our response has addressed your concerns. If you have any additional questions or suggestions, please feel free to let us know.

Best regards,

Authors

Reviewer_XAzfOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper asks a foundational question: what should constitute the state of a World Action Model (WAM)? The authors argue that latents from pretrained video models entangle appearance (texture, color) with physical state (geometry, pose, occupancy), while actions really only change the latter. The policy is therefore factorized into three stages: semantic planning, physical rollout, and executable action. The design is instantiated as a Mixture-of-Transformers on a Wan video backbone, with a chain-attention mask controlling information flow, a joint flow-matching objective, and a two-stage recipe. Experiments cover six simulation benchmarks, six ablations, a depth-corruption study, and three real-robot tasks, claiming SOTA-level manipulation without any embodied action pretraining.

Critical strengths

The paper answers with controlled evidence what carries generalization in WAMs. Removing the depth stream collapses success on randomized scenes from 54.1% to 2.4% while clean scenes are barely affected, and the RGB-side ablations supply the complementary half of the story. Among the current wave of "scale more action data" WAM papers, this is a concrete, testable piece of counter-evidence that could genuinely change how people build these models.

The breadth and candor of the evaluation. Six benchmarks, tens of thousands of evaluation episodes, real-robot ID/OOD tests, and appendices that report raw counts, protocol differences, and even facts that work against the authors.

Critical weaknesses

First, the attribution of the gains over baselines is not fully isolated. All simulation evaluations feed the policy ground-truth simulator depth while the main baselines are RGB-only, and the key ablation removes depth input and depth prediction together, so "depth as a better prediction target" (the paper's thesis) and "depth as an extra observation modality" remain entangled. I am aware of the corrupted-depth study in A.6, but that answers a different question — robustness to noisy depth when retrained with that noise — and corrupted ground truth is still geometrically much cleaner than depth estimated from RGB; the real-world evaluation uses a refined-depth pipeline, which hints at feasibility but is not a controlled comparison. I hope the authors can provide a stronger and more rigorous controlled comparison that removes the variable "depth as an extra observation" and substantiates the central claim that depth is a better prediction target — results on a single benchmark would suffice.

Second, the SOTA claims appear overclaimed relative to the paper's own tables. "Consistently outperforms state-of-the-art methods by a large margin across six benchmarks" is hard to square with π0.5 leading on LIBERO-Plus and LIBERO-Pro, or with RoboDojo leaderboard entries sitting above the 15B model. More importantly, the weakest cells in the tables — Position on LIBERO-Pro and Memory on RoboDojo — point at the design's real ceiling (a single final-frame goal, no memory mechanism), yet go undiscussed. Bringing the claims back to what the tables support, and engaging with these failure modes, would make the paper stronger, not weaker.

Questions for authors

Q1:Could you provide a controlled comparison that isolates "depth as a prediction target" from "depth as an extra input" even on a single benchmark?

Q2:Please unify the model naming — the main text calls the larger model PrismWAM-15B, while Appendix A.1, A.2.1 and Table 14 call it PrismWAM-14B — and confirm whether the 15B RoboTwin numbers were evaluated under the same episode protocol as the main-table baselines (Table 14 suggests 20 episodes per task versus 100 for the 5B main table).

Q3:if I understand correctly, each control step requires jointly denoising the future depth latents and the action chunk (10 Euler steps), i.e., a generative rollout sits inside the control loop, while the claim of "fast inference" is currently only qualitative. Could you report wall-clock latency per action chunk (or the achievable control frequency) on specified hardware and compare against representative baselines such as OpenWAM-α, π0.5, starVLA, and FastWAM? A short study of how success varies with the number of denoising steps would further help readers judge the cost–performance trade-off, which matters for real-robot deployment.

3 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We sincerely thank you for the careful and constructive evaluation, and are encouraged by your recognition of the evidence on generalization in WAMs, the breadth of the experiments, and the transparency of the evaluation. We address your concerns below.

W1 / Q1. "Depth as a prediction target" versus "depth as an extra observation".

We clarify that our claim is a little broader than "depth is a better prediction target": we see depth as a suitable modality for describing the physical state transition, which includes both the current state the model conditions on and the future state it predicts. The observed depth and the predicted depth are therefore two parts of one state representation.

Following your suggestion, we also ran the controlled comparison on RoboTwin clean-to-random (50 tasks × 20 episodes per condition), varying the input and the rollout modality separately. As shown below, with the same depth observation an RGB rollout does not transfer to randomized scenes, so the gain is not explained by the extra observation alone, and predicting depth without observing it helps only partly. The full benefit appears when both the observed and the predicted state are in depth, which is consistent with the claim above. We will add this table to Section 4.3 and clarify the claim in the revision.

Clean / Random success (%)Rollout in RGBRollout in depth
RGB input only83.00 / 2.4088.70 / 17.40
RGB + current depth input87.50 / 13.6087.40 / 54.10

W2. SOTA claims and the weakest cells.

We will restate our claims as: without embodied action pretraining, PrismWAM is in the top tier on all six benchmarks, and on several of them it remains competitive with action-pretrained methods such as OpenWAM-α and π0.5.

Following your suggestion to examine the weaker cells, Position on LIBERO-Pro and Memory on RoboDojo, we tried an extended nine-frame window on RoboDojo that includes the initial observation and a longer context: Memory improves from 10.45 to 30.46 and the overall score from 20.73 to 25.10. Although our method improves generalization under several physically relevant variations, these benefits do not extend to all forms of generalization. LIBERO-pro Position perturbations include switching of look-alike objects, placing demands on visual recognition that may benefit from broader semantic pretraining. We will report the Memory improvement and discuss this remaining limitation and its possible causes in the revision.

Q2. Model naming and evaluation protocol.

On the naming, PrismWAM-14B and PrismWAM-15B refer to the same scaled variant built on the Wan2.1-T2V-14B backbone. We will make this clear in the revision.

On the protocol, the main table is meant to report 100 episodes per task throughout, but the RoboTwin results of this variant came from 20 episodes per task. The aligned results are shown below, and we will update the main table and the appendix accordingly.

PrismWAM-14BCleanRandomAvg.
RoboTwin-C2R, 20 episodes per task (paper)93.0065.0079.00
RoboTwin-C2R, 100 episodes per task (aligned)91.9066.1079.00
RoboTwin-Full, 20 episodes per task (paper)94.9095.7095.30
RoboTwin-Full, 100 episodes per task (aligned)95.2695.6695.46

Q3. On "fast inference".

Your understanding of the denoising procedure is correct. The phrase “fast inference” was intended to highlight the computational savings from omitting future RGB generation at inference time. To quantify efficiency, we report the measured inference latency on a single RTX 5090: 183 ms per action chunk for PrismWAM, compared with 187 ms for OpenWAM-α on the same hardware. Our implementation uses infrastructure optimizations similar to those of OpenWAM. We will replace the qualitative “fast inference” wording with these measurements in the revision.

Thank you for your rebuttal. Most of my concerns have been addressed, and I will raise my score. By the way, I assume that the reported 183 ms per action chunk for PrismWAM does not refer to the 14B model, correct? If the 14B model were deployed on a real robot, what would the latency be? In particular, what would the round-trip latency be on the real robot?

Dear Reviewer XAzf,

We sincerely appreciate your time and effort in reviewing our work and providing insightful feedback. We’re glad our response has addressed your concerns, and we answer your follow-up question below.

As you noted, the 183 ms reported earlier refers to PrismWAM-5B, the model used in our real-robot experiments. We further tested PrismWAM-14B alongside another WAM with a 14B backbone, DreamZero-14B, on a single NVIDIA B300, with the denoising steps and visual resolution aligned; as PrismWAM does not use classifier-free guidance (CFG), we also include a CFG-off version of DreamZero. The table below reports the round-trip latency, measured with the same deployment pipeline as in our real-robot experiments, which covers encoding and communication, plus depth refinement for PrismWAM. It shows that PrismWAM-14B’s latency is on par with the model of similar size.

PrismWAM-14BDreamZero-14B (CFG on)DreamZero-14B (CFG off)
Round-trip latency (ms)382596403

If you have any additional questions or suggestions, please feel free to let us know.

Best regards,

Authors

Reviewer_aBSfOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents PrismWAM, which separates semantic understanding from physical state modeling in World Action Models. RGB is used for semantic planning, while depth is used to model physical state changes and guide action generation. The method is evaluated on several simulation and real-world manipulation benchmarks, where it outperforms a number of existing VLA and WAM baselines. Ablations further show the importance of the depth rollout and the proposed information flow.

Critical strengths
  1. This paper presents a clear and intuitive decomposition of semantic planning, physical rollout, and action generation. The use of RGB and depth for different roles is well motivated.
  2. This paper is clearly written and easy to follow.
  3. Experiments on benchmark and real-world settings are comprehensive.
Critical weaknesses
  1. I am slightly concerned about the fairness of the clean-to-random comparison, since the depth VAE has already seen randomized RoboTwin depth during pretraining.
  2. A natural baseline would be a standard joint-generation setting where RGB, depth, and action tokens can interact through full attention, similar to prior approaches that jointly model RGB and depth.
Questions for authors
  1. The abstract describes the method as operating from “only the initial RGB observation,” while the policy formulation and inference procedure also use RGB history, current depth, language, and proprioception. Could the authors clarify the intended meaning of this statement?
3 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We sincerely thank you for the positive and constructive evaluation, and are encouraged by your recognition of the decomposition, the distinct roles of RGB and depth, and the comprehensive experiments. We address your concerns below.

W1. Fairness of the clean-to-random comparison.

Following your suggestion, we retrained the depth VAE without any randomized RoboTwin episodes and retrained the policy on top of it, keeping all other settings unchanged. As shown below (50 tasks × 20 episodes per condition), our method maintains its performance in randomized scenes within the evaluation noise of this protocol, so the clean-to-random result does not rely on the depth VAE having seen randomized scenes.

Depth VAE training dataCleanRandomAvg.
AgiBot + RoboTwin clean + randomized (paper)87.4054.1070.75
AgiBot + RoboTwin clean only (new)88.5053.7071.10

For context, we originally included the randomized episodes to make full use of the simulated data: RoboTwin has 2,500 clean episodes against 25,000 randomized ones. We will include this ablation in the revision.

W2. Joint generation with full attention.

To compare chain attention with full attention, we trained a variant in which RGB, depth and action tokens attend to each other in all six cross-stream directions, in addition to self-attention within each stream (same protocol, 50 tasks × 20 episodes per condition).

AttentionCleanRandomAvg.
Chain attention (paper)87.4054.1070.75
Full attention (new)85.6040.3062.95

Full attention yields lower success than chain attention by 13.80 points on randomized scenes, with a 7.80-point decrease in average success. This suggests that the chain design, where RGB guides the depth rollout and actions are grounded in depth only, suits the decoupled representation better than letting all streams mix freely.

Q1. "Only the initial RGB observation".

Thank you for pointing this out. The phrase was meant to describe semantic planning only: the goal RGB frame is predicted from RGB inputs, without attending to depth. As written it is inaccurate, since language and proprioception are also used through cross-attention. The current depth frame is also an input, but only after the semantic planning phase. We will correct the wording in the revision.

Thanks for your efforts. Most of my concerns have been addressed. I'll raise my score.

Dear Reviewer aBSf,

We sincerely appreciate your time and effort in reviewing our work and providing insightful feedback. We’re glad our response has addressed your concerns. If you have any additional questions or suggestions, please feel free to let us know.

Best regards,

Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response (1/2)

Dear Reviewers, ACs, and SACs,

We sincerely thank all reviewers for their thoughtful and constructive feedback. We appreciate the recognition of our motivation, method, and experiments. Reviewers highlighted:

  • Reframing World Action Models from modeling how the world looks to how it responds to action, through a clear and well-motivated decoupling of physical state from semantic context (aBSf, XAzf, N8UG).
  • Providing controlled evidence on what carries generalization in WAMs (aBSf, XAzf).
  • The strong manipulation performance over existing VLA and WAM baselines in simulation and on the real robot (aBSf, N8UG).
  • The comprehensive evaluation across six benchmarks and real-robot ID/OOD tests (aBSf, XAzf).

We have incorporated the reviewers’ suggestions and added new experiments and analyses to strengthen the submission.


Summary of Key Revisions and Additions

  • Expanded baseline comparisons, aligned evaluations, and full training recipes (Fig. 3; Appendices A.1.4 and A.2.1). (XAzf, N8UG)
  • New controlled study showing optimal transfer with depth as both observed input and predicted output (Sec. 4). (XAzf, N8UG)
  • Further validation of generalization capability using a depth VAE trained without random episodes (Table 25). (aBSf, N8UG)
  • Additional analyses of depth rollout accuracy and full-attention / unshared-block alternatives (Tables 4 and 26). (aBSf, N8UG)
  • Quantified inference latency replacing qualitative speed claims (Appendix A.1.1). (XAzf, N8UG)
  • Clarification of decoupled state representation scope and expanded limitations. (XAzf, N8UG)

The detailed revision summary is provided in Appendix C. We appreciate the reviewers’ feedback and remain committed to improving the work. The following sections summarize our key responses to each reviewer.

AuthorsOpenReview ↗

General Response (2/2)

A Summary of Our Rebuttal

Reviewer aBSf

  • Main concerns: (1) Whether generalization holds when the depth VAE is trained without random episodes; (2) comparison with a full-attention baseline; (3) the phrase “only the initial RGB observation”.
  • Our response: (1) A depth VAE retrained without randomized episodes maintains randomized-scene success (54.10% → 53.70%); (2) a full-attention variant drops to 40.30% on randomized scenes, supporting the chain-attention design; (3) the wording is corrected.

Reviewer XAzf

  • Main concerns: (1) Whether the gain comes from depth as a prediction target or from depth as an extra observation; (2) calibrate performance claims and more discussions on weak cells; (3) unify naming and evaluation protocol for the larger model; (4) report quantitative inference latency.
  • Our response: (1) A controlled comparison showing that, with the same depth observation, an RGB rollout reaches 13.60% on randomized scenes against 54.10% for the depth rollout; (2) restated claims, an added limitations discussion, and an extended memory window that raises RoboDojo Memory from 10.45 to 30.46; (3) unified model name and aligned 100-episode results; (4) measured latency of 183 ms per action chunk on a single RTX 5090.

Reviewer N8UG

  • Main concerns: (1) Provide full training specifications for the larger model and expand the RoboDojo comparison with stronger baselines; (2) clarify the scope of “physical modelling” relative to depth representations; (3) evaluations beyond task success rates to support depth modeling quality; (4) further analysis of the depth VAE, weak benchmark cells, inference latency, and shared visual blocks.
  • Our response: (1) The training recipe and 100-episode results for PrismWAM-14B, with Liber-0 and GPT-6-Astra added to the RoboDojo comparison; (2) controlled experiments with depth as input and an RGB rollout, and wording that presents depth as the geometric part of the physical state; (3) a direct depth-rollout evaluation, where PrismWAM has the lowest error on clean and randomized scenes; (4) an ablation where the depth VAE retrained without randomized episodes, a discussion of both weak cells, measured latency, and an unshared-block variant with comparable performance.

All three reviewers have responded and raised their ratings. We thank the reviewers, ACs, and SACs for their time and consideration.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.