- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
I use "human" to denote my hand-written opinions in the following review and I use "Opus" to denote Opus 5.5.
Human:
This paper presents a model called PrismWAM to separately model semantic apperance and physical properties. This paper demonstrate that the resulting model has better manipulation pereformance.
Critical strengths
Human:
-
This idea of disentangeld modeling could be very useful for WAM.
-
This paper is well written, especially the methodology section. It's very easy to see what authors propose to do.
-
The experimental tables look sounds, and the empirical gain looks huge from the graph (although w/ some concerns).
Opus appreciates the ablation study and thorough appendix.
Critical weaknesses
- About the headline results:
Opus: '''Headline claims rest on an unspecified larger model, and stronger baselines are left out of the main figures. The abstract, the intro's three bullet claims, and Fig. 1(c) all use the larger model (79.00, 95.30, 25.22). This model is called "PrismWAM-15B" in Fig. 3 and Table 1 but "PrismWAM-14B" in Appendix A.1/A.2.1. Its architecture, initialization, and training recipe are never described. A.2.1 gives only the evaluation protocol, which uses 20 episodes per task, versus 100 for the 5B model and the baselines. More seriously, Fig. 3(c) and Table 4 present PrismWAM-15B as the top RoboDojo method, ahead of DM0.5 at 24.9. But Table 28 in the paper's own appendix lists Liber-0 Preview (30.74), Liber-0 Lite (29.24), and GPT-6-Astra (28.97), all above 25.22. These rows never appear in the main-text figures or tables.'''
Human: these conflicts from Opus are true. As a researcher in this field, I don't believe baselines are so bad. For example, in RoboDojo of Figure 1, the baselines are below 10%, but the authors report a score at 25.22%. I don't think this reflects the truth, and stronger baselines are not incorporated. See https://robodojo-benchmark.com/LeaderBoard.
- About the claimed physical modeling:
Human: the proposed physical modeling is only depth modeling, which may not be useful. We need to rely on RGB to infer physical properties. Also, depth might not be enough to reflect many useful informations, such as contact. The naming of physical modeling is misleading. Please also refer to conflicts raised by Opus. These points also point to concerns in experiments.
Opus: "Predicting physical rollouts" is confounded with "having a depth sensor." Almost every baseline is RGB-only, while PrismWAM gets ground-truth simulator depth as input. The "w/o depth stream" ablation removes depth as both input and prediction target, so the drop from 54.1 to 2.4 cannot separate the two. The paper's central claim is about what the WAM state should be. The essential missing control is therefore: depth given as an observed input (and optionally to the action tower), but future prediction kept in RGB, or no future prediction at all (FastWAM-style). There is a "Depth-only" column in Tables 24/25 (71.38 / 24.76), but it is never described. A.5.2 refers to "the earlier depth-only system in ??", which is a broken reference. The RoboTwin C2R pattern also points toward the simpler explanation. Every RGB-only model collapses under texture and lighting randomization (starVLA 3.16, FastWAM 1.90, the no-depth ablation 2.40). That is exactly the "appearance robustness" story RQ1 tries to move beyond.
- Human: it's not sure why the authors measure the final task performance. The goals of this model are not necessarily related to the success rate. And more focus should be whether the physical properties are really modeled through the depth.
Human: Opus give a 2, I agree. These concerns are hard to address in the rebuttal period, especially the headline results and core contribution framing.
Questions for authors
The questions are raised by Opus.
- What is the full training recipe for PrismWAM-14B/15B? Why is it evaluated with 20 episodes per task while the baselines use 100?
- What happens with depth given as input but no depth rollout prediction (and RGB future prediction kept, or none)? What exactly is the "Depth-only" column in Tables 24/25?
- How does C2R Random performance change if the depth VAE excludes RoboTwin randomized episodes?
- Why does PrismWAM do so poorly on LIBERO-Pro Position (6.55%) and LIBERO-Plus camera shifts, given the physical-state motivation?
- Why are Liber-0 and GPT-6-Astra omitted from Fig. 3(c) and Table 4?
- What are the inference latency and achievable control frequency?
- Does using separate (unshared) visual blocks for RGB and depth change the results?
4 comments
Rebuttal by Authors (2/2)
Q2. Depth given as input but no depth rollout prediction
Please see the table in W2 above. "Depth-only" in Tables 24/25 is the ablation variant without the RGB stream and language inputs.
Q3. Depth VAE without RoboTwin randomized episodes.
Following your suggestion, we retrained the depth VAE without any randomized RoboTwin episodes and retrained the policy on top of it, with all other settings unchanged.
| Depth VAE training data | Clean | Random | Avg. |
|---|---|---|---|
| AgiBot + RoboTwin clean + randomized (paper) | 87.40 | 54.10 | 70.75 |
| AgiBot + RoboTwin clean only (new) | 88.50 | 53.70 | 71.10 |
Performance on randomized scenes is essentially unchanged, within the evaluation noise of this protocol. The clean-to-random result does not rely on the VAE having seen randomized scenes.
Q4. LIBERO-Pro Position and LIBERO-Plus camera shifts.
LIBERO-Pro Position. This suite includes switching look-alike objects, which requires semantic identification that comes largely from pretraining; even the strongest models stay below 30%. PrismWAM reaches 6.55%, above starVLA, FastWAM, Spatial-Forcing and 4D-WAM, which are all below 4%.
LIBERO-Plus camera shifts. Viewpoint change is hard for most world action models, because the policy depends heavily on the visual stream. FastWAM reaches 16.01% and the pretrained OpenWAM-α 33.21%; PrismWAM reaches 60.60%, nearly twice the latter. There is still a gap to π0.5 (71.23%), and we will discuss both cases further in the revision.
Q6. Inference latency.
Each inference denoises the future depth latents and the action chunk together in 10 Euler steps, with RGB features cached and no RGB or depth frames decoded. With infrastructure optimizations similar to those of OpenWAM, PrismWAM takes 183 ms per inference on a single RTX 5090, on par with OpenWAM-α at 187 ms on the same hardware.
Q7. Separate (unshared) visual blocks.
To assess the effect of parameter sharing, we trained a variant with two separate backbones, one for RGB (semantic planning) and one for depth (physical rollout), keeping the same causal attention order and all other training settings. As shown below, the two are comparable, with differences within the evaluation noise of this protocol, so the results do not depend on sharing the visual blocks. Detailed results will be added to the appendix.
| Visual blocks | Clean | Random | Avg. |
|---|---|---|---|
| Shared (paper) | 87.40 | 54.10 | 70.75 |
| Unshared (new) | 89.20 | 51.70 | 70.45 |
Thanks for the rebuttal. I think it address most of my concerns. I'll increase my score.
Dear Reviewer N8UG,
We sincerely appreciate your time and effort in reviewing our work and providing insightful feedback. We’re glad our response has addressed your concerns. If you have any additional questions or suggestions, please feel free to let us know.
Best regards,
Authors
Rebuttal by Authors (1/2)
We sincerely thank you for the detailed and constructive evaluation, and are encouraged by your recognition of the potential of disentangled modelling for WAMs, the clarity of the methodology, and the thorough ablations and appendix. We address your concerns below.
W1/Q1/Q5. Headline results.
The scaled model. We clarify that PrismWAM-14B and PrismWAM-15B refer to the same scaled variant, initialized from Wan2.1-T2V-14B. Its training recipe is given below and will be added to the appendix.
Its RoboTwin results in the submission came from 20 episodes per task. Re-evaluated with 100 episodes per task, matching the baselines, RoboTwin-Full gives 95.26 Clean / 95.66 Random / 95.46 Avg. (previously 94.90 / 95.70 / 95.30), and clean-to-random gives 91.90 / 66.10 / 79.00 (previously 93.00 / 65.00 / 79.00). We will update the main table and the appendix.
Baselines. Each benchmark has many entries, so in the main comparisons we include open-source VLA and WAM policies, the family our method belongs to. We will state this scope explicitly, keep following the official leaderboard, and add newer and stronger baselines in the revision including Liber-0 and GPT-6-Astra. The RoboDojo baseline results are taken from the official leaderboard, where they are verified by the benchmark team.
W2. The claimed physical modelling.
Depth as input versus depth rollout. Following your suggestion, we added the requested control, giving depth as an observed input while keeping the future rollout in RGB (RoboTwin clean-to-random, 50 tasks × 20 episodes per condition). As shown below, with the same depth input an RGB rollout does not transfer to randomized scenes, which indicates that the gain comes from modelling the physical rollout in depth rather than from having a depth sensor, and supports depth as a suitable modality for the physical state. "Depth-only" refers to a legacy ablation variant without the RGB stream and language inputs. We will fix the reference and define this column in the revision.
On the naming. Our proposal is to separate the physical state from the semantic context; depth is how we instantiate the physical state in this work. Other physical modalities such as occupancy and contact are future extensions to explore of the same principle, as noted in our limitations and future works part. We will refine the wording to make clear that this work is an exploratory step on the geometric side, not a complete physical model.
W3. Why measure final task performance.
We agree that task success alone does not show how well depth is predicted. Following your suggestion, we add an evaluation of the depth rollout against ground-truth future depth on held-out episodes (50 tasks × 10 episodes × 4 timesteps per split, head camera, 8 future frames). Entries are MAE in mm (lower is better); "moving" pixels are those whose depth changes by more than 10 mm. As shown below, PrismWAM achieves lower MAE across both splits and pixel subsets, indicating better preservation of geometric features from a geometric reconstruction perspective.