- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Causeway targets causal leakage in action-conditioned embodied world models through the observation-to-video attention path, a model predicting pre-contact frames can see post-contact scene content. It intergrates Unified Causal Attention which partitions attention keys by modality and timestamp and licenses each query by role. Moreover, BioSaM adds geometric anchor/recent memory, a conditioning-noise cap and per-frame diffusion timesteps. Finally, the Causal Leakage Probe injects contradictory action and goal pairs and compares debiased energy in a forbidden window against a licensed one.
Critical strengths
-
UCA is effective. Partitioning keys into scene, goal, action, and video, and licensing each query by role and timestamp, fits inside a standard attention kernel. For example, with weights and seeds held fixed, removing UCA leaves frame metrics almost unchanged but nearly doubles FVD and drops retention from 0.87 to 0.76.
-
The ablation study is very comprehensive. Each BioSaM component is removed individually with the same weights and seeds, and the Cosmos3-Nano comparison uses the same backbone, making it a recipe comparison rather than a cross-model one.
-
The paper is well-presented. The paper is easy to follow, with dedicated figures and tables. The structure is well-organized and the methods make sense.
Critical weaknesses
-
The paper does not measure UCA's effect on leakage. Table 5 reports PSNR, SSIM, LPIPS, FVD, retention, context length, and sampling time, so it shows only fidelity effects. However, the authors do not test the w/o-UCA checkpoint. Since it is the key contribution of this paper, I think it should be included.
-
The input conditioning still carries the answer and the probe cannot tell. CLP contradicts the action class or adds a “keep still” goal prefix, but the scene text and object locations remain, so a model can locate what to manipulate without using the action, and the probe cannot find that. Therefore, it's hard to tell if the proposed methods really work.
-
Leakage is not tested in real-world robots. The paper does not test the probe's calibration on real-robot data or a large-scale simulator, where it makes its fidelity and ablation claims, and no single dataset shows both the leakage statistic and the effect of removing UCA. Whether the probe transfers is therefore untested.
Questions for authors
-
Please run CLP on the w/o-UCA checkpoint from Table 5 on the dataset used there.
-
Since scene tokens are generated from the task description, and the goal text contains object locations, both are conditioning inputs visible under UCA. Can you run the probe with scene captions generated from a contradictory task, or with the goal stripped of object locations? I think this experiment can better demonstrate the effectiveness of your methods.
-
What does the language conditioning contribute? The w/o-language row is 24.09 PSNR and 8.9 FVD against 24.29 and 8.9 with language, and the text notes that most of the improvement over the official checkpoint is already present without it. Is the VLM memory doing work these metrics don't capture, and can you show it on a task where language matters?
-
The paper claims the first black-box test for causal leakage. On what dimension does CLP differ from MiraBench and FACT, both of which are cited once and never compared?