- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a geometric auxiliary training framework for world action models. This paper's method extends ImageWAM with two lightweight prediction heads, one head uses the predicted visual latent to predict 2D target object centroid and 3D target object displacement, while the other uses the predicted action chunk to predict the same things. Both heads' outputs are aligned with the 2D and 3D values actually computed from the real endpoint images, serving as auxiliary losses to help the model train its gemoetic and spatial capabilities.
Critical strengths
-
Combines complementary 2D and 3D information. The per-view centroid/area changes retain image-space information, while the reconstructed 3D displacement provides a shared world-coordinate reference across views. The validity and confidence weighting is a sensible response to missing views and unreliable 3D reconstruction.
-
No additional inference-time requirements. In this method, two projection heads are added only during training, and everything added extra at inference time is removed. Therefore, this method is essentially a model-agnostic training approach that can be conveniently applied to various WAM models as additional auxiliary supervision.
Critical weaknesses
-
The overall improvement is small. The gains are +0.17 percentage points on standard LIBERO, +0.87 points on LIBERO-Plus, and +0.47 points on RoboTwin overall. RoboTwin Randomized performance decreases, and several LIBERO-Plus perturbation axes also decrease.
-
The relationship between the projected visual effect and actual decoded-image geometry is unverified. e_edit is predicted from a latent difference through an auxiliary head. It is not computed by decoding predicted z to an image and measuring the target-object geometry in that image. Therefore, the current objective does not guarantee that the learned proxy corresponds to the geometric change actually depicted by the endpoint image.
-
Due to the above two issues, along with the fact that the parameter counts of the two trainable projection heads do not appear to be explicitly stated in the paper, I have concerns about whether these auxiliary supervision losses change the backbone in a meaningful way.
-
The method relies on extensive task-specific data processing. LIBERO uses SAM3; RoboTwin reads directly from the simulator; and the real-robot setup uses target masks, calibration, joint states, and forward kinematics. These processing pipelines do not appear to be very clearly presented in the paper. The paper also does not report these preprocessing times, cache size, GPU cost, or actual training overhead.
-
The attention visualizations use an early GeoEffect V1 variant rather than the final Gae-WAM checkpoint and cover only a few qualitative examples. The paper does not show that the final action branch systematically attends more to target regions, nor that this attention is correlated with action accuracy or task success
Questions for authors
-
Does the geometric loss actually change the WAM, or is it mostly absorbed by the two heads? Could you test whether the projection heads can still align well if you detach backbone and two project heads or freeze backbone?
-
How close is e_edit to the geometry of the image the model actually predicts? e_edit is read from a latent difference by a learned head; it is not measured on the predicted endpoint. Could you fully denoise predicted z offline, decode it with the VAE, run the same SAM3 + visual-hull pipeline on the decoded image, and compare the resulting e_img with e_edit and e*?
-
Could you give the exact architecture and parameter count of H_edit and H_action? The only detail I found is a hidden size of 256, and Table 19 covers RoboTwin only; I could not find the auxiliary loss weights, noise range, or head settings used for LIBERO. The approximately 2.18M parameters mentioned in Appendix E belong to FullAlign's state predictor, not to Gae-WAM.
-
Is the method intended only for fine-tuning, and can it scale? It seems that Gae-WAM is applied only during downstream policy training, without robot-policy pretraining. The reference relies on per-task prompt maps and hand-written track-selection rules, and each Astribot task is trained as a separate model mentioned in table 8. How much compute and manual effort do SAM3 tracking and visual-hull construction require, for example for the LIBERO frames? How are the Astribot masks obtained? I could not find this described. Would the approach be applicable to large, heterogeneous pretraining data where the target object is not specified?