- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes the Geometric Action Model (GAM), which adapts a pretrained geometric foundation model for language-conditioned robot manipulation. The method inserts a causal future predictor between the shallow and deep layers of the geometric backbone. It predicts future geometric features and action representations, which are then processed by the remaining pretrained layers to produce actions and future depth predictions. Training combines action regression with future-feature and future-depth supervision. Experiments on LIBERO, LIBERO-Plus, RoboCasa-Kitchen, and real-robot tasks show competitive manipulation performance, particularly under camera perturbations, with relatively low inference latency.
Critical strengths
Using a geometric foundation model as the backbone for robot policy learning is an interesting direction. Rather than treating pretrained geometric features as an additional input, GAM integrates future prediction and action generation into the geometric backbone itself. This design could be useful for manipulation tasks where spatial relationships and camera changes matter. Table 2 provides evidence for the importance of geometric initialization and future prediction. On LIBERO-Plus, the DA3-pretrained model achieves 85.5% success with future supervision, compared with 79.3% using DINO initialization. Removing future supervision reduces the DA3-based result to 72.8%. The future-depth evaluation in Appendix B also provides some evidence that the model learns to predict geometric changes.
The robustness and efficiency results are another strength. GAM achieves 83.1% success under camera perturbations on LIBERO-Plus, compared with 73.4% for the strongest baseline. The real-robot experiments are relevant as well, although I have a concern about their evaluation protocol. In terms of efficiency, GAM takes 17.5 ms per action-chunk prediction under the matched runtime setting, compared with 29.2 ms for π0.5 and 70.1 ms for OpenVLA-OFT. The separately reported 6.9 ms optimized deployment latency is also useful. These results suggest that the approach offers a favorable balance between robustness and inference cost.
Critical weaknesses
My main concern is whether the contribution of the deep geometric decoding layers has been sufficiently isolated. One of the paper's architectural claims is that predicted action representations benefit from passing through the remaining pretrained GFM layers instead of being decoded directly from the causal predictor. Table 16 in Appendix C.1 provides a relevant comparison: the direct-action variant achieves 84.1% on LIBERO-Plus-Object, while the full model reaches 90.6%. This is a meaningful difference. However, both variants are initialized from a robot-pretrained GAM checkpoint, and the direct-action variant changes its output pathway only during downstream post-training. I am therefore not sure whether the improvement comes specifically from geometric processing in the deeper layers or partly from the advantage of retaining a decoding pathway already used during pretraining. The ablation supports the full architecture, but it does not completely separate these factors. A comparison in which the direct-action architecture is trained from the corresponding pretraining stage would be more convincing.
I also have concerns about the real-robot evaluation protocol. Appendix A indicates that GAM executes eight actions before receiving the next observation, whereas π0.5 and Spatial Forcing execute ten. Keeping the original configurations of the baselines is understandable for a system-level comparison, but this difference matters when evaluating robustness. A shorter execution horizon allows more frequent observation updates and potentially faster correction of action errors, especially during contact-rich manipulation or under camera perturbations. As a result, it is difficult to attribute the real-robot performance differences entirely to the geometric representations. I would like to see a comparison using the same execution horizon, or at least an analysis of how sensitive the results are to this setting. Since each real-robot task is evaluated with 20 trials per condition, reporting the underlying success counts and uncertainty estimates would also help establish the reliability of the observed improvements.
Questions for authors
-
In Table 16, the direct-action variant is initialized from a robot-pretrained GAM checkpoint. Has the direct-action architecture also been evaluated when trained from the corresponding robot pretraining stage? If not, how can the contribution of the deep geometric decoder be separated from the advantage of the pretrained decoding pathway?
-
How sensitive are the real-robot results to the action execution horizon? Would GAM maintain its advantage if all methods executed the same number of actions between observations?
-
Could the authors provide the per-task success counts and explain how the uncertainty estimates in Figure 4 were computed? This would help assess the reliability of the real-robot comparisons.
4 comments
Response to the Reviewer wxJZ (2/2)
[W2, Q3] Per-task evaluation.
Table R8 reports per-task success counts for all methods. The error bars in Figure 4 were mislabeled: they show the minimum and maximum success rates across two sessions of 10 trials each, not the standard deviation stated in the caption. We changed the bars to this range but did not update the caption, which still says standard deviation. We apologize for this error and have corrected the caption of Figure 4 and added the full per-task and per-session success counts to Appendix A.4 (Table 11) of the revised manuscript.
[Table R8. Per-task results.] Each task and camera condition is evaluated in two sessions of 10 trials. Entries report (S1+S2)/N and the pooled success rate.
| Real-World Task | Setting | π0.5 | Spatial Forcing | GAM (Ours) |
|---|---|---|---|---|
| Task 1: Pick & Place | ID | (8+8)/20 (80%) | (8+9)/20 (85%) | (10+10)/20 (100%) |
| OOD | (7+6)/20 (65%) | (7+6)/20 (65%) | (8+9)/20 (85%) | |
| Task 2: Stack Milk & Cube | ID | (7+6)/20 (65%) | (3+4)/20 (35%) | (8+7)/20 (75%) |
| OOD | (1+2)/20 (15%) | (1+1)/20 (10%) | (6+5)/20 (55%) | |
| Task 3: Pot & Pan on Cooktop | ID | (2+3)/20 (25%) | (1+0)/20 (5%) | (3+4)/20 (35%) |
| OOD | (1+0)/20 (5%) | (0+0)/20 (0%) | (2+2)/20 (20%) | |
| Task 4: Insert Cube in Covered Pot | ID | (5+3)/20 (40%) | (2+4)/20 (30%) | (5+6)/20 (55%) |
| OOD | (3+3)/20 (30%) | (1+2)/20 (15%) | (4+3)/20 (35%) | |
| All Four Tasks | ID Total | (22+20)/80 (52.5%) | (14+17)/80 (38.75%) | (26+27)/80 (66.25%) |
| OOD Total | (12+11)/80 (28.75%) | (9+9)/80 (22.5%) | (20+19)/80 (48.75%) |
Dear reviewer wxJZ,
We sincerely appreciate your time and thoughtful efforts in reviewing our manuscript. As the discussion period will end in three days, we would like to gently remind you in case you have any remaining comments. We believe that we have sincerely and successfully addressed your concerns, supported by the corresponding additional experimental results. If you have any questions or concerns, please do not hesitate to let us know. We remain fully available for further discussion and are prepared to conduct additional experiments to address any remaining concerns during the discussion period.
Best regards,
Authors
Dear Reviewer wxJZ,
Thank you very much for your reply and for confirming that our additional experiments and clarifications have resolved all of your concerns. We truly appreciate your time, constructive engagement, and positive support of our work, which have made our paper much stronger.
Best regards,
Authors
Response to the Reviewer wxJZ (1/2)
We thank the reviewer for the positive assessment of our work, for recognizing the value of integrating future prediction and action generation directly into a geometric backbone, and for the precise, constructive suggestions on isolating the deep geometric decoder and analyzing the real-robot evaluation protocol. We address each point below.
We thank the reviewer for pointing out this important distinction. We agree that the suggested experiment more directly isolates the contribution of our architectural integration. As the reviewer suggested, we additionally trained the direct-action architecture with its own action pathway from the start of robot pretraining. In this variant, the action token is decoded directly but everything else matches GAM.
[Table R1. Action-decoding pathway, controlled from robot pretraining.]
Variants (a) and (b) have the same parameter count, training data, pretraining budget, and future-feature and future-depth supervision. This makes variant (a) the matched baseline with an independent action decoder trained from the corresponding pretraining stage that the reviewer suggested.
GAM outperforms variant (a), so pretraining the direct-action pathway does not close the gap. This clearly shows the benefit of routing per-view action tokens alongside predicted future geometry tokens through the deep geometric decoding layers to fuse multi-view 3D structure.
We have added this end-to-end pretrained direct-action ablation to Appendix C.1 (Table 23) of the revised manuscript.
We thank the reviewer for raising this thoughtful point regarding the real-robot evaluation protocol. As reviewer mentioned, we kept each method’s original trained configurations to not disadvantage the baselines. However, we agree that actions within an open-loop chunk may struggle to react to new observations, so a shorter execution horizon can in principle correct errors sooner.
Nevertheless, the difference is small in wall-clock terms. Our demonstrations are recorded at 15 Hz, so 8 and 10 actions correspond to 0.53 s and 0.67 s between observations, a difference of about 133 ms. We therefore expect a two-action difference to have little effect on error correction, which we verify below.
As the reviewer suggested, we additionally evaluated π0.5 executing the first 8 of its 10 predicted actions, following the practice of OpenVLA-OFT [A]. We conduct this comparison on two new, more challenging real-world tasks (see also R1 [W2-2, Q3]): Task 5 (opens a wooden drawer, places the yellow cube inside, and closes the drawer; 100 demonstrations) and Task 6 (unplugging a power cord and inserting the plug into another socket, a contact-rich task requiring precise alignment; 30 demonstrations). Because of the rebuttal timeline, we evaluate each condition with 12 trials and compare against π0.5, the stronger of the two real-world baselines in the paper. Table R7 reports π0.5 at both horizons on these two tasks.
[Table R7. Additional challenging real-world tasks with matched execution horizon.]
Executing 8 instead of 10 actions leaves π0.5’s pooled in-domain success unchanged and only slightly improves its success under the camera shift. GAM maintains its advantage at the matched horizon.
We have added this matched execution-horizon sensitivity analysis to §5.2 and Appendix A.5 (Table 13) of the revised manuscript.
[A] Kim, Moo Jin, Chelsea Finn, and Percy Liang. "Fine-tuning vision-language-action models: Optimizing speed and success." RSS’25.