- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
GF-VLA learns scene latents supervised by future depth and motion latents supervised by sparse, temporally ordered UVD end-effector traces. Motion queries attend to scene latents, and the resulting features condition a flow-matching action expert. Geometry decoders are training-only. The evaluation covers LIBERO, LIBERO-Plus, RoboCasa-GR1 and real manipulation on two platforms, with controls for geometry dimensionality and supervision targets.
Critical strengths
- Combining endpoint scene geometry with within-chunk motion geometry is a clear spatial-control motivation. Training-only decoders preserve action-only deployment, and applying the representation to both grippers and dexterous hands gives the method practical breadth.
- The evaluation includes meaningful 2D/3D motion, current/future depth, and single-branch controls. On RoboCasa the joint model reaches 71.9% versus the base VLA's 65.6%; real-robot easy/hard improvements also reproduce from the stated counts. The appendix retains task-level regressions, making the empirical picture more informative than aggregate gains alone.
Critical weaknesses
- The evidence for dual coupling specifically is limited. Table 5 reports 70.8% with scene supervision alone and 71.9% jointly, so the marginal joint gain is 1.1 points out of the 6.3-point total. Without independent-training uncertainty or an attention control retaining both losses, most of the evidence supports future-scene supervision rather than the necessity of the coupling mechanism.
- The geometric diagnostics measure agreement between predicted and executed motion, not accuracy against desired motion or successful task completion; a consistently wrong policy can still agree with its own predictions. The calibrated UVD/depth target-generation pipeline is also insufficiently specified for direct reproduction from the supplied materials.
Questions for authors
- Decision-relevant mechanism check: is the 1.1-point joint-over-scene gain stable across available runs, and what happens when scene-to-motion attention is removed while retaining both geometry losses and a comparable parameter budget? If this control is unavailable, please narrow the dual-coupling claim.
- Diagnostic/reproducibility clarification: please distinguish prediction-execution consistency from goal- or demonstration-referenced accuracy, including on failed rollouts, and specify the calibration/target-generation pipeline. Which LIBERO and LIBERO-Plus comparisons use the same underlying base VLA and training budget?
5 comments
Response (Part2)
- Real-world branch ablations
We further examine Bricks Disassembly and Hanger Placement, evaluating each variant in 30 trials per task and condition. Easy conditions resemble demonstration collection. Hard conditions introduce distractor bricks and unseen hook positions, respectively, increasing the demands on robust execution and spatial generalization.
| Task | Condition | Base | Scene-only | Motion-only | Full |
|---|---|---|---|---|---|
| Bricks Disassembly | Easy | 53.3 | 60.0 | 56.7 | 63.3 |
| Bricks Disassembly | Hard | 36.7 | 36.7 | 46.7 | 50.0 |
| Hanger Placement | Easy | 50.0 | 56.7 | 56.7 | 63.3 |
| Hanger Placement | Hard | 23.3 | 26.7 | 30.0 | 33.3 |
| Two-task average | Easy | 51.7 | 58.3 | 56.7 | 63.3 |
| Two-task average | Hard | 30.0 | 31.7 | 38.3 | 41.7 |
The averages weight the two tasks equally.
Removing the motion branch from Full reduces success by 10.0 pp under Hard conditions, compared with 5.0 pp under Easy conditions. Under Hard conditions, Motion-only exceeds Base by 8.3 pp, while Scene-only exceeds Base by 1.7 pp. Full remains best, with a further 3.3 pp gain over Motion-only. These comparisons support complementary contributions from the two branches, with a larger incremental motion benefit under the evaluated conditions that place greater demands on execution and generalization.
[Attention control retaining both geometry losses]
To isolate the coupling mechanism, we trained a control that retains both query sets, both geometric prediction heads and losses, and motion-to-Action-Expert conditioning, while removing only motion queries' access to scene latents. Both branches continue to read vision-language features. This preserves the parameterized modules and query counts, changing the attention access rather than removing a branch or adding parameters. We evaluate this control on RoboCasa's in-container placement and closure tasks.
| Model | Seed 7 | Seed 27 | Seed 1137 | Mean ± Std |
|---|---|---|---|---|
| Full | 64.7 | 64.3 | 69.0 | 66.0 ± 2.6 |
| Without scene-to-motion access | 62.7 | 60.7 | 65.0 | 62.8 ± 2.2 |
Results across three evaluation seeds on in-container placement and closure tasks.
Removing the pathway lowers success in every reported evaluation, with a mean decrease of 3.2 pp. This provides evidence for the directed scene-to-motion connection beyond the presence of both supervision losses.
R2. Response to W2 and Q2: Interpreting geometric consistency together with task success
SR measures goal completion, while UV and depth errors measure prediction–execution consistency. Motion latents represent future EEF movement and condition the Action Expert. The consistency diagnostic examines whether the decoded motion agrees with the movement actually produced by the resulting actions. We agree that an incorrect motion can still be executed consistently, so low geometric error alone does not establish task success.
- Why not directly compare with a demonstration?
Demonstrations provide training supervision, but online rollouts can visit different states, and multiple trajectories can complete the same task. Direct comparison with one demonstration can therefore mix motion error with differences in state, execution timing, and valid trajectory choice. We instead compare predicted and executed EEF reference points at matched time offsets in the same camera frame. UV error measures their mean absolute image-coordinate difference in pixels, and depth error measures their absolute optical-axis difference in millimeters.
- Overall results and successful versus failed rollouts
We report SR alongside UV and depth consistency, followed by separate errors for successful and failed rollouts. All evaluations use the same trained GF-VLA checkpoint.
| Evaluation seed | Overall SR (%) ↑ | UV error (px) ↓ | Depth error (mm) ↓ |
|---|---|---|---|
| 7 | 71.9 | 3.36 | 8.09 |
| 27 | 72.6 | 3.29 | 7.84 |
| 1137 | 75.2 | 3.25 | 7.74 |
| Evaluation seed | Success UV (px) | Failure UV (px) | Success depth (mm) | Failure depth (mm) |
|---|---|---|---|---|
| 7 | 3.30 | 3.38 | 7.71 | 8.65 |
| 27 | 3.27 | 3.26 | 7.65 | 8.17 |
| 1137 | 3.24 | 3.23 | 7.59 | 8.11 |
Successful and failed rollouts show similar UV errors, while successful rollouts have lower depth errors across all three evaluation seeds. These results illustrate that consistency and task outcome capture different aspects of policy behavior. We therefore analyze SR together with UV and depth consistency to assess both task completion and the agreement between predicted and executed motion. This joint interpretation provides a more complete view of policy performance than either success rate or geometric consistency alone.
Response (Part3)
R3. Response to W2 and Q2: Calibration coordinates and construction of UVD and depth targets
Scene targets use future depth observations at the action-chunk endpoint, while motion targets express temporally ordered EEF reference points within the same chunk in the designated camera's UVD coordinates. Appendix A.1, Geometric Target Preparation, specifies the simulation replay, reference-point definitions, projection convention, and real-world depth sources. We summarize the construction here.
- Simulation
The released LIBERO and RoboCasa-GR1 training data do not directly provide all depth maps and camera parameters needed for our geometry targets. We restore demonstration episodes from the corresponding HDF5 simulator records, replay the recorded states, and rerender camera views while retaining the original actions. This provides frame-aligned metric depth, camera parameters, and EEF reference-point positions. LIBERO uses the gripper center; RoboCasa-GR1 uses the midpoint between the thumb and index-finger positions of each hand.
- Projection and temporal alignment
The reference point is transformed from its recorded world or robot frame into the designated RGB camera frame using the calibrated extrinsics, then projected using intrinsics adjusted for image preprocessing. U and V are image coordinates; D is camera-frame depth along the optical axis. For training, U and V are divided by image width minus one and image height minus one, respectively, and D remains in meters. Motion traces sample temporally ordered positions over the action chunk, including its start and endpoint. Scene targets use depth at that same endpoint. LIBERO supplies scene-depth supervision from the agent and wrist views; RoboCasa-GR1 uses the main view.
- ARX-Lift2S
RealSense D405 cameras provide depth with depth-to-RGB alignment enabled. Motion targets are constructed from recorded EEF poses and the center of each gripper. Both gripper trajectories are projected into the central primary RGB camera, which defines their UVD coordinates. Scene supervision uses measured depth from that camera at the chunk endpoint.
- RM75
The external camera is a RealSense L515 and the wrist camera is a D435, also with depth-to-RGB alignment enabled. Motion targets use the recorded EEF poses and gripper-center reference point, projected into the fixed external RGB camera. Scene supervision uses measured endpoint depth from both external and wrist cameras. The wrist view adds scene supervision without changing the motion-coordinate frame.
The motion and scene targets are associated with the same demonstration action interval and serve as training supervision. Future depth and recorded EEF poses are not supplied as inputs to the deployed policy, and the geometry decoders are training-only.
R4. Response to Q2: LIBERO-to-LIBERO-Plus transfer and consistent score aggregation
For GF-VLA across the two benchmarks, the model and training budget are unchanged. We train on the original LIBERO demonstrations and directly evaluate the same checkpoint on all 10,030 LIBERO-Plus instances, without further training or adaptation. GF-VLA uses the Qwen3.5-4B backbone and flow-based Action Expert. LIBERO-Plus introduces different evaluation conditions without an additional training stage.
All methods in our LIBERO-Plus comparison follow the same zero-shot transfer protocol. Their policies are trained on LIBERO data and then evaluated on LIBERO-Plus without additional training on LIBERO-Plus. This evaluates how LIBERO-trained policies generalize to the benchmark's perturbed conditions.
We have also unified the aggregation of LIBERO-Plus scores. Because its seven evaluation categories contain different numbers of instances, we compute Overall SR by weighting each category's success rate by its instance count. The source-reported Overall scores for VLA-JEPA and PILOT correspond to the simple, unweighted average of their seven category scores. We retain their published category scores and recompute Overall using the same instance-weighted rule, updating VLA-JEPA from 79.5% to 78.0% and PILOT from 81.0% to 80.3%. These updates reaggregate the published results and do not involve new model evaluations. Appendix B now provides the result sources, evaluation settings, category instance counts, and details of these aggregation updates.
Response (Part1)
We thank the reviewer for recognizing the spatial-control motivation for combining endpoint scene geometry with within-chunk motion geometry, the action-only deployment and applicability to both grippers and dexterous hands, and the informative controls and transparent reporting of task-level results. Below, we address these points, clarify the purpose of the geometric diagnostics, and specify the target-generation and benchmark-comparison settings.
R1. Response to W1 and Q1: Incremental benefits of joint supervision and evidence for scene-to-motion coupling
[Method motivation]
Motion forcing explicitly supervises the EEF's 3D trajectory throughout an action chunk, encouraging the policy to capture where the EEF should move and how its movement unfolds over time. The resulting motion latents condition the Action Expert. Scene forcing supplies complementary spatial context by supervising scene depth at the endpoint of the same chunk, and motion queries read these scene latents before conditioning action generation.
We examine the contribution of motion forcing across evaluation seeds and task conditions, and separately test the directed pathway while retaining both geometry losses.
[Additional evaluations and task-dependent contributions]
Table 5 evaluates each trained model on 24 RoboCasa-GR1 tasks with 50 rollouts per task, totaling 1,200 episodes per model. We extend this comparison to three evaluation seeds. The variants use the same backbone, demonstration data, observation inputs, and training setup. Each variant keeps its trained checkpoint fixed across evaluations.
Results across three evaluation seeds.
Full exceeds Scene-only in all three evaluations, with a reported mean gain of 1.8 pp, and achieves the highest success rate among the four variants in each evaluation. These results show that the joint-over-scene gain recurs beyond the original evaluation.
The branch ablations remove the corresponding branch rather than only disabling its loss. Scene-only removes the motion queries, UVD prediction head, motion loss, and motion-to-Action-Expert connections. Its Action Expert uses native vision-language features without directly reading scene latents. Motion-only removes the scene queries and depth supervision while retaining motion supervision and motion-to-Action-Expert conditioning. The attention control below addresses the pathway separately.
Scene supervision already provides useful geometry for action learning. We believe that the incremental benefit of motion forcing becomes more apparent when tasks require more precise movement and stronger spatial coordination. We therefore separate RoboCasa's two task categories. In-container placement and closure requires placing an object within a constrained space and subsequently closing an articulated container, whereas the other category involves object pick-and-place.
Results across three evaluation seeds. Placement + closure denotes in-container placement followed by closure.
Adding motion forcing to Scene-only yields 5.2 pp on placement and closure, compared with 0.6 pp on pick-and-place. This supports its usefulness when successive EEF movements must remain coordinated with changing relationships among the hand, object, and container.