- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes 3DVLA, a framework for augmenting pretrained vision-language-action models with explicit 3D information. Its components include multi-view spatial fusion, object-centric instance tokens, a coordinate-conditioned feature predictor trained with masked self-supervision, and uncertainty-guided geometric aggregation. The framework uses pseudo-labels from a frozen perception model and completion coordinates from a pretrained point-cloud completion network. Evaluation covers LIBERO-Plus, RoboTwin 2.0, and three real-world manipulation tasks, with integrations into multiple VLA backbones. The reported results, including the five-run evaluations in the appendix, show improvements over the corresponding base policies.
Critical strengths
-
The framework provides a potentially useful way to incorporate geometry into pretrained VLAs. Preserving the frozen VLM backbone while introducing instance and completion tokens is practically appealing. Integration with multiple policy backbones provides evidence that the approach is not restricted to a single architecture.
-
The aggregate performance improvements have meaningful empirical support. Tables 9–10 report five-run results: LIBERO-Plus improves from 83.9 ± 1.2 to 86.2 ± 0.8, while X-VLA on RoboTwin improves from 69.7 ± 0.9 to 80.6 ± 1.2 on Easy and from 39.0 ± 3.1 to 48.2 ± 2.4 on Hard. The real-world evaluation in Table 3 provides additional evidence of practical utility. These results make the overall system worth investigating.
Critical weaknesses
-
The comparisons do not sufficiently establish the incremental value of the proposed architecture over simpler geometric augmentation. The method uses depth, camera calibration, pretrained perception supervision, and a pretrained completion model. However, the experimental description does not clearly establish matched input access and post-training budgets for the corresponding baselines. Appendix A specifies the proposed two-stage training schedule, but does not adequately explain the matching baseline protocol. Moreover, although Section 2 discusses SpatialVLA, PointVLA, and GeoVLA, the experiments include neither a directly relevant 3D-aware VLA comparison nor a same-backbone alternative using simpler 3D positional encoding or point-feature injection. DP3 is included, but does not control for the pretrained VLM backbone. This leaves unresolved how much of the improvement requires the proposed instance-and-completion design. Table 8 further exposes an input-protocol ambiguity: perturbing depth maps substantially degrades the π0/π0.5 baselines, yet their use of depth is not explained.
-
The central geometric mechanisms are insufficiently specified and isolated by the evidence. In Section 3.3.1, the pretrained PointAttn network already supplies the completed geometry used to construct ; the proposed predictor generates features conditioned on these coordinates. The explicitly defined cosine and variance losses in Eqs. (12)–(13) concern masked observable tokens, leaving the learning signal for wholly unobserved completion locations unclear. The A-to-B ablations jointly introduce self-supervision, completion queries, and predictor-generated features, so they do not establish the specific benefit of retaining the masked predictor at inference. Separately, Eq. (5) assumes globally associated pseudo-target identities and valid-view sets , without explaining how these are constructed from view-specific 2D detections. Unprojecting 2D centers does not generally produce a shared physical object centroid across views. These omissions concern the mechanisms underlying the paper’s main claims, rather than peripheral implementation details.
Questions for authors
-
Clarification of the comparison protocol: What inputs, pretrained checkpoints, trainable modules, and post-training budgets are used for each corresponding baseline? In Table 8, why does perturbing only depth maps affect the π0/π0.5 baselines? Do these baselines consume depth, or does the perturbation also affect other observations?
-
Evidence needed to assess the contribution: Under matched geometric inputs and training budgets, how does the method compare with a simpler 3D positional-encoding or point-feature-injection baseline? For the completion branch, can the authors distinguish the benefit of the external completion coordinates, masked self-supervision, and retaining the predictor at inference?
-
Clarification of global instance supervision: How are independent per-view pseudo-detections grouped into the global targets and valid-view sets used in Eq. (5)? How are inconsistent depth-derived centers, missing detections, and nearby same-class instances handled?
-
Clarification of completion learning: Which losses provide supervision to , and how are target view identifiers assigned to completion coordinates? If completion features rely on generalization from masked visible locations and downstream action supervision, what evidence directly evaluates this generalization?