- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
TARGET-VLA augments a vision-language-action policy with compact representations retrieved from frozen SAM3 and Depth Anything 3 models. Instruction-derived object masks and an end-effector mask provide additive spatial biases to query-to-feature cross-attention, while the pretrained features supply the retrieved content. The segmentation branch jointly summarizes the prompted regions, whereas the depth branch separately retrieves object and end-effector information before aggregation. These tokens condition an action expert together with VLM and robot-state tokens. Evaluations cover multiple simulation benchmarks, zero-shot transfer to LIBERO-Plus, and three real-world tasks. The reported comparisons show improvements over unguided retrieval and over supplying masks as additional feature inputs.
Critical strengths
-
A useful comparison of how task-conditioned masks are used. Table 3 compares TARGET-VLA with both unguided query retrieval and a variant that supplies the same masks as feature inputs, using the same frozen visual sources and 104 final conditioning tokens. The improvement over the mask-input variant from 4.45 to 4.68 in CALVIN average task-chain length, and from 77.0% to 84.3% in Success@5, is meaningful evidence that the retrieval interface matters. This is more informative than comparing only against a VLM-only policy.
-
Encouraging improvements across evaluation settings. Relative to unguided retrieval, success increases from 81.9% to 88.8% on zero-shot LIBERO-Plus, and the equal-weight average across the three real-world in-domain tasks increases from 62.8% to 83.3%. The breadth of these results supports the practical value of task-conditioned visual retrieval beyond one simulation benchmark, even though the individual real-world sample sizes are modest.
Critical weaknesses
-
The design does not directly address the precision motivation. The mask bias favors task-relevant regions; it does not specify contact-sensitive structure or alignment requirements within them. Fine geometry may remain available in DA3 features, so coarse masks do not invalidate the method. However, whole-task success cannot distinguish improved geometric precision from better target selection/background suppression. I regard this as useful visual retrieval, but the distinctive benefit of soft-mask guidance over simpler region guidance remains unclear. Matched bounding-box or binary-mask attention baselines would better establish that contribution.
-
The central mechanism is not fully isolated or validated across training runs. The unguided variant removes DA3 validity gating and embeddings together with the attention bias (Section 4.4; Appendix C.4). Thus, its comparison measures a package of changes. The mask-input control is valuable, but whether it preserves validity conditioning is unclear. Moreover, CALVIN ablations use one training seed; sequence-level bootstrap intervals do not capture training variability. This limits confidence that the specific attention design consistently explains the advantage over the strongest controlled alternative.
Questions for authors
-
Contribution and evidence: What evidence connects the retrieval improvements to finer manipulation accuracy, rather than better target selection/background suppression? A failure analysis separating selection errors from alignment/contact errors would help. Separately, how does soft-mask guidance compare with bounding-box or binary-mask guidance under matched visual sources, validity conditioning, and token budgets? Either targeted analysis would materially inform my assessment; I am not requesting a new benchmark suite.
-
Clarification: Does “Query retrieval + mask input” retain the same DA3 validity gating and embeddings as TARGET-VLA? With these components fixed, what improvement comes from enabling only the attention bias?
-
Reliability: Does the advantage over unguided and mask-input retrieval persist across independent training seeds? Means and dispersion for CALVIN average task-chain length and Success@5 would address uncertainty not captured by the sequence-level bootstrap.