- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper explores fine-tuning a VLA to be conditioned on various visual cues to indicate the target object and evaluating the effectiveness of these augmentations in terms of task success, tested on four VLA backbones involving simulation and real-world tasks. The main contribution is the collection of results from doing these experiments, specifically that adding such augmentations improve task performance, and that live annotations of the target object is better than static annotations, with delayed annotations not having a great impact when the delay is short.
Critical strengths
I believe the paper's systematic evaluation of different visual guidance methods is thorough, and the claims that the authors draw are well-supported by results from their experiments.
Critical weaknesses
My main concern with this paper is on the novelty, as the paper serves as an evaluation of several rudimentary method of visually indicating objects and does not report significant results that will benefit the broader community. The results that the authors draw are not surprising: adding visual guidance improves performance over language-only, real-time annotations are better than stale ones, and wrist cameras matter more than exocentric camera. Other findings are also not informative. For example, the authors claim that static annotations hurt wrist cameras more than exocentric cameras, but for a fixed external camera, the annotation will always be coincident with the target object until the object is moved, but for a wrist camera, a static annotation is not accurate whenever there is motion on the robot arm, so other than the very beginning of the motion, the annotation serves as distraction rather than useful information. When the authors are acknowledging prior work like ControlVLA and VP-VLA that is already using real-time visual annotation, there does not seem to be major findings from this paper that could further inform future work along similar areas.
Questions for authors
- For systems that already use visual cues to guide VLAs, which of your findings could change how a practitioner uses these systems?
- For the case of static visual cues, if they are removed from the wrist cameras and only kept on the exocentric camera, does task success improve meaningfully?