- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes Change2Act, a method that uses image-goals as prompts for VLAs through an object-centric reasoning module. It operates by encoding spatial semantic differences into a descriptor that is mapped to goal tokens through several pre-trained models (SAM, DINO, GroundingDINO). Together with the original goal image, a pre-trained robot policy () is adapted towards producing actions that reach the goal.
Critical strengths
- The overall idea is simple and intuitive (the implementation less so).
- Error prediction accuracy is a bit better than prior methods.
Critical weaknesses
- The underlying assumption that text prompting requires "substantial human effort” compared to image goal is misleading. How would one acquire an image-goal in the real world? It would typically require a human to organize the objects into the desired configuration, which arguably might require more effort than just text (and the human has to complete the task before the robot). I suggest toning down the claim and provide real-world examples where goal-images are easily acquired.
- Novelty and prior work: Goal-VLA appears to be doing something similar but on more complex tasks. There is also prior work that utilizes SAM-based masks and/or visual reasoning such as VP-VLA and Liu et al..
- Complex masking and correspondence pipeline: to obtain a change mask, several stages are required that might require low-level tuning, such as adaptive thresholding and morphological processing. Additional filtering stages that might be brittle. The values for all of these stages are nowhere to be found.
- Reliance on pre-trained models: the model largely relies on the quality of several pre-trained models (SAM, DINO, GroundingDINO) and heuristics tuning for filtering.
- Limited benchmark: model is only evaluated on the saturated LIBERO dataset.
- Evaluation: this is the most significant concern - the authors only measure action prediction error rather than task success. This is a weird choice for evaluation, and not reporting success rates raises serious concerns regarding the soundness and validity of the proposed approach (there is one case study, but definitely not enough).
- Limitations are not discussed.
Questions for authors
See Weaknesses. I don't believe this paper is ready for publication.