- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes a new method for one-shot/few-shot learning for VLAs. It breaks an expert demo into two information sources: semantics and positional. Semantics are identified using SAM2, CoTracker, and SigLIP, where objects from the demonstration and current scene are matched. Then they can transfer by considering the arc of movement in the demonstration, and accounting for the differences in position of the current scene minus the demonstration scene using a linear interpolation over time. This allows them to create an approximate sequence of states in the new scene, which they somehow(?) covert into an action draft. This action draft is then converted into a viable action via flow matching.
Critical strengths
I like the idea of extracting semantic information from a demonstration and mapping that to information in the new scene. That is a nice way of using that information explicitly rather than implicitly as in prior methods.
The experiments look good. Any one shot performance in real life above 0% is awesome.
Critical weaknesses
Presentation can be hard to following. For example, line 198, Q^D is the demonstration position, Q^E is the scene/execution position I think? This should be clearly stated. I get how \tilde{e} is like a corrected trajectory in the new scene, but how is this converted into an action draft (line 214). The action draft is an approximate action, which you then using flow matching to "correct" into a viable action? Figures 1 and 2 are not interpretable to me.
Questions for authors
- How do you convert \tilde{e} into an action draft? Is this just something like IKD from the approximate state trajectory
- Is \tilde{e} refined during the trajectory?
- How do you choose the stopping times?
- How do you know what dimensions cannot be recovered from the monocular view? Do you assume prior knowledge on this?
- You only pretrain, there is no training for every new task, correct?