- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper presents a RL method for learning from video demonstrations. The idea is to use a VLM to describe what happens in the videos using a symbolic STL representation, learn the parameter values of the STL representation using a grounding set available from the target domain, and use the representation to define both dense and sparse rewards for robotic learning.
Critical strengths
The topic is very relevant and and the approach is sensible and interesting.
Critical weaknesses
The paper seems to sell the method with somewhat misleading terms. The abstract talks about cross-embodiment transfer and Fig. 1 demonstrates transfer from animals or humans to robots, but the experiments do not seem to consider cross-embodiment transfer at all. Without the cross-embodiment transfer motivation the paper would be less fascinating but using this motivation needs demonstration and evidence. Also, the abstract talks about “observation-only videos”, i.e., without annotation, but from Section 4.1 it seems that entity roles and predicate vocabulary must be pre-defined before the VLM can learn the temporal structure, which seems like a weakness.
The experiments seem insufficient. More benchmarks and comparison methods could be considered (e.g. from the list of recent works with dense rewards). Also, more ablations would be necessary to understand the contributions of the different aspects of the method (filtering, two timescale rewards, etc.).
Questions for authors
Clarification: the method introduces some hyperparameters. How were their values learned?
Suggestion: the text is a bit verbose at places, e.g., the Introduction. It would be useful to compress and instead show some illustrations of the method/datasets and/or additional experiments.
Thank you for raising the question of what transfers across embodiments. In our manipulation experiments, a human demonstrates each task and a robot arm learns to perform it. In locomotion, the source is a video of a dog on a treadmill and the target is the Barkour quadruped. What transfers is the symbolic description of behavior, such as the order of placement and release or the coordination of foot contacts. The predicates are then grounded for the target robot, which trains its own policy. We will clarify this scope in the abstract and Figure 1. Reuse of a single grounded specification across multiple target robots remains to be evaluated.
By observation-only, we mean that the source video contains no action labels or manually annotated events. We provide a domain vocabulary, and the VLM identifies which predicates are supported by the clip and how they relate over time. All four manipulation tasks use the same vocabulary and prompts.
In the revision, we plan to compare filtered and unfiltered specification banks, evaluate local-only against local + progress rewards.