acceptodds
Withdrawn from ICLR 2027

Video2STL: Grounding VLM-Generated Temporal Specifications for Robot Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Video-based policy learning is particularly promising, as it illustrates target behaviors without requiring action annotations or embodiment-matched demonstrations. A central challenge, however, is deciding what information should be transferred from the video to the robot. Existing approaches commonly convert visual observations into scalar similarity or value signals, or ask foundation models to directly generate reward code. While effective, these approaches can make the temporal structure of a task difficult to inspect, ground, and reuse. We present Video2STL, a framework that converts observation-only videos into parametric Signal Temporal Logic (STL) specifications and uses the resulting formal representation for robot learning. A vision-language model first extracts an embodiment-independent semantic event trace and then constructs a bank of symbolic temporal specifications. The model determines the task structure, while numerical predicate thresholds and temporal bounds are grounded from successful robot trajectories. For policy learning, we separate short- and long-timescale temporal information: short-horizon specifications provide dense rewards through rolling-window quantitative robustness, while a causal monitor over a retained long-horizon specification provides one-time progress rewards for valid temporal prefixes. The same representation is intended to support cross-embodiment transfer from human or animal videos to robot control while remaining interpretable at every stage. Across four manipulation tasks, Video2STL achieves average success-once and success-at-end, compared with for native dense PPO and for Text2Reward; in quadruped locomotion, the Qwen-3.8 and GPT-5.6-based Video2STL policies achieve success across velocities from to while remaining competitive in high-speed energy efficiency. Project webpage: https://video2stl.github.io/video2stl.

Primary area:
reinforcement learning
Keywords:
Robot Learning, Signal Temporal Logic, Vision-Language Models, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_WMPTOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper presents a RL method for learning from video demonstrations. The idea is to use a VLM to describe what happens in the videos using a symbolic STL representation, learn the parameter values of the STL representation using a grounding set available from the target domain, and use the representation to define both dense and sparse rewards for robotic learning.

Critical strengths

The topic is very relevant and and the approach is sensible and interesting.

Critical weaknesses

The paper seems to sell the method with somewhat misleading terms. The abstract talks about cross-embodiment transfer and Fig. 1 demonstrates transfer from animals or humans to robots, but the experiments do not seem to consider cross-embodiment transfer at all. Without the cross-embodiment transfer motivation the paper would be less fascinating but using this motivation needs demonstration and evidence. Also, the abstract talks about “observation-only videos”, i.e., without annotation, but from Section 4.1 it seems that entity roles and predicate vocabulary must be pre-defined before the VLM can learn the temporal structure, which seems like a weakness.

The experiments seem insufficient. More benchmarks and comparison methods could be considered (e.g. from the list of recent works with dense rewards). Also, more ablations would be necessary to understand the contributions of the different aspects of the method (filtering, two timescale rewards, etc.).

Questions for authors

Clarification: the method introduces some hyperparameters. How were their values learned?

Suggestion: the text is a bit verbose at places, e.g., the Introduction. It would be useful to compress and instead show some illustrations of the method/datasets and/or additional experiments.

1 comment

Thank you for raising the question of what transfers across embodiments. In our manipulation experiments, a human demonstrates each task and a robot arm learns to perform it. In locomotion, the source is a video of a dog on a treadmill and the target is the Barkour quadruped. What transfers is the symbolic description of behavior, such as the order of placement and release or the coordination of foot contacts. The predicates are then grounded for the target robot, which trains its own policy. We will clarify this scope in the abstract and Figure 1. Reuse of a single grounded specification across multiple target robots remains to be evaluated.

By observation-only, we mean that the source video contains no action labels or manually annotated events. We provide a domain vocabulary, and the VLM identifies which predicates are supported by the clip and how they relate over time. All four manipulation tasks use the same vocabulary and prompts.

In the revision, we plan to compare filtered and unfiltered specification banks, evaluate local-only against local + progress rewards.

Reviewer_RzvKOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Video2STL, a pipeline that converts an observation-only video into a reward for robot reinforcement learning. A VLM selects semantic events from a manually defined predicate vocabulary and converts them into parametric STL formulas. Successful target-robot trajectories are then used to fit predicate thresholds and temporal bounds and to discard incompatible formulas. The retained specifications provide dense local robustness rewards and sparse long-horizon progress rewards.

The method is evaluated on quadruped locomotion and four ManiSkill3 manipulation tasks. The results are mixed: Video2STL improves performance on some sequential manipulation tasks, but offers no success-rate advantage over Text2Reward in locomotion and is less energy-efficient at most tested velocities. More importantly, the paper does not directly validate whether the generated STL formulas faithfully capture the source videos, nor whether they preserve alternative valid ways of completing the same task.I

Critical strengths
  1. The intermediate representation is interpretable.
    Unlike free-form reward code, the STL formulas expose the events, ordering constraints, and persistence requirements being optimized. This makes failures easier to diagnose. The LiftPegUpright example is a good illustration: the specification requires release and stability even though the benchmark does not.

  2. The evaluation covers different temporal structures.
    The paper considers both periodic locomotion and sequential manipulation. Results on StackCube and PlaceSphere suggest that explicit temporal structure can be useful when tasks contain ordered stages.

Critical weaknesses
  1. The paper never establishes that the generated STL is the correct task specification.

    A finite set of successful trajectories is consistent with many different STL formulas. The proposed filtering step only checks whether successful trajectories satisfy a formula; it does not determine whether the formula is faithful to the video, necessary for success, or sufficient to define the task. A weak formula may pass while omitting essential behavior, and an overly specific formula may encode incidental choices from the single demonstration.

    This is not merely hypothetical. In LiftPegUpright, the generated formula requires release and stability, turning one demonstrated strategy into a stricter objective than the benchmark. More generally, each manipulation task uses only one video, so alternative valid strategies may be discouraged simply because they were not demonstrated.

    The fixed, manually written predicate vocabulary further limits the claim. The method can recombine existing predicates, but it cannot handle a task-critical concept outside the ontology without manually adding both the predicate and its grounding function. Its demonstrated generalization is therefore closed-vocabulary composition, not general video-to-specification learning.

  2. The experiments do not show that STL adds value beyond existing reward construction.

    STL robustness as an RL reward and temporal-logic specification mining are established ideas. The main novelty here is the video-to-STL pipeline, yet that conversion is not evaluated against human-written STL, existing specification-mining methods, or reference annotations.

    Text2Reward is a useful baseline, but not a controlled ablation: it differs in its inputs, supervision, grounding, and final reward terms. In locomotion, Text2Reward matches Video2STL in success and achieves substantially lower cost of transportation at most velocities. Video2STL also retains conventional tracking and safety rewards and uses successful robot trajectories for grounding and filtering. It is therefore unclear whether the observed performance comes from the video, the STL representation, the successful robot data, or conventional reward shaping.

    Finally, the grounded and filtered specification is target-specific. Moving to a new robot requires new predicate implementations, successful trajectories, grounding, filtering, and policy training. The paper demonstrates a morphology gap between the source video and one target robot, but not reuse of the same specification across multiple target embodiments.

Questions for authors
  1. How reliable is the video-to-STL step in practice?
    How often does the method produce a valid and usable specification, and how much do the results change when the same video is processed multiple times?

  2. How do the authors know that the generated STL still captures the intent of the video?
    The current filtering step only checks whether successful trajectories satisfy a formula. This does not rule out formulas that are too weak, incomplete, or simply describe a different behavior. Is there any human evaluation or testing on failed trajectories to verify that the retained formulas express the intended task?

  3. Where does STL provide a clear performance benefit?
    In locomotion, Text2Reward reaches the same success rate and has lower CoT at most velocities. Video2STL also performs poorly on LiftPegUpright at the end of the episode. Could the authors clarify which results demonstrate an advantage over direct reward generation?

  4. How far can the method generalize with a manually defined predicate vocabulary?
    The current system can only use predicates already included in the ontology, and each predicate needs a corresponding grounding function. What happens when a new task requires a concept that is not in the vocabulary? Would extending to such a task require additional manual design?

1 comment

Thank you for these questions. Our contribution is the video-to-STL pipeline: it uses a demonstration to infer temporal relationships among events and makes those relationships explicit in the reward used for policy learning.

  1. We processed the same locomotion video with GPT, Gemini, and Qwen. All three produced specifications that could be grounded and used to train a policy. GPT and Qwen achieved 100% tracking success at every tested velocity. Gemini achieved 100% at the tested velocities from 0.5 to 1.3 m/s. This gives some evidence that the pipeline works across model choices.

  2. We inspected the generated formulas qualitatively to check whether their events and ordering were meaningful. We did not use those judgments to edit the formula banks or retune thresholds. The expert-consistency filter checks compatibility with successful robot behavior. The submission does not include an independent human annotation study or an evaluation on failed trajectories.

  3. PlaceSphere gives the clearest result: Video2STL achieved 91% success-once and 75% success-at-end, compared with 0% on both metrics for Text2Reward (Figure 2). On StackCube the two methods performed similarly. In locomotion, GPT and Qwen matched Text2Reward’s tracking success. Text2Reward was more energy efficient at low and medium speeds, while Qwen had lower mean CoT at the three highest tested speeds.

  4. All four manipulation tasks use the same predicate vocabulary and the same prompts. Locomotion uses a separate domain vocabulary. Within each vocabulary, the method can construct different temporal requirements without us writing an STL template for each task. If an essential concept cannot be expressed with the available predicates, the prompt asks the model to flag it. Supporting that concept would require a predicate definition and a grounding function. Likewise, our experiments demonstrate transfer from human or animal video to a robot with a different morphology. Reuse of the same symbolic specification across multiple target robots remains to be evaluated.

Reviewer_PrmcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Video2STL, a pipeline that converts observation-only videos into banks of parametric Signal Temporal Logic (STL) specifications, grounds their numerical parameters from successful target-robot trajectories, filters inconsistent formulas, and uses the retained specs to train RL policies. The core idea is a clean separation: the video, via a two-stage VLM process, determines symbolic temporal structure while the target robot data determines thresholds and timing. For learning, the authors combine short-horizon rolling-window robustness (dense reward) with a causal prefix monitor over a long-horizon “backbone” specification (sparse, non-farmable progress), and introduce sign-preserving smooth min/max operators. Experiments on quadruped locomotion (Barkour in MJX) and four ManiSkill3 manipulation tasks show competitive or improved performance over hand-engineered dense rewards and Text2Reward across several metrics.

Critical strengths
  1. The separation of symbolic task structure from robot-specific numerical quantities is a useful design choice. An explicit specification makes the transferred requirements easier to inspect, and the distinction between local feedback and one-time milestone rewards is sensible.

  2. The method produces promising results on PlaceSphere and improves substantially over native dense PPO on StackCube. Reporting both success-once and success-at-end is helpful, and the authors also discuss and analyze the LiftPegUpright failure rather than hiding it in the average.

Critical weaknesses
  1. I could not determine which version of the method produced the reported results. Sections 4.2–4.4 describe trace-only synthesis, a disjoint 70/30 grounding/filtering split, and filtering by the tenth percentile of full-trajectory robustness. Appendix A.1.2 instead retains the video during synthesis and uses pooled fifth-percentile per-step robustness; it also describes configuration-derived parameters for Gemini. Tables 4 and 6 include additional tracking and safety rewards. These differences affect the central claim that video supplies structure while robot data supplies numerical grounding. The manipulation appendix lists candidate formulas but does not specify the final local bank, backbone and numerical monitor sufficiently to reconstruct the reward.

  2. The experiments do not isolate the benefit of video-derived structure from expert calibration and the other reward components. The manipulation calibration-set sizes are not reported, and there are no video-versus-text or local-versus-progress ablations. Locomotion success matches Text2Reward, which is more energy-efficient at low and medium speeds. More importantly, expert consistency does not establish that a specification is sufficient for task success: the LiftPeg upright tolerance is 15.9 degrees, exceeding the reported 11.8-degree tipping angle, and final success is only 5%. The additional release requirement makes this a harder objective than the native benchmark, but does not resolve the failure of calibration to support stable release.

Questions for authors
  1. Please check the main-text and appendix procedures and identify which version produced each result. Which numerical settings and reward terms were fitted from robot data, and which were supplied separately?

  2. How many successful robot trajectories are used per manipulation task, how are they obtained, and how sensitive are the results to their number? As additional evidence, a video-versus-text comparison with identical calibration data would clarify the video contribution. If action labels are available, a same-data imitation baseline such as Diffusion Policy would also be informative.

  3. Which backbone and monitor are used for each manipulation task? How are expired deadlines, repeated event starts and persistence handled? A local-only versus local-plus-progress ablation would help establish the contribution of the proposed reward split.

1 comment

Thank you for highlighting where the manuscript did not clearly distinguish the shared methodology from domain-specific implementation details. Both locomotion and manipulation follow the same high-level pipeline: extracting a semantic event trace from video, generating a parametric STL bank from that trace, and grounding and filtering the specifications using successful target-robot trajectories. We acknowledge that the descriptions in the main text and appendix need to be reconciled to make the experimental procedures clear and consistent.

The STL generation stage is prompted to construct specifications from the semantic event trace. Our reference to the Conversations API concerned the conversational workflow, but the appendix did not clearly distinguish the input supplied in the 2nd stage prompt from the context retained across stages. We will clarify this in the revised version.

For expert-consistency filtering, manipulation uses a 0.10 quantile, whereas locomotion uses a 0.05 quantile. Both follow the same procedure except for the numerical values. We will describe the domain-specific settings and robustness aggregation explicitly.

For locomotion, the treadmill video provides evidence about gait coordination. Following a commanded velocity is an additional task objective, with the command varying across trials. We therefore include an explicit velocity-tracking terms alongside the video-derived STL reward. We will make their respective roles clear in the main text.

In the revision, we also plan to compare video-derived and text-derived STL specifications on a manipulation task and provide implementation details for grounding and filtering.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.