acceptodds
Under review as a conference paper at ICLR 2027

T-LESSA: TEMPORAL-LOGIC EVENT SPECIFICATION STEERING OF VIDEO DIFFUSION ATTENTION

93 views

Abstract

Recent advancements in generative video promise to transform challenging scenarios into training data: rare driving situations for autonomous vehicles, synthetic scenes for robot simulation, and realistic deepfakes for stress testing detection models. However, in practice, we still prompt a model and pray to get faithful videos. While a large body of research exists on generating good video, there is a lack of methods to enforce temporal and logical requirements (e.g., ordering, negation, co-occurrence) during video generation. Prior work has focused on fine-tuning with preference feedback or verifiable rewards. However, these methods require additional training and reduce a temporal-logic requirement to a scalar reward. This limitation has motivated a recent line of work that steers video generation at inference time without any training. Although these approaches control when each required event appears, they do not ensure that the generated videos align with their temporal and logical requirements. Therefore, we introduce *T-LESSA*, a training-free method that steers video generation toward satisfying a temporal-logic requirement. We show that *T-LESSA* improves temporal completion by **14.3%** and, across four backbones, temporal-logic satisfaction by **21.8%** over the unsteered baseline, at 1.7% to 13.5% additional generation time. Furthermore, *T-LESSA* improves compositional alignment in 14 of 16 backbone-category pairs on T2V-CompBench, and largely preserves video quality.

Primary area:
generative models
Keywords:
Video Generation, Text-to-Video, Formal Method, Inference-time guidance, Temporal Logic

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_SaGKOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes T-LESSA, a training-free method that steers text-to-video diffusion models toward satisfying temporal-logic (TL) specifications. An LLM agent translates a prompt into a finite-trace TL specification and identifies the phrase for each atomic event. During denoising, the cross-attention mass on each event phrase is used as a soft score for when the event occurs, and a differentiable satisfaction score is computed by modeling the scores as a probabilistic trace over the specification's automaton. The gradient of this score updates a bias on the corresponding attention logits. The paper also introduces TLVBench, with 400 prompts across four specification classes (negation, conjunction, ordering, and their combination). Across four backbones, T-LESSA improves TL satisfaction by 21.8% on average under VLM judges and NeuS-V, improves temporal completion on StoryEval and TCBench, and adds 1.7% to 13.5% generation time.

Critical strengths

S1. A thorough experimental evaluation with consistent improvements. The method is evaluated on four backbones with different architectures, including CogVideoX with joint attention, using three judges (two VLMs and NeuS-V), three seeds per prompt, and significance tests against the unsteered backbones (Table 1). T-LESSA improves TL satisfaction on all four backbones, with average relative gains of 19.7%, 32.1%, 20.7%, and 14.9% on Wan2.1-14B, CogVideoX-5B, VideoCrafter2, and AnimateDiff. The gains extend to external benchmarks. On TLVB-ORDER, TCR rises from 0.290 to 0.360 on Wan2.1 and from 0.060 to 0.140 on CogVideoX (Table 4a), and compositional alignment on T2V-CompBench improves in 14 of 16 backbone-category pairs (Table 2). Overall VBench quality decreases by less than 0.006 in all cases (Table 3), and the supplementary material provides an extensive set of qualitative examples in which steering visibly changes the generated events. The paper also includes additional experimental checks. For example, it verifies that SwitchCraft's advantage on Wan2.1 is not an artifact of its pipeline (App. C.3), compares two objectives for co-occurrence (Table 8), and measures the runtime and memory cost (Tables 10 and 11).

Why it is consequential: The improvements hold beyond the authors' own benchmark and across architectures that differ in their attention design, which provides strong evidence that the method works in practice.

S2. A principled, general, and practical way to express temporal requirements as a guidance objective. The method translates a prompt into a finite-trace TL specification and converts it into a differentiable objective by combining cross-attention scores with the specification's automaton. This construction builds on established probabilistic model checking, and the resulting score agrees with classical TL evaluation on binary traces (Theorem 1). As a result, different requirements, such as ordering, absence, and co-occurrence, are handled by the same procedure, whereas existing guidance methods can state only some of them (Table 9). The objective is also not tied to attention steering, since replacing the objective of a latent-steering method with it improves temporal accuracy (Table 4b). In addition, the method requires no training, no additional model forward passes, and no external model in the denoising loop. It adds 1.7% generation time on Wan2.1-14B and 13.5% on CogVideoX-5B (Table 10), whereas latent optimization with TempoControl takes 4.5 times longer and 2.2 times the memory (Table 11).

Why it is consequential: This provides a single, general interface for specifying temporal and logical requirements at inference time, which can be combined with different control mechanisms and applied to existing video models with little overhead.

Critical weaknesses

W1. Further validation of the ordering results would strengthen the paper. The paper evaluates the method on four backbones and several external benchmarks, and provides an extensive set of qualitative examples in the supplementary material. However, the main results rely on 7B to 8B VLM judges that see eight frames, and their agreement with human judgment is not reported. The TL specifications of TLVBench are also produced by the same LLM agent used in the method, whose reliability the authors note is not yet measured. In addition, Table 4a, which verifies the ordering results with external metrics, compares T-LESSA with the unsteered backbones alone, while SwitchCraft, which is designed for ordering, scores higher than T-LESSA on TLVB-ORDER for Wan2.1-14B under both VLM judges (Table 1).

Why it is consequential: Temporal ordering is a central requirement that distinguishes this work from existing guidance methods, so a direct comparison with these methods and a human check would make the main claim more convincing. (See Q1–Q3.)

W2. The modeling choices behind the satisfaction score could be examined further. Following prior attention-based guidance methods, T-LESSA uses the cross-attention mass on an event phrase as a soft score for the event, and the steering clearly changes the generated events. However, the scores are treated as independent across time and events, although events in a video persist over consecutive frames. Under this assumption, moderate scores spread over several temporal positions already bring the score of an event close to one (App. C.1), and it is unclear how well such a value indicates whether the event actually appears. In addition, the supplementary videos suggest that steering occasionally realizes an event in an unnatural way. For example, for a prompt in which a woman kneads dough, rolls it flat, and then cuts circles out of it, three cut pieces of dough suddenly appear while she is still rolling.

Why it is consequential: This assumption shapes what the steering actually optimizes, so understanding its effect, together with how often such artifacts occur, would clarify when a higher satisfaction score corresponds to a more faithful video. (See Q4–Q7.)

Questions for authors

Overall, I found this paper interesting and well executed. Controlling the order, absence, and co-occurrence of events is an important problem for video generation, and the paper addresses it with a training-free method that applies to different backbones at a small additional cost. Expressing these requirements as a single objective derived from the prompt is an appealing idea, and the steering visibly changes the generated events while largely preserving video quality. The evaluation covers four backbones and several external benchmarks, and the supplementary material provides an extensive set of qualitative examples. The questions below are mainly intended to further strengthen the evidence for the ordering results and to clarify how the modeling choices behind the satisfaction score relate to the generated events.

[R] = additional result requested, [C] = clarification.

1. Evaluation (W1)

  • Q1 [R] (W1). Could the authors report a human evaluation on a subset of TLVBench, and the agreement between the VLM judges and human annotators?
  • Q2 [R] (W1). App. B.1 reports that 86.3% of the translations passed the agent's own validation on the first attempt, but this measures agreement between two LLM calls rather than accuracy. Could the authors report a human check of the translated specifications and event phrases on a subset of prompts?
  • Q3 [C/R] (W1). Table 4a verifies the ordering results with TC-Score, TCR, and StoryEval, but compares T-LESSA with the unsteered backbones alone. Since SwitchCraft scores higher than T-LESSA on TLVB-ORDER for Wan2.1-14B under both VLM judges (Table 1), a comparison with it on these metrics would be informative. Is there a reason the guidance baselines are not included in Table 4a? If feasible, including SwitchCraft (on Wan2.1) and Video-MSG (on CogVideoX), which were already run for Table 1, would help.

2. Satisfaction score and effects of steering (W2)

  • Q4 [C/R] (W2). The satisfaction score treats the event scores as independent across time and events. How does this assumption affect the steering? For example, how well does the satisfaction score SφS_\varphi at the end of steering correlate with the satisfaction judged by NeuS-V or the VLM judges across videos?
  • Q5 [C/R] (W2). In the supplementary videos, objects sometimes appear abruptly to realize an event, or two objects are blended. For example, in the dough example, cut pieces appear while the woman is still rolling the dough. Could the authors discuss whether such artifacts come from the base model or from the steering, and report how often they occur with and without steering? The ordering examples are also shown for one backbone, and additional examples across backbones and classes, including failure cases, would help.
  • Q6 [C/R] (W2). For negation, the steering uses a larger negative bias, with a bound of 6 instead of 3 on Wan2.1-14B (Table 6), to suppress an event named in the prompt. Could such strong suppression also remove related content or make the generated scenes less varied, for example by removing similar objects along with the excluded one? Since three seeds per prompt are already generated, a diversity measure across seeds on TLVB-NEG, with and without steering, would help clarify this.
  • Q7 [C] (W2). In Table 3, the Spatial score of CogVideoX-5B decreases from 0.5920 to 0.5087 with steering, while it increases on the other backbones. Could the authors discuss this case?

Suggestions (not affecting the assessment)

  • The caption of Table 4a states that all three metrics are computed on TLVB-ORDER, while Sec. 5 states that StoryEval is evaluated on all 423 of its own prompts. Clarifying which prompt set is used for each column would help.
  • Sec. 3 and Sec. 4.3 are precise but difficult to follow for readers without a background in model checking. A small worked example, such as computing the satisfaction score SφS_\varphi for a two-event ordering over a few temporal positions, would make the construction much easier to understand.
7 comments
AuthorsOpenReview ↗

Official Comment by Authors

We thank the reviewer (SaGK) for their careful observations and thoughtful feedback. The reviewer acknowledged T-LESSA’s thorough experimental evaluation and consistent improvements, as well as its principled, general, and practical approach to temporal logic guidance. Below, we address the weaknesses and questions raised by the reviewer.

(W1-Q1) Human Evaluation on a subset of TLVBench. We have requested IRB review for this evaluation, which is currently under review. Subject to approval, we hope to share the results during the review period. Otherwise, we plan to include them in the camera-ready version once the study is completed.

(W1-Q2) Text-to-TL Reliability. Please see the comment.

(W1-Q3) Temporal Evaluation (TC-Score, TCR, and StoryEval) between T-LESSA and SwitchCraft. We thank the reviewer for suggesting this comparison. Our original experiment evaluated temporal ordering against unsteered generation, using TC-Score, TCR, and StoryEval to complement NeuS-V. We have completed the additional TC-Score and TCR comparisons using existing videos. StoryEval requires new generation on 423 prompts. In our setup, SwitchCraft takes approximately 30.5 minutes per video on an A100 80GB, giving an estimated 215 GPU-hours for one complete run before evaluation. Hence, completing all additional StoryEval comparisons is not feasible within the rebuttal period, as other researchers are also using the shared cluster for their rebuttal experiments.

However, TC-Score and TCR can use the TLVB-ORDER videos already generated for Tab. 1. Hence, we show comparisons with SwitchCraft on Wan2.1-14B and Video-MSG on CogVideoX-5B:

BackboneMethodTC-Score ↑TCR ↑
Wan2.1-14BUnsteered.7025.3567
SwitchCraft.7750.4133
T-LESSA.7725.4633
CogVideoX-5BUnsteered.4563.1150
Video-MSG.4538.1050
T-LESSA.5400.1850

On Wan, T-LESSA achieves a TC-Score close to SwitchCraft's and a 12.1% higher TCR. On CogVideoX, T-LESSA exceeds Video-MSG on both metrics, with a 76.2% higher TCR. These results provide additional evidence of temporal completion beyond comparisons with unsteered T2V models.

AuthorsOpenReview ↗

Official Comment by Authors

(W2-Q4) Independence Assumption. As stated in Sec. 5.3, independence is an assumption used to compute the attention-based satisfaction scores. The automaton still encodes the temporal and logical relations between events. This assumption can affect both the estimated satisfaction and the gradient used to steer attention.

As requested, we examined whether the final internal score correlates with satisfaction as judged by NeuS-V or the VLM judges across 2,088 TLVBench videos for which we could retrieve internal scores from the logs. We could not find the saved internal scores for Wan ORDER, so those videos were excluded. The correlations are weak and mixed, with Pearson coefficients ranging from −.190 to +.174.

Our paired comparisons in Tab. 1 measure whether guidance improves satisfaction for each prompt. However, the correlation analysis is essentially asking whether higher internal scores identify videos with higher external satisfaction scores across different prompts. We compute internal scores from attention, not from the generated videos. Hence, guidance can improve satisfaction on average without the final internal score consistently identifying videos with higher external satisfaction.

(W2-Q5) Abrupt appearances and blended objects. We appreciate the reviewer's thoughtful observation and acknowledge that objects sometimes appear abruptly to realize an event, or two objects are blended. We evaluated all TLVB-ORDER and TLVB-CONJ videos from Wan2.1-14B and CogVideoX-5B in Tab. 1 using Qwen3.8-27B (suggested by the reviewer, C821) and 16 frames per video. We asked about unexplained object appearances and fused objects, flagging videos when P(yes) > .5. The table reports flagged videos which we verified it:

BackboneMethodAbrupt appearances (ORDER) ↓Fused objects (CONJ) ↓
Wan2.1-14BUnsteered15/3004/300
T-LESSA (Ours)10/3003/300
CogVideoX-5BUnsteered5/30010/300
T-LESSA (Ours)5/3008/300

Note that the number of flagged video from our method are lower or unchanged. We believe these artifacts partly reflect limitations of the T2V backbones, rather than being unique to our guidance. We provide additional examples across backbones and classes, including failure cases in the supplementary (./rebuttal/incorrect_ord_conj_examples)

(W2-Q6) Negation trade-off Our VLM judges score the required scene event separately from the excluded object. While joint NEG scores improve by 16.5–27.7% on Wan2.1-14B, required-event scores decrease by 2.6% under Qwen and 0.2% under InternVL3. In addition, as suggested, we evaluated all 100 TLVB-NEG prompts with three seeds per prompt using frame-averaged LPIPS. We compared all three seed pairs at eight uniformly sampled frame positions. On Wan, LPIPS decreases from .6269 to .6152, a 1.86% reduction. Thus, T-LESSA improves joint NEG satisfaction with a minor 1.86% reduction in across-seed LPIPS diversity on Wan2.1-14B. We will report this trade-off alongside the satisfaction gains.

(W2-Q7) Spatial score on CogVideoX-5B. Our TL objective uses spatially averaged attention and does not directly constrain object positions. To understand this decrease, we re-ran VBench’s detector and reproduced all 2,688 official frame scores. Zero-scored frames with an undetected named object increased from 31.6% to 39.8%. However, the average score also decreased from .939 to .890 among frames where both objects were detected. We acknowledge this spatial-alignment trade-off. Our hypothesis is that CogVideoX’s joint text–video attention may contribute to this backbone difference.

Reviewer_SaGKOpenReview ↗

Follow-up on W1: Evaluation and comparison with baselines

I sincerely thank the authors for their thorough and thoughtful responses. I greatly appreciate the considerable effort that went into the additional experiments, the re-evaluation with a stronger judge, and the supplementary pages that make the audit and the flagged examples easy to inspect. Below are my follow-up comments, organized by the weaknesses and questions in my original review.

(W1-Q1) Human evaluation.

Thank you for taking this request seriously and for initiating the IRB process. I understand that a formal human study requires IRB approval and may not be completed within the discussion period. Since the main results currently rely on automatic judges, I would appreciate it if the human study could be included in the final version.

(W1-Q2) Text-to-TL reliability.

Thank you for the careful audit and for clarifying that the main results use specifications from templates or official annotations. The comparison between two LLMs and the per-category breakdown are very helpful. However, after going through the audit page, I noticed that some labels seem inconsistent, particularly with respect to ordering.

  • U056: "A hand puts a coin on a button in a box, presses the button, and then a toy cat pops its head out of the box, and uses its paw to take the coin back into the box."

    • Terra (incorrect): FIRST_ONSET(presses the button < toy cat pops its head out)
    • Sol (correct): FIRST_ONSET(Hand puts a coin on the button in the box and presses the button < Toy cat pops its head out and uses its paw to take the coin into the box)

    Terra drops the first and last events, while Sol merges the four events into two. Both lose part of the ordering, but only Terra is marked incorrect.

  • U035: "A gorilla beats its chest, sits down, and starts to eat."

    • Terra (correct): FIRST_ONSET(gorilla beats its chest < gorilla sits down < gorilla starts eating)
    • Sol (correct): EVENTUALLY(A gorilla beats its chest) & EVENTUALLY(The gorilla sits down) & EVENTUALLY(The gorilla starts to eat)

    The first requires the three events to occur in order, while the second is satisfied by any order, yet both are marked correct. Since App. B.1 instructs the agent not to infer an order from sentence order, it is unclear which of the two the criteria consider correct.

The authors conclude that the agent translates prompts quite accurately. Since this conclusion is based on the correct/incorrect labels, could the authors clarify who assigned them and by which criteria?

In addition, my understanding from the response is that the agent was not used for T2V-CompBench and StoryEval in the experiments, and that the specifications for all 800 and 423 prompts, respectively, were derived from the official annotations. Could the authors describe how this was done, for example manually or by rule?

(W1-Q3) Comparison with guidance baselines.

Thank you for running the TC-Score and TCR comparison with SwitchCraft and Video-MSG within the limited rebuttal period.

I find this comparison particularly important, since ordering appears to be where the automaton contributes the most. For presence, absence, conjunction, and co-occurrence, the objectives reduce to closed-form products over frames (App. C.1 and C.2), which seems conceptually similar to existing attention objectives (e.g., Attend-and-Excite [Chefer et al., 2023] and TempoControl [Schiber et al., 2025]). For ordering, the automaton tracks which events have already occurred, so this is where the TL formulation can most clearly distinguish itself from methods such as SwitchCraft.

However, the original submission compared the sequence-completion metrics only against the unsteered backbones (Table 4a), and the new comparison does not show a consistent advantage over SwitchCraft. Compared with SwitchCraft on Wan2.1-14B, T-LESSA scores lower under the Qwen judge, is roughly on par under InternVL3, NeuS-V, and TC-Score, and is higher on TCR. A StoryEval comparison would therefore help clarify whether the TL objective offers an advantage over a method designed specifically for ordering. I understand the cost of a full run, but even a comparison on a subset of StoryEval prompts would be informative.

As a minor question, the unsteered and T-LESSA rows differ slightly from Table 4a. Is this because all three seeds are used here?

Reviewer_w5d1OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes training-free guidance for video generation using temporal logic. An LLM converts the prompt into a specification. The method uses cross-attention as soft event scores, computes a differentiable satisfaction score, and updates attention biases during denoising. Experiments cover four backbones, a new TLVBench and external benchmarks. The results show improved alignment with relatively small additional generation time.

Critical strengths
  1. The method turns temporal-logic specifications into a differentiable guidance objective. It provides an explicit way to express event order, joint presence and exclusion, which is a useful design for controllable video generation.

  2. The controller works through attention biases without retraining the backbone. Experiments on several backbones and the latent-guidance study explore whether the objective is useful beyond one implementation. This supports the practical reuse of the approach.

Critical weaknesses
  1. The Text-to-TL component is not sufficiently validated. Section 5 relies on it for benchmarks without provided specifications, while the limitations acknowledge that translation reliability was not measured. Missing events or incorrect temporal and negation relations could affect whether the method follows the original prompt, even when it follows the generated specification.
Questions for authors
  1. Can you manually check a small sample of Text-to-TL outputs and report the main errors? For incorrect translations, a comparison using the original and corrected specifications would help show their effect on the final videos.
2 comments
AuthorsOpenReview ↗

Official Comment by Authors

We thank the reviewer (w5d1) for their careful observations and thoughtful feedback. The reviewer acknowledged T-LESSA’s explicit, differentiable formulation of temporal logic requirements and its practical reuse through training-free attention control across multiple backbones. Below, we address the weaknesses and questions raised by the reviewer.

(W1) Text-to-TL Reliability. Please see the comment.

Thank you for clarifying how the specifications were obtained. Knowing that TLVBench uses templates and that the StoryEval and T2V-CompBench specifications come from official annotations substantially addresses my concern about those results. Please make this distinction clear in Section 5.

The manual audit also answers my question about translation errors. The translator used in the paper was correct on 37 of 50 VBench prompts, but on none of the nine temporal-style prompts. This limits my confidence in its use with free-form prompts. The effect of correcting these errors on the generated videos remains unclear.

I maintain my rating of 3 (Weak acceptance).

Reviewer_C821OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces T-LESSA, a training-free method that translates text prompts into temporal-logic specifications and uses differentiable satisfaction scores to steer video diffusion attention, improving event ordering and adherence to logical constraints.

Critical strengths

The idea is interesting.

Critical weaknesses
  1. The choice of Qwen2.5-VL-7B as an evaluator requires stronger justification, particularly given the availability of newer models such as Qwen3.8. Its susceptibility to hallucinations raises concerns about the reliability of the reported evaluation results. Validation with a stronger evaluator or human judgments would help establish confidence in the findings.

  2. The practical relevance and generalizability of the method remain unclear. Modern video generators, including Seedance, Wan3.0, and the open-source MiniMax H3, appear to handle temporal ordering substantially more reliably. The authors should therefore demonstrate that the targeted failure modes remain prevalent in current models and that their approach provides benefits beyond those achievable through improvements in training data and compute. In comparison, spatial distortions in generated motion seem to be a more pressing research problem. The proposed method also introduces repeated feedback-driven adjustments, whose practical benefit should be assessed against the simpler alternative of regenerating a video when a rare failure occurs. Its compatibility with unified generation-and-editing models is another concern: modifying internal attention could potentially affect other tasks, and such interactions require investigation.

  3. The complexity of the approach appears disproportionate to the demonstrated gains. In particular, the reliability of the natural-language-to-logic conversion is insufficiently established. For a complex action sequence, such as a martial arts routine, it is unclear whether the method can reliably distinguish and encode the precise order of individual movements. Given these unresolved issues and the relatively modest improvements demonstrated on weaker base models, the current evidence does not convincingly establish the method’s practical value.

Questions for authors

See weakness

2 comments
AuthorsOpenReview ↗

Official Comment by Authors

We thank the reviewer (C821) for their careful observations and thoughtful feedback. The reviewer acknowledged the idea is interesting.

(W1) Stronger VLM evaluator

We agree that the choice of old models such as Qwen2.5-VL-7B is a limitation. We note that Qwen3.8-27B was released on August 14, 2026, after our internal paper deadline. As suggested, we re-evaluated 4,776 existing videos from Wan2.1-14B and CogVideoX-5B with Qwen3.8-27B, keeping the evaluation script and setup unchanged except for the evaluator model. T-LESSA improves the mean score in all eight comparisons:

BackboneMethodNEG ↑CONJ ↑ORDER ↑TEMPLOG ↑
Wan2.1-14BUnsteered.569.686.579.363
T-LESSA (Ours).641.742.655.497
CogVideoX-5BUnsteered.565.146.176.171
T-LESSA (Ours).667.235.306.334
AuthorsOpenReview ↗

Official Comment by Authors

(W2/3) Practical Relevance, Efficiency, and Scope of our work

We respectfully disagree that improvements in the state-of-the-art (SOTA) video generative model remove the need for explicit temporal control. We clarify the practical contribution, computational cost, and more below:

1. Practical Relevance. We believe that retraining, increasing model size, and inference-time guidance involve different trade-offs in training cost, inference cost, and control over specific requirements. Our method is about how to improve compliance with these requirements in a frozen video generative model without retraining. Hence, although we agree that our method's benefit may be achievable through improvements in training data and compute, our method avoids the cost of collecting additional training data and retraining the video generative model.

2. Temporal Ordering. We agree that recent SOTA video generative models have improved in handling temporal ordering. However, these improvements do not by themselves establish reliable compliance with explicit temporal constraints. We introduce our method as a complementary approach that combines neural video generation with symbolic temporal specifications to improve temporal ordering.

3. Repeated Feedback and Regeneration. Our method updates the attention bias during denoising without additional forward passes or backpropagation. We report that this increases generation time by 1.7% on Wan and 13.5% on CogVideoX. We agree that regenerating a failed video is another option. However, regeneration retries the same prompt (prompt-and-pray), whereas our method uses explicit temporal and logical requirements to guide the generation process itself.

4. Comparison to Video Editing Models. Our study addresses inference-time generation control rather than video editing. Our contributions are 1) formulating a differentiable TL score and 2) using this score to steer a frozen diffusion-based T2V model at inference time. We ask how to guide a frozen T2V model toward specified rules or requirements during generation. These research directions are complementary, and progress in video editing does not diminish the value of inference-time generation control.

5. Complex Action Sequences. Since our method complements the T2V model, its effectiveness still depends on the backbone's ability to represent and generate the required actions. A visually plausible video can still omit an event, reverse the requested order, or include forbidden content. Our experiments measure these failures explicitly. We do not claim precise control of complex martial-arts routines. Our experiments establish improvements for the event types and specifications evaluated in the paper.

6. Text-to-TL Reliability. Please see the comment.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Text-to-TL Validation & Reliability

We agree with the concerns raised by all three reviewers about the reliability of our Text-to-TL component.

First, we design the Text-to-TL component to convert text to TL without guided specifications for practicability, converting any prompts to associated TL specifications. However, to provide accurate experiment results, we use a template for TLVBench: each prompt is derived from its specification's event phrases by a fixed template. Therefore, all 400 released prompts match their template exactly. Second, for benchmarks such as T2V-CompBench and StoryEval, the specifications are derived from official annotations without changing their original prompts. Third, the only benchmark for which we use the agent to translate text to TL is VBench. Section 5 incorrectly notes the T2V-CompBench and StoryEval specifications to the agent, and we will detail this in the experimental setup.

As requested, we manually check a small sample of Text-to-TL outputs. We compare the agent's output using two LLM models, GPT-5.6 Terra, which we use in the paper, and GPT-6.1 Sol. The translator uses zero-shot prompting. All data and details have been added to the supplementary material (./rebuttal/text2tl_audit.html).

VBench: We sampled 50 of the 517 VBench prompts and got accuracies of 74.00% (Terra) and 92.00% (Sol). However, the VBench sample alone is insufficient for auditing temporal ordering and negation. Therefore, we ran the agent to translate sampled prompts from StoryEval for temporal ordering and TLVBench for logical relationships. The agent with Terra performs poorly on the Temporal style dimension because Terra tends to keep only the style phrase ("pan left"):

DimensionGPT-5.6 TerraGPT-6.1 Sol
Temporal style0/95/9
Spatial relationship7/88/8
Multiple objects13/1313/13
Background7/77/7
Motion & subject5/66/6
Imaging & overall5/77/7
All37/5046/50

StoryEval: We sampled 76/423 and got 80.26% (Terra) and 84.21% (Sol):

CategoriesGPT-5.6 TerraGPT-6.1 Sol
Matched the official events26/3027/30
First event(s) missing14/2015/20
Comma-only list15/1616/16
Other event missing5/96/9
Event split in two1/10/1
All61/7664/76

TLVBench: We sampled 20 of the 100 prompts in each of the Negation and Conjunction categories, and both Terra and Sol achieved 100.00% accuracy on both. In conclusion, we observe that if events and objects are clearly stated in the prompt, the agent can translate the prompt into the TL spec quite accurately.

AuthorsOpenReview ↗

Additional Clarification

Please see the additional clarification provided in response to reviewer SaGK's request.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.