acceptodds
Under review as a conference paper at ICLR 2027

ChronoGuide: Training-Free Temporal Control for Video Generation via Cross-State Attention

Abstract

Despite major gains in visual quality, text-to-video models remain unreliable at precise temporal control. Objects may appear early or late, persist too long, or fail to return on time, and actions may peak off cue. We identify a structural source of these failures in spatiotemporal self-attention. The same cross-frame communication that supports coherence also transfers features between frames requiring different states, such as presence and absence, pulling them away from the requested schedule. ChronoGuide addresses this conflict with a training-free cross-state attention mechanism. Its core operator preserves live attention weights and same-state contributions, replacing only values crossing state boundaries with matched features computed under the receiving frame's requested state. This targets cross-state interference while retaining same-state communication, without training or iterative latent optimization. We further introduce a mixed-lifecycle object evaluation covering entry, exit, bounded presence, and return, and a complementary evaluation using an open-vocabulary detector. Across all benchmark panels evaluated against TempoControl, ChronoGuide improves every reported metric. On the standard 80-prompt one-object Wan benchmark, it reaches 90.06% temporal accuracy, 83.88% presence accuracy, and 96.25% absence accuracy versus TempoControl's 83.56%, 79.75%, and 87.38%. It also uses substantially less peak VRAM than TempoControl, enabling deployment on lower-memory GPUs and batched inference. ChronoGuide transfers across continuous and discrete video diffusion models, including CogVideoX and URSA.

Primary area:
generative models
Keywords:
Video Generation, Temporal Control, Diffusion Models, Flow Matching, Attention, Controllable Generation, Inference Guidance

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_GwNMOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ChronoGuide to control when objects appear, disappear, and return in generated videos. The method preserves attention connections between frames but replaces features transmitted across different requested states with features that match the receiving frame’s requirements. This helps reduce premature appearances and delayed disappearances. The method requires no retraining and is evaluated through object timing tasks, mixed-lifecycle scenarios, ablation studies, and human evaluation.

Critical strengths
  1. Controlling when objects appear and leave is a useful capability for video generation. On the mixed-lifecycle benchmark, temporal accuracy improves from 66.37% with TempoControl to 82.44% with ChronoGuide. Accuracy during intervals when the object should be present also improves, indicating that the method does not achieve better absence scores simply by suppressing the object throughout the video. These results support the method’s effectiveness in controlling the timing of object visibility.

  2. The authors compare configurations with and without feature exchange while retaining the same text masks. They also evaluate alternatives such as blocking cross-state connections, using features conditioned on the wrong state, and globally replacing values. These experiments support the proposed exchange rule rather than merely showing that adding a control module improves the scores.

Critical weaknesses
  1. The demos provide a good illustration of the method’s effectiveness: objects generally appear or disappear at the requested times. However, some videos exhibit artifacts or deformations, which may be related to the training-free nature of the method.
  2. “Being detectable at sampled time points” and “remaining consistently visible throughout the video” are not the same thing. For example, an object that should remain visible after appearing might briefly disappear and reappear, or unexpected duplicate instances might emerge. Such behavior would affect the quality of temporal control. That said, these issues seem largely absent from the videos I watched, so I am unsure whether such failures occur.
Questions for authors

A training-free approach offers a relatively low-cost solution, but incorporating training might allow better adaptation and improve the results. Could the proposed method be developed into an integrated control component within a video generation model, with training used to improve controllability and generation quality?

3 comments
AuthorsOpenReview ↗

Response to Reviewer GwNM (1/3): Artifacts and Visibility

Thank you for the review, and for finding timing control "a useful capability for video generation" and noting that the method "does not achieve better absence scores simply by suppressing the object". TA = temporal accuracy (YOLO primary); brackets are 95% intervals; native = the unguided host; we name the GPU with each result. Example videos for all panels are in the supplementary gallery (website/index.html).

W1. "… some videos exhibit artifacts or deformations, which may be related to the training-free nature of the method."

We agree that some videos show artifacts. A new blinded audit finds no duplicate or severe deformation in ChronoGuide's clips, and average MUSIQ is above the other training-free methods and each host's native and TempoControl rows:

  • Blinded multi-rater visual audit (a fixed random subset of 24 prompts × 3 methods, B200): no rater marked a duplicate instance or a severe deformation in any of the 24 ChronoGuide clips.
  • Image quality (MUSIQ) against other training-free methods (same scorer and initial noise): ChronoGuide is higher on average, +3.05 [1.95, 4.14] over five TempoControl seeds (B200; interval over prompts), +2.87 [1.1, 4.7] over SwitchCraft on one-object (B200) and +2.30 [0.56, 4.16] over our Wan2.1 port of TS-Attn on mixed lifecycles (RTX 4090).
  • Image quality in Table 1: on every host and panel, ChronoGuide's MUSIQ is above that host's native and TempoControl rows (TempoControl's published numbers; URSA is our run), e.g. movement 66.97 vs 60.46 and 63.24.

W2. "'Being detectable at sampled time points' and 'remaining consistently visible throughout the video' are not the same thing."

We agree, so we scored all 81 frames of ChronoGuide, TempoControl and the unguided host (B200, same initial noise). The sampled score tracks every-frame visibility closely:

  • Every frame: agreement with the schedule is 90.35% for ChronoGuide on one-object (TempoControl 81.60%, host 74.66%) and 82.64% on mixed lifecycles (65.74%, 56.73%), within one point of the sampled TA for every method.
  • Disappear-and-return is rare: a gap of at least 0.25 s inside a requested-present interval occurs in 1/80 one-object and 4/80 mixed ChronoGuide videos. On mixed lifecycles the main remaining failure, persistence after a requested exit, is less than half as common as for TempoControl:
Failures over all 81 framesOne-object CG / TempoControl / nativeMixed CG / TempoControl / native
Longest run of missed frames while the object should be present, incl. late entry ↓ (frames, mean per video)4.06 / 6.31 / 8.906.71 / 8.96 / 10.03
Object still detected ≥ 0.25 s after its requested exit ↓ (% of videos)— (no requested exit)30.00 / 62.50 / 76.25%
  • Most gaps last 1–3 frames. Where OWLv2 was also run on a missed frame, it found the object in 35 of 39 ChronoGuide gaps, consistent with detector misses. All ChronoGuide gaps cover 1.3% (one-object) and 2.6% (mixed) of requested-present frames.
  • The 21-point sampled metric registers every such failure of 0.25 s or more: we injected each failure type into perfectly timed detections and scored them with the released evaluator. Any dropout or false detection of 0.25 s or more lowers our 21-point score, while the 20-point one-object protocol can miss one. Filling every ChronoGuide gap would raise presence by only 1.00 (one-object) and 2.67 (mixed) points.
  • Duplicates never change the score (the metric asks only whether the object is detected), so we count them: two target boxes for at least 0.5 s in 3/80 one-object and 2/80 mixed ChronoGuide videos.
  • The revision will add these measures, the injected-failure analysis and the success/failure figure, and state the every-frame result in the main text.
AuthorsOpenReview ↗

Response to Reviewer GwNM (2/3): Training ChronoGuide into an Integrated Component

Q. "Could the proposed method be developed into an integrated control component within a video generation model, with training used to improve controllability and generation quality?"

We think so, and we tested two routes on Wan2.1-1.3B: (1) distilling the exchange into small adapters, which could also remove the auxiliary branches, and (2) fine-tuning on ChronoGuide's own videos. Thanks for the question; it led to one of the main new experiments in our response. Targets come from generated videos and ChronoGuide's own predictions, so no human-labelled or external video is needed. Further routes are quality-filtered targets, which could also reduce artifacts at transitions, and fine-tuning with a temporal detector reward, guarded against rewarding a suppressed object. Training gives up one property: today ChronoGuide needs no training on a new host, only an interface to its attention layers.

We trained on generated prompts over 120 non-COCO objects that do not occur in the test set (six held out). Each variant and its endpoints were frozen before its run. Later variants were informed by earlier validation diagnostics, and the test results of all three variants are reported. Our baseline throughout is text permissions: ChronoGuide's frame-wise text masks without the exchange, so each frame attends to the object's words only when the object is requested there. The three students differ in what they learn and where: the first is fine-tuned on finished videos labelled with the schedule they show, while the other two copy ChronoGuide's exchange directly, one with a LoRA across all attention layers and one with small adapters only where the exchange acts.

  • LoRA on ChronoGuide's videos (route 2): rank-32 LoRA on the attention projections of all 30 blocks, trained with Wan's flow-matching loss on ChronoGuide and unguided videos, each labelled with the schedule it actually shows (by OWLv2, an open-vocabulary detector).
  • Exchange distillation: the same LoRA trained to reproduce ChronoGuide's exchange-corrected prediction in the first 10 of 50 denoising steps, where its correction is largest.
  • Value adapters: separate present-state and absent-state adapters on the self-attention value and output projections (the exchange acts only on self-attention values), also trained to reproduce the exchange.

The students learn part of the exchange. We first checked this in latent space, before decoding any video. On 20 development prompts (L40S), over the first 10 of 50 denoising steps, the two distillation students (exchange distillation and the value adapters) stay closer to ChronoGuide's latents than text permissions alone: their squared error to ChronoGuide is 36.2% [20.6%, 51.2%] and 39.5% [27.0%, 53.3%] lower. On held-out objects, the value adapters reproduce part of ChronoGuide's one-step correction: a fraction of 0.2254 [0.1772, 0.2646] pooled over samples, or 0.1682 [0.0820, 0.2507] averaged per prompt, where 1 would be an exact copy.

AuthorsOpenReview ↗

Response to Reviewer GwNM (3/3): Video Results of the Trained Students

Each student is one training run on one host, evaluated from one noise seed. All arms are scored on the full official one-object test set: all 80 prompts, none of whose objects appear in training. They start from the same seed-42 noise and are scored the same way (YOLO primary; RTX 4090). Each student runs the host's two passes per step, while ChronoGuide runs four.

The value adapters reach 87.00 TA, against 88.75 for ChronoGuide and 74.69 for native Wan, at 1.11× native denoising time instead of ChronoGuide's 2.81×. All three students gain 10.69–12.31 TA over native, and the two distillation students have higher average MUSIQ than ChronoGuide (descriptive). Against text permissions alone, two of the three are higher on average in TA (+1.06, +1.12), and all three on OWLv2 (+3.62 to +5.31, each per-variant interval above zero). On YOLO TA, the improvement is not established at the family-wise 95% level.

Full one-object test set (80 prompts)LoRA on ChronoGuide's videosExchange distillationValue adapters
Trainable parameters23.6M23.6M2.95M
Student TA ↑85.3886.9487.00
− native Wan (74.69), Bonferroni-adjusted within each variant's study+10.69 [5.54, 15.83]+12.25 [7.69, 17.12]+12.31 [7.87, 17.06]
− text permissions (85.87), per-variant 95% CI (unadjusted)−0.50 [−4.69, 3.63]+1.06 [−1.31, 3.50]+1.12 [−2.12, 4.50]
Presence / absence vs permissions−14.25 / +13.25−4.88 / +7.00−4.50 / +6.75
MUSIQ vs permissions−2.88−0.97−1.03
OWLv2 (secondary): − permissions, per-variant 95% CI (unadjusted)+5.31 [2.19, 8.38]+3.62 [1.44, 5.62]+4.25 [2.12, 6.50]
  • All three students are well ahead of native Wan, and against text permissions, all three trade presence for absence. On one-object, most of the gain over native comes from the frame-wise permissions, which are part of ChronoGuide. The exchange adds absence (+12.00 [7.98, 16.40] over permissions in this run) and carries the mixed-lifecycle gains.
  • Runtime: in separate RTX 4090 batch-1 measurements, the LoRA on ChronoGuide's videos, exchange distillation and the value adapters took 1.14×, 1.11× and 1.11× their native Wan denoising-loop times. ChronoGuide took 2.81× native in the same timing job as that LoRA.
  • Image quality (descriptive, added after the results): the two distillation students have higher average MUSIQ than ChronoGuide (+1.18 [0.36, 2.01] and +1.13 [0.05, 2.13]). Their absence control is weaker: sustained early detections occurred in 36.25% and 38.75% of their videos, against 21.25% with ChronoGuide and 56.25% with native Wan.
  • Matched teachers: the two distillation students copy ChronoGuide's exchange in the first 10 steps only. That 10-step teacher's own gain over text permissions is +3.00 TA, with an interval that includes zero ([−0.69, 6.50]). For the LoRA on ChronoGuide's videos, the pre-registered primary test, adding it to full ChronoGuide, gives +0.69 TA [−3.31, 4.62].
  • Fewer weights, close scores: the value adapters use one-eighth of exchange distillation's trainable weights (2.95M vs 23.6M), and their scores are within 0.63 points of it on every metric in the table (descriptive).

We hope these results answer your question about training and your concerns about visual quality, and we are happy to discuss any point further.

Reviewer_QXyoOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ChronoGuide, a training-free method for temporal control in text-to-video generation. It computes presence- and absence-conditioned value branches, and in spatiotemporal self-attention it replaces only the values exchanged between frames with different requested states, keeping the live attention weights and same-state values unchanged. The method is evaluated on TempoControl’s one-object, two-object, and movement benchmarks, plus a new mixed-lifecycle benchmark covering entry, exit, and return. Experiments on Wan 2.1, CogVideoX, and URSA report gains over TempoControl in YOLO/OWLv2-based temporal accuracy and lower peak GPU memory, supported by ablations and a human study.

Critical strengths
  1. Clean and simple mechanism with well-designed routing ablations. The core operator is easy to understand and clearly differs from optimization-based approaches such as TempoControl. The routing controls in Appendix D.5 (wrong-state donors, cross-state blocking, global replacement) each test a specific design choice, and together they provide convincing evidence that receiver-matched cross-state routing contributes to temporal adherence, particularly on the mixed-lifecycle benchmark. This matters to my assessment because it gives the method a plausible mechanism rather than only end-to-end gains.

  2. Extensive evaluation and practical efficiency. The paper evaluates on three hosts (continuous and discrete), adds a mixed-lifecycle benchmark covering entry, exit, and return, checks detector sensitivity with OWLv2, and reports multiple seeds. Peak GPU memory is also substantially lower than TempoControl (about 22 GiB vs. 115 GiB). This matters because it makes the approach practically usable on commodity GPUs and makes the reported trends more credible.

Critical weaknesses
  1. The two-object results rely heavily on an additional composition extension. The headline improvement depends on spatial priors, regional retrieval, same-state value replacement, and text amplification. As acknowledged in Appendix K, the complete extension no longer preserves live attention weights or same-state values. This makes it difficult to attribute the two-object improvement to the proposed core mechanism, weakening the paper's central claim.

  2. Insufficiently controlled baseline comparisons. TempoControl's accuracy and latency are taken from published results, while ChronoGuide is evaluated locally. The significance tests ignore the baseline's seed variance, and the latency comparison uses different experimental setups. Comparisons with other training-free methods are also missing. Although the paper includes component ablations, the contribution of cross-state exchange to the two-object results remains statistically uncertain. These limitations weaken the evidence for the method's claimed superiority.

Questions for authors
  1. Contribution of the core mechanism. Given that the TWO headline result depends heavily on the composition extension, how much of the improvement can be attributed specifically to cross-state exchange? Does the exchange gain remain significant on the full 82-prompt benchmark?

  2. Statistical validity. Can the authors evaluate TempoControl across multiple seeds under the same setup and report whether ChronoGuide's improvements remain statistically significant?

  3. Inference efficiency. Can the authors provide a matched latency comparison with TempoControl using identical hardware, software, and timing protocols?

  4. Baseline coverage. Can the authors compare against other training-free temporal-control methods, such as SwitchCraft and TS-Attn, under the same evaluation protocol?

5 comments
AuthorsOpenReview ↗

Response to Reviewer QXyo (1/3): Two-Object Attribution and Controlled Comparisons

Thank you for the careful review. We are glad you found the core operator "clean and simple" and the routing controls "convincing evidence that receiver-matched cross-state routing contributes to temporal adherence". TA = temporal accuracy (YOLO primary); brackets are 95% intervals; we name the GPU with each result.

W1. "The two-object results rely heavily on an additional composition extension. … This makes it difficult to attribute the two-object improvement to the proposed core mechanism, weakening the paper's central claim."

Removing the exchange gives the largest observed drop in two-object TA. Two controls on all 82 prompts, same noise:

  • Exchange corrections off, composition untouched (B200; Q1): TA falls by 17.02 [10.74, 23.40].
  • Leave-one-out of the components (separate L40S run; table below): removing the exchange gives the largest drop, 13.94 [7.55, 20.44], against 10.10 for the whole composition extension. This arm also removes regional retrieval, which feeds the exchange.
RemovedTA lost [95% CI]Most affected metricIts loss, pp [95% CI]
Cross-state exchange13.94 [7.55, 20.44]Absence30.37 [20.98, 40.24]
Composition extension as a whole10.10 [3.95, 16.61]Presence17.18 [7.32, 26.61]
Spatial text permissions7.03 [0.41, 13.65]Presence12.75 [2.99, 22.39]
Text amplification5.63 [1.51, 9.82]Absence10.00 [3.29, 17.20]
Regional retrieval4.47 [−0.81, 9.87]Presence10.75 [2.99, 18.74]
  • The exchange carries most of the timing, composition the presence: without the exchange, the second object appears before its requested entry. Composition keeps it visible once it should be there.
  • Text amplification supports absence: it boosts each object's words only where that object is requested. Without it, early detections of the second object nearly double (14.15% → 26.83% of absent bins).
  • These effects are conditional and not additive. The exchange was the pre-specified primary comparison. After multiple-comparison adjustment, the amplification and composition intervals still lie above zero.
  • The core operator keeps the live attention weights and same-state values exactly wherever it runs alone: one-object, mixed lifecycles and six other hosts. On three of them (80 one-object prompts, same initial noise, B200), with the exact test prompts, it raises TA by +19.50 [15.44, 23.62] on Wan2.1-14B, +9.06 [4.06, 14.12] on LTX-2 (19B) and +7.50 [2.81, 12.38] on LTX-2.5 (22B), with higher average MUSIQ on all three. Because composition changes attention to place objects, the revision will label two-object as core operator plus extension. On LTX-2 (one-object, same 80 prompts), where only the core runs, removing the exchange costs 4.56 TA [0.94, 8.06].

W2. "Insufficiently controlled baseline comparisons. TempoControl's accuracy and latency are taken from published results, while ChronoGuide is evaluated locally. … Comparisons with other training-free methods are also missing."

We ran the controls you asked for, and ChronoGuide stays ahead in each. We re-ran TempoControl over six seeds (Q2). ChronoGuide has higher TA on every seed, and +4.34 on average over seeds 43–47, seed variance included. On one B200 (Q3), it takes 12.08% less time and 89.63% less peak memory. Against SwitchCraft and TS-Attn under our protocol (Q4), it has higher TA on every panel and a higher average on 27 of the 28 comparisons. Table 1's TempoControl rows for Wan and CogVideoX are its published results. On one-object, TempoControl's published TA (83.56) is above the mean of our own re-runs over seeds 43–47 (82.19), so the published row is not a weak baseline. Mixed lifecycles is our new test set, so its TempoControl row (66.37 TA) is our run from ChronoGuide's initial noise.

AuthorsOpenReview ↗

Response to Reviewer QXyo (2/3): The Exchange on All 82 Prompts, Seeds and Latency

Q1. "Given that the TWO headline result depends heavily on the composition extension, how much of the improvement can be attributed specifically to cross-state exchange? Does the exchange gain remain significant on the full 82-prompt benchmark?"

Yes: removing only the exchange costs 17.02 TA [10.74, 23.40] (OWLv2: 14.81 [9.18, 20.62]), almost all through absence. As you asked, we switched it off on all 82 prompts, keeping composition and noise fixed.

Two-object, 82 prompts (YOLO, B200)TA ↑Presence ↑Absence ↑MUSIQ ↑
Full ChronoGuide67.7760.3175.9870.88
Composition on, exchange off50.7557.4343.4170.32
Difference [95% CI]+17.02 [10.74, 23.40]+2.88 [−4.88, 10.76]+32.56 [22.07, 43.05]+0.56 [−1.21, 2.45]
  • Against TempoControl's published two-object TA (53.17), composition without the exchange stays below it (50.75), while the core configuration without composition already exceeds it (57.26, in the paper).
  • The full arm reproduces Table 1 within 0.06 TA. On the paper's 40 development prompts the mean gain is smaller (+6.43 TA [−2.50, 16.55]), as absence without the exchange is easier there (60.25% vs 43.41% here).

Q2. "Can the authors evaluate TempoControl across multiple seeds under the same setup …?"

Yes: ChronoGuide leads in TA on all six seeds, and the TA gain's 95% interval stays above zero with TempoControl's seed variance included. As you asked, we re-ran TempoControl (its headline configuration) on B200 GPUs on the 80 one-object prompts for seeds 42–47, from ChronoGuide's initial noise. The test uses five further seeds (43–47). Seed 42, our headline run's stored noise, is reported separately and also favours ChronoGuide (+8.19 TA [4.19, 12.12]). Each 95% interval treats a different source of variation as random: the prompts, the seeds (your request), or both. Bold intervals exclude zero.

ChronoGuide − TempoControl, seeds 43–47MeanBootstrap over prompts (seeds fixed)Across seeds (paired t-test)Bootstrap over prompts and seeds
Temporal accuracy ↑+4.34[2.67, 5.96][0.61, 8.06], p = 0.032[0.84, 7.57]
Presence ↑+3.90[0.92, 6.68][0.78, 7.02][−0.58, 8.45]
Absence ↑+4.78[2.06, 7.46][−0.29, 9.84][0.14, 9.29]
MUSIQ ↑+3.05[1.95, 4.14][−3.64, 9.74][−1.37, 7.63]

Over seeds 43–47, mean TA is 86.53 for ChronoGuide and 82.19 for TempoControl (Table 1's 90.06 is the seed-42 run). ChronoGuide is ahead on every one of these seeds in TA, presence and absence, with lower seed-to-seed spread (SD 1.12 vs 2.45). The revision will add these means to the main text.

Q3. "… a matched latency comparison with TempoControl using identical hardware, software, and timing protocols?"

With each method's released defaults on one B200, ChronoGuide takes less time and far less memory (text encoding to decoding; two prompts, three timed runs each after a warm-up):

ChronoGuideTempoControl
Time per video ↓294.63 s335.11 s
Peak memory ↓10.83 GiB104.39 GiB
  • 89.63% less peak memory (TempoControl backpropagates through the model) and 12.08% less time.
  • The paper's 213 s and 21.94 GiB kept the model on the GPU with TF32 on (TempoControl: 114.91 GiB). Here both methods run at their released defaults, with the model offloaded and TF32 matrix multiplication off.
  • A faster implementation of the same computation (Wan2.1-1.3B, L40S; reply to kJbL, Q3) cuts ChronoGuide's time by 14.91–15.30% (TA 90.00 vs 90.56).
  • Guiding only the first 10 of 50 steps as well brings ChronoGuide to 1.18–1.20× the unguided host (L40S), with TA 90.50 (90.56 at full guidance).
AuthorsOpenReview ↗

Response to Reviewer QXyo (3/3): Other Training-Free Baselines

Q4. "… compare against other training-free temporal-control methods, such as SwitchCraft and TS-Attn, under the same evaluation protocol?"

ChronoGuide leads both in TA on every panel, including TS-Attn's own host. It is higher on average on 27 of the 28 comparisons, 23 of them with 95% intervals above zero, and no interval favours a baseline. We ran both under our protocol, with the same prompts, initial noise, scorer and GPU type in each pair. Differences are ChronoGuide − baseline over 80 paired prompts, in bold where ChronoGuide is higher on average.

Baseline, host / panel (GPU)ΔTA ↑ΔPresence ↑ΔAbsence ↑ΔMUSIQ ↑
SwitchCraft, Wan2.1-1.3B / one-object (B200)+6.44 [3.0, 10.0]+14.25 [7.7, 20.9]−1.38 [−4.1, 1.5]+2.87 [1.1, 4.7]
SwitchCraft, Wan2.1-1.3B / mixed (B200)+16.19 [11.61, 20.71]+4.23 [−2.13, 10.95]+29.92 [22.09, 37.87]+1.87 [−0.78, 4.64]
TS-Attn, Wan2.1-1.3B / one-object (RTX 4090)+13.12 [8.31, 17.81]+10.88 [2.38, 19.76]+15.38 [10.10, 20.62]+0.05 [−1.77, 2.06]
TS-Attn, Wan2.1-1.3B / mixed (RTX 4090)+25.12 [19.88, 30.18]+5.79 [1.08, 10.63]+47.31 [37.78, 56.92]+2.30 [0.56, 4.16]
TS-Attn, Wan2.2-A14B (its own host) / one-object (L40S)+21.50 [17.00, 25.81]+33.88 [25.49, 41.89]+9.12 [4.15, 14.56]+3.79 [−0.14, 7.64]
TS-Attn, Wan2.2-A14B (its own host) / mixed (L40S)+24.94 [19.88, 29.94]+16.48 [6.74, 26.63]+34.65 [27.26, 42.41]+4.77 [2.22, 7.42]
  • At SwitchCraft's own scale (Wan2.1-14B, one-object, B200): TA +14.94 [10.81, 19.00], presence +23.88 [16.13, 31.11], absence +6.00 [1.88, 10.51], MUSIQ +10.65 [7.98, 13.25].
  • Every TA gain has a 95% interval above zero. The gains are largest on mixed lifecycles, where both baselines often show an object after its requested exit.
  • SwitchCraft's one-object absence lead (1.38, interval including zero) comes with 14.25 points lower presence.
  • SwitchCraft (CVPR'26) runs its public Wan2.1 code. Its event phrases come from the same span annotations (one-object: the same span vocabulary), with the mapping fixed on development prompts.
  • TS-Attn (ICLR'26) releases code only for Wan2.2-A14B. We run that pipeline unchanged, with ChronoGuide on the same model at its 1.3B settings (better on 67 and 69 of 80 prompts). For the Wan2.1-1.3B setting its paper reports, we ported the code with minimal changes. Switched off, it matches stock Wan2.1, and switched on, it beats its own unguided output on one-object (+4.25 TA [1.06, 7.94]).

We hope these controls settle the attribution and baseline questions, and we are happy to discuss any point further.

Reviewer_kJbLOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper targets precise temporal control in text-to-video generation, such as an object appearing, exiting, or returning at a requested time, or an action peaking on cue. It argues that spatiotemporal self-attention transfers features between frames that should be in different states (present vs. absent), pulling them away from the schedule. ChronoGuide is training-free and works in three steps:

  1. It runs two auxiliary branches of the frozen model with text-permission masks: one where the object is “absent” and one where it is “present.”
  2. In self-attention, it keeps the live attention weights.
  3. It replaces only the values that cross a state boundary with the values from the branch matching the receiving frame’s requested state.

Optional additions are a Haar-band CFG rescaling, an action gain, and, for two objects, a composition extension with spatial permissions and regional retrieval. The paper also introduces a mixed-lifecycle benchmark (entry/exit/return) and adds OWLv2 to the YOLO-based evaluation. On Wan2.1-1.3B, one-object temporal accuracy (TA) rises from TempoControl’s 83.56 to 90.06, with large gains on mixed lifecycles (66.37 → 82.44). The method also transfers to CogVideoX and the discrete URSA model, and uses far less memory than TempoControl.

Critical strengths
  1. Effective training-free temporal control. ChronoGuide improves one-object temporal accuracy from 83.56% to 90.06% and two-object accuracy from 53.17% to 67.83% over TempoControl on Wan2.1. The improvements on mixed object lifecycles are particularly encouraging.
  2. Well-designed mechanism ablations. The comparisons against cross-state blocking, wrong-state donors, and global value replacement (Table 4 and Appendix D) provide useful evidence that receiver-matched value routing is preferable to simpler alternatives.
  3. Broad experimental evaluation. The paper evaluates three different video-generation architectures, includes five-seed robustness results for one-object control, and supplements detector-based metrics with a human study. The substantial reduction in peak memory relative to TempoControl is also practically valuable.
Critical weaknesses
  1. Limited support for general temporal control. The evaluated method primarily handles one time-varying concept. In the two-object setting, the second object remains present rather than following an independent schedule (Section 3.3). Therefore, the results do not establish whether the approach can handle multiple independently timed objects or overlapping events, which are important for compositional video generation.
  2. Temporal accuracy does not always reflect semantic correctness. Object timing is mainly measured through YOLOv10/OWLv2 detection, while action timing uses the second with maximum optical flow. The authors acknowledge that optical flow does not verify action identity, and the additional semantic evaluators were unreliable (Appendix I). Consequently, the reported movement improvements do not necessarily demonstrate that the requested action occurs at the correct time.
  3. Substantial inference overhead on some models. Although ChronoGuide is memory-efficient relative to TempoControl, it requires additional state-conditioned branches. Appendix G reports approximately 3× the native Wan latency, 5× for URSA, and 10× for CogVideoX. This limits the practical appeal of the method, particularly for applications requiring fast generation.
Questions for authors
  1. Can ChronoGuide support two or more objects with independently varying temporal schedules? What happens when their required presence intervals overlap or conflict?
  2. Can the authors provide human-annotated action-timing results to verify that improvements in optical-flow timing correspond to the requested actions rather than unrelated motion?
  3. What accounts for the large inference overhead on CogVideoX and URSA? Could the number of auxiliary branch computations be reduced without substantially sacrificing temporal accuracy?
  4. In the two-object setting, how much of the improvement comes specifically from cross-state value routing rather than the additional regional composition and text-amplification operations? The exchange-only ablation in Appendix D.6 shows a positive trend, but its confidence intervals include zero.
5 comments
AuthorsOpenReview ↗

Response to Reviewer kJbL (1/3): Scope, Metrics and Cost

Thank you for the detailed review and for noting the "well-designed mechanism ablations" and "broad experimental evaluation". TA = temporal accuracy (YOLO primary); brackets are 95% intervals; we name the GPU with each result.

W1. "Limited support for general temporal control. … the results do not establish whether the approach can handle multiple independently timed objects or overlapping events."

We extended the operator to independently timed objects and tested it (details in Q1). The submitted method controls one time-varying concept per video (in the two-object setting the second object is held present), and the revision will state this scope. Because the operator is written in terms of requested states, the extension keeps the same rule. Each frame now has a joint state (A and B each present or absent), and the exchange routes values between frames whose joint states differ, using the donor read that matches the receiver's joint state. Overlap is the state "both present", and conflicts are "A only" and "B only". At most four donor reads per step are needed. We tested it on 32 new object pairs, each object with its own schedule. Joint success (the share of time in which both objects are in their requested state) rises over the unguided host by +45.24 when the two schedules are disjoint, and by +29.61 when they overlap and the prompt says both appear together (Wan2.1-1.3B). On Wan2.2-A14B, the overlapping case gains +37.80.

W2. "Temporal accuracy does not always reflect semantic correctness. … the reported movement improvements do not necessarily demonstrate that the requested action occurs at the correct time."

Objects: a second detector, every-frame scoring and a blinded human audit agree with the sampled metric. OWLv2 agrees in direction with YOLO on the headline results. Scoring all 81 frames (B200) changes TA by less than one point for every method (reply to GwNM). In a blinded multi-rater visual audit (24 prompts × 3 methods, B200), agreement with the schedule is 94.6% (one-object) and 94.0% (mixed) for ChronoGuide, +10.5 pp [5.4, 15.9] above TempoControl (Krippendorff's α = 0.89).

Actions. We agree. A blinded human study of all 100 movement prompts now checks the action (Q2): ChronoGuide 57.3% vs 38.0% for TempoControl-11 (its 11-step setting) and 7.3% unguided.

W3. "Substantial inference overhead on some models. … This limits the practical appeal of the method, particularly for applications requiring fast generation."

Two changes cut the overhead on every host (Q3): a faster implementation of the same computation, and ending the intervention early in denoising. The paper's 10.41× (CogVideoX) and 5.03× (URSA) are single B200 measurements. As whole calls on one L40S, the released code costs 4.85–4.94× and 3.44–3.48×, respectively. The faster implementation alone brings these to 2.57–2.70× and 2.06–2.09×, keeps CogVideoX within 0.38 TA, and gives URSA 72.25 TA (71.94 released). With early release too, the three hosts run at 1.18–1.20× (Wan), 1.29–1.32× (CogVideoX) and 1.21× (URSA) unguided time. With both changes, Wan's TA is 90.50 (90.56 at full guidance). On CogVideoX and URSA, early release trades some TA for speed (Q3 gives every host's accuracy).

AuthorsOpenReview ↗

Response to Reviewer kJbL (2/3): Independently Timed Objects and Human Action Timing

Q1. "Can ChronoGuide support two or more objects with independently varying temporal schedules? What happens when their required presence intervals overlap or conflict?"

Yes, through the joint-state exchange (W1), and it helps most where the requests conflict. We ran a pilot on 32 new object pairs on Wan2.1-1.3B (L40S). Each object has its own schedule, in overlapping (A alone, both, B alone) and disjoint versions, and both arms share prompts and initial noise. The last row repeats the overlapping study, with the prompt saying both appear together, on Wan2.2-A14B (L40S).

Joint success ↑ (YOLO / OWLv2)Joint-state ChronoGuideUnguidedΔ, YOLO [95% CI]
Disjoint schedules72.02 / 80.95%26.79 / 25.89%+45.24 [34.08, 56.99]
Overlapping schedules48.96 / 52.53%28.12 / 33.18%+20.83 [14.14, 27.08]
Overlapping, prompt says both appear together64.43 / 71.43%34.82 / 37.50%+29.61 [21.43, 38.69]
Wan2.2-A14B: overlapping, prompt says both appear together76.79 / 85.12%38.99 / 44.49%+37.80 [27.98, 47.32]
  • Conflicting requirements show the largest gains: while A is present and B must be absent, both objects are in their requested states in 61–73% of bins (host 19–46%, across detectors and schedules). Once A has left and B is present, the figure is 62–86% (host 5–21%). On Wan2.2, ChronoGuide adds +58.33 and +56.70 points in these two phases, where the unguided host is right in only 25.52% and 11.16% of bins.
  • Overlapping schedules gain less on Wan2.1-1.3B: +20.83, against +45.24 when the schedules are disjoint. Overlap needs both objects visible at once, which the smaller host renders less readily. Two results are consistent with a limit of the host rather than of the method:
    • Unguided, the larger Wan2.2-A14B shows both objects together more often than Wan2.1 (+19.14 [0.78, 38.67]).
    • On Wan2.2-A14B, ChronoGuide raises joint success by +37.80, and both objects appear together more often than unguided on average (79.30% vs 73.44%, +5.86 [−8.98, 21.48]).
  • A second object that must stay present is placed by the paper's composition extension, which raises presence from 42.24 to 60.42 (82-prompt panel).

Q2. "Can the authors provide human-annotated action-timing results …?"

Yes: ChronoGuide leads on the paper's metrics and with human raters.

  • The paper's metrics (Table 1), against TempoControl's published result: the motion peak falls in the requested second in 61.00% of ChronoGuide videos, against 54.00% for TempoControl and 19.00% unguided, with higher MUSIQ (66.97 vs 63.24 and 60.46).
  • Human raters, against TempoControl's stronger movement setting: optical flow does not verify the action, so, as you asked, multiple blinded raters annotated all 100 movement prompts. We compare with TempoControl-11, the setting TempoControl ships for movement: 11 guided steps instead of 5, slower but stronger on motion timing. ChronoGuide is unchanged (both on a B200; unguided on an RTX 4090). Raters marked when the named action peaks, without seeing the method or requested time, using definitions fixed in advance. A clip without the action counts as a miss.
Human annotation, 100 movement promptsChronoGuideTempoControl-11Unguided
Action's strongest moment in the requested second ↑57.3%38.0%7.3%
ChronoGuide − method, paired by prompt [95% CI]+19.3 pp [9.3, 28.0]+50.0 pp [41.7, 58.3]

Raters agree on 78.44% of paired timing judgments (Krippendorff's α = 0.52). The revision will present movement as motion-window timing, with this human result.

AuthorsOpenReview ↗

Response to Reviewer kJbL (3/3): Inference Cost and Two-Object Attribution

Q3. "What accounts for the large inference overhead on CogVideoX and URSA? Could the number of auxiliary branch computations be reduced without substantially sacrificing temporal accuracy?"

On Wan, most of the added time is the two state-conditioned branches (the state reads). On CogVideoX and URSA, it is the released code's many small masked-attention calls. Both can be cut without retraining.

Timing. We timed whole CLI calls, including loading and saving, on one L40S for two development prompts. Each range, such as 632–637 s, gives the two prompts' values, here and in W3.

  • Wan: unguided Wan runs the transformer twice per step, ChronoGuide four times (adding the two state reads). A profile splits the extra 362 s per video into the state reads (61%), exchange corrections (23%), permission-masked text attention with an explicit softmax instead of Wan's fused kernel (14%), and bookkeeping (3%). The second and third are implementation choices.
  • CogVideoX and URSA: the number of attention calls, not passes. Because each frame can have its own text mask and requested state, the released code runs attention separately for each frame. That is tens of thousands of small, GPU-inefficient calls per video instead of about 1,500. On URSA, they account for at least 68% of the added time.

1. A faster implementation of the same computation: 14.91–15.30% less time on Wan, 45.32–46.98% on CogVideoX and 40.03–40.07% on URSA. The exchange corrections reuse the live queries and keys, so they are computed only over receiver × sender tokens and renormalized exactly (log-sum-exp). The permission-masked text attention uses a fused kernel with the same mask, and compatible frames share attention calls. Mean TA on the 80 test prompts is 90.00 vs 90.56 for the released code. The fused kernels only change the order of floating-point additions: on every sampled attention output, the difference from the released code is below 0.19% of the output's size (relative L2 error).

2. Releasing the intervention early (table: Wan2.1-1.3B). After the release step, the two state reads are skipped, removing 50% (first 25) or 80% (first 10) of the auxiliary computations:

Wan2.1-1.3B, steps with the interventionTime per video (L40S; × full guidance) ↓One-object TA ↑, 80 prompts (YOLO / OWLv2)
all 50632–637 s90.56 / 93.69
first 25452–455 s (0.71×)90.50 / 93.38
first 10345–347 s (0.54–0.55×)89.81 / 92.94

TA stays within one point (lower by 0.06 and 0.75), MUSIQ within 0.12, and absence is lower by 0.62 and 1.50. The temporal layout is mostly set in the early denoising steps, as the paper's development study shows.

3. Both changes, on all three hosts (multiples of unguided time; one L40S; released → optimized):

Steps with the intervention (time ↓)Wan2.1-1.3BCogVideoX-2BURSA
all 502.38–2.41× → 2.02–2.05×4.85–4.94× → 2.57–2.70×3.44–3.48× → 2.06–2.09×
first 251.65–1.69× → 1.48–1.52×2.94–2.96× → 1.77–1.80×2.23–2.25× → 1.52×
first 101.24–1.27× → 1.18–1.20×1.76–1.78× → 1.29–1.32×1.49× → 1.21×
  • Accuracy (80 prompts, YOLO TA, L40S). Wan, both changes: 90.19 (first 25), 90.50 (first 10). CogVideoX: the faster code stays within 0.38 TA of the released code at every setting. Early release costs 1.56 TA (first 25) and 5.50 (first 10). URSA: full guidance gives 71.94 (released) and 72.25 (faster code), and every early-release setting stays within 69.88–72.12.

Q4. "… how much of the improvement comes specifically from cross-state value routing …?"

Removing the exchange causes the largest drop of any component: 13.94 TA [7.55, 20.44], against 10.10 for the composition extension (leave-one-out on all 82 prompts, same noise, L40S). The loss is almost all absence: without the exchange, the second object appears too early. This arm also removes regional retrieval, which feeds the exchange. Removing only the exchange corrections (B200) lowers TA by 17.02 [10.74, 23.40].

We hope these results address your points on scope, evaluation and cost, and we are happy to discuss any of them further.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Revision Summary

Dear reviewers and AC,

We have uploaded the revision, with new and changed text in blue. Most of the blue is experiments added following the reviews and text we rewrote to make the paper easier to follow, not changes to ChronoGuide. In brief:

  • Exchange (QXyo, kJbL). On all 82 two-object prompts, TA falls by 17.02 points when the exchange is off and composition stays on (App. D.5). In the component study, removing it costs more TA than removing any other component (13.94 points, App. K.8).
  • TempoControl re-run (QXyo). We re-ran TempoControl on six seeds (42–47): ChronoGuide has higher TA at every seed (Fig. 10). Averaged over seeds 43–47, its mean leads on all four metrics: +4.34 TA, +3.90 Pres, +4.78 Abs and +3.05 MUSIQ (Table 3, with 95% intervals). With both released codes on one B200, ChronoGuide takes 12.08% less time and 89.63% less peak VRAM (App. G.2, Table 26).
  • Training-free baselines and larger hosts (QXyo). Under one protocol, ChronoGuide beats SwitchCraft and TS-Attn on TA in all seven comparisons, by 6.44–25.12 points, and on 27 of the 28 metric comparisons (Table 2, App. P). It raises one-object TA by 7.50–19.50 on three larger hosts (App. Q).
  • Cost (kJbL). With optimized code and guidance stopped after 10 of 50 steps, Wan runs at 1.18–1.20× its unguided time (L40S; App. G.3). On the 80 one-object test prompts, its TA stays within one point of full guidance (90.50 vs 90.56 with the released code). Stopping at step 10 also does not lower MUSIQ (57.00 vs 56.96, released code; App. E.3).
  • Actions and several objects (kJbL). Blinded raters place the action's peak in the requested window for 57.3% of ChronoGuide videos vs 38.0% for TempoControl (11-step setting; App. O.3). Two objects can be timed independently (+20.83–45.24 joint success on development pairs, App. K.9).
  • Visibility and training (GwNM). Scoring every frame changes TA by under one point (App. O). In a blinded visual audit, no rater marked a duplicate or severe deformation in any ChronoGuide clip (App. O.2). Small students trained from ChronoGuide raise TA by 10.69–12.31 points over native Wan (App. R).

Thank you for your time. If anything in the new material is unclear, please tell us in your thread and we will answer promptly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.