- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes ChronoGuide to control when objects appear, disappear, and return in generated videos. The method preserves attention connections between frames but replaces features transmitted across different requested states with features that match the receiving frame’s requirements. This helps reduce premature appearances and delayed disappearances. The method requires no retraining and is evaluated through object timing tasks, mixed-lifecycle scenarios, ablation studies, and human evaluation.
Critical strengths
-
Controlling when objects appear and leave is a useful capability for video generation. On the mixed-lifecycle benchmark, temporal accuracy improves from 66.37% with TempoControl to 82.44% with ChronoGuide. Accuracy during intervals when the object should be present also improves, indicating that the method does not achieve better absence scores simply by suppressing the object throughout the video. These results support the method’s effectiveness in controlling the timing of object visibility.
-
The authors compare configurations with and without feature exchange while retaining the same text masks. They also evaluate alternatives such as blocking cross-state connections, using features conditioned on the wrong state, and globally replacing values. These experiments support the proposed exchange rule rather than merely showing that adding a control module improves the scores.
Critical weaknesses
- The demos provide a good illustration of the method’s effectiveness: objects generally appear or disappear at the requested times. However, some videos exhibit artifacts or deformations, which may be related to the training-free nature of the method.
- “Being detectable at sampled time points” and “remaining consistently visible throughout the video” are not the same thing. For example, an object that should remain visible after appearing might briefly disappear and reappear, or unexpected duplicate instances might emerge. Such behavior would affect the quality of temporal control. That said, these issues seem largely absent from the videos I watched, so I am unsure whether such failures occur.
Questions for authors
A training-free approach offers a relatively low-cost solution, but incorporating training might allow better adaptation and improve the results. Could the proposed method be developed into an integrated control component within a video generation model, with training used to improve controllability and generation quality?
3 comments
Response to Reviewer GwNM (2/3): Training ChronoGuide into an Integrated Component
Q. "Could the proposed method be developed into an integrated control component within a video generation model, with training used to improve controllability and generation quality?"
We think so, and we tested two routes on Wan2.1-1.3B: (1) distilling the exchange into small adapters, which could also remove the auxiliary branches, and (2) fine-tuning on ChronoGuide's own videos. Thanks for the question; it led to one of the main new experiments in our response. Targets come from generated videos and ChronoGuide's own predictions, so no human-labelled or external video is needed. Further routes are quality-filtered targets, which could also reduce artifacts at transitions, and fine-tuning with a temporal detector reward, guarded against rewarding a suppressed object. Training gives up one property: today ChronoGuide needs no training on a new host, only an interface to its attention layers.
We trained on generated prompts over 120 non-COCO objects that do not occur in the test set (six held out). Each variant and its endpoints were frozen before its run. Later variants were informed by earlier validation diagnostics, and the test results of all three variants are reported. Our baseline throughout is text permissions: ChronoGuide's frame-wise text masks without the exchange, so each frame attends to the object's words only when the object is requested there. The three students differ in what they learn and where: the first is fine-tuned on finished videos labelled with the schedule they show, while the other two copy ChronoGuide's exchange directly, one with a LoRA across all attention layers and one with small adapters only where the exchange acts.
- LoRA on ChronoGuide's videos (route 2): rank-32 LoRA on the attention projections of all 30 blocks, trained with Wan's flow-matching loss on ChronoGuide and unguided videos, each labelled with the schedule it actually shows (by OWLv2, an open-vocabulary detector).
- Exchange distillation: the same LoRA trained to reproduce ChronoGuide's exchange-corrected prediction in the first 10 of 50 denoising steps, where its correction is largest.
- Value adapters: separate present-state and absent-state adapters on the self-attention value and output projections (the exchange acts only on self-attention values), also trained to reproduce the exchange.
The students learn part of the exchange. We first checked this in latent space, before decoding any video. On 20 development prompts (L40S), over the first 10 of 50 denoising steps, the two distillation students (exchange distillation and the value adapters) stay closer to ChronoGuide's latents than text permissions alone: their squared error to ChronoGuide is 36.2% [20.6%, 51.2%] and 39.5% [27.0%, 53.3%] lower. On held-out objects, the value adapters reproduce part of ChronoGuide's one-step correction: a fraction of 0.2254 [0.1772, 0.2646] pooled over samples, or 0.1682 [0.0820, 0.2507] averaged per prompt, where 1 would be an exact copy.
Response to Reviewer GwNM (3/3): Video Results of the Trained Students
Each student is one training run on one host, evaluated from one noise seed. All arms are scored on the full official one-object test set: all 80 prompts, none of whose objects appear in training. They start from the same seed-42 noise and are scored the same way (YOLO primary; RTX 4090). Each student runs the host's two passes per step, while ChronoGuide runs four.
The value adapters reach 87.00 TA, against 88.75 for ChronoGuide and 74.69 for native Wan, at 1.11× native denoising time instead of ChronoGuide's 2.81×. All three students gain 10.69–12.31 TA over native, and the two distillation students have higher average MUSIQ than ChronoGuide (descriptive). Against text permissions alone, two of the three are higher on average in TA (+1.06, +1.12), and all three on OWLv2 (+3.62 to +5.31, each per-variant interval above zero). On YOLO TA, the improvement is not established at the family-wise 95% level.
| Full one-object test set (80 prompts) | LoRA on ChronoGuide's videos | Exchange distillation | Value adapters |
|---|---|---|---|
| Trainable parameters | 23.6M | 23.6M | 2.95M |
| Student TA ↑ | 85.38 | 86.94 | 87.00 |
| − native Wan (74.69), Bonferroni-adjusted within each variant's study | +10.69 [5.54, 15.83] | +12.25 [7.69, 17.12] | +12.31 [7.87, 17.06] |
| − text permissions (85.87), per-variant 95% CI (unadjusted) | −0.50 [−4.69, 3.63] | +1.06 [−1.31, 3.50] | +1.12 [−2.12, 4.50] |
| Presence / absence vs permissions | −14.25 / +13.25 | −4.88 / +7.00 | −4.50 / +6.75 |
| MUSIQ vs permissions | −2.88 | −0.97 | −1.03 |
| OWLv2 (secondary): − permissions, per-variant 95% CI (unadjusted) | +5.31 [2.19, 8.38] | +3.62 [1.44, 5.62] | +4.25 [2.12, 6.50] |
- All three students are well ahead of native Wan, and against text permissions, all three trade presence for absence. On one-object, most of the gain over native comes from the frame-wise permissions, which are part of ChronoGuide. The exchange adds absence (+12.00 [7.98, 16.40] over permissions in this run) and carries the mixed-lifecycle gains.
- Runtime: in separate RTX 4090 batch-1 measurements, the LoRA on ChronoGuide's videos, exchange distillation and the value adapters took 1.14×, 1.11× and 1.11× their native Wan denoising-loop times. ChronoGuide took 2.81× native in the same timing job as that LoRA.
- Image quality (descriptive, added after the results): the two distillation students have higher average MUSIQ than ChronoGuide (+1.18 [0.36, 2.01] and +1.13 [0.05, 2.13]). Their absence control is weaker: sustained early detections occurred in 36.25% and 38.75% of their videos, against 21.25% with ChronoGuide and 56.25% with native Wan.
- Matched teachers: the two distillation students copy ChronoGuide's exchange in the first 10 steps only. That 10-step teacher's own gain over text permissions is +3.00 TA, with an interval that includes zero ([−0.69, 6.50]). For the LoRA on ChronoGuide's videos, the pre-registered primary test, adding it to full ChronoGuide, gives +0.69 TA [−3.31, 4.62].
- Fewer weights, close scores: the value adapters use one-eighth of exchange distillation's trainable weights (2.95M vs 23.6M), and their scores are within 0.63 points of it on every metric in the table (descriptive).
We hope these results answer your question about training and your concerns about visual quality, and we are happy to discuss any point further.
Response to Reviewer GwNM (1/3): Artifacts and Visibility
Thank you for the review, and for finding timing control "a useful capability for video generation" and noting that the method "does not achieve better absence scores simply by suppressing the object". TA = temporal accuracy (YOLO primary); brackets are 95% intervals; native = the unguided host; we name the GPU with each result. Example videos for all panels are in the supplementary gallery (
website/index.html).We agree that some videos show artifacts. A new blinded audit finds no duplicate or severe deformation in ChronoGuide's clips, and average MUSIQ is above the other training-free methods and each host's native and TempoControl rows:
We agree, so we scored all 81 frames of ChronoGuide, TempoControl and the unguided host (B200, same initial noise). The sampled score tracks every-frame visibility closely: