acceptodds
Withdrawn from ICLR 2027

CineDirector: Executable Shot Programs for Video Generation with Understanding in the Loop

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

A cinematic sequence must preserve a director's intent across deliberate visual discontinuities: each shot has a place in time, a camera path, and a cast that persists when the viewpoint changes. We introduce CineDirector, a framework that connects these requirements through an executable shot program. The program schedules transitions and binds camera, identity, and narrative conditions to individual shots, while a layered verifier checks generated footage against the same contract and revises failed shots locally. Director Distillation then learns from accepted revisions, carrying feedback into subsequent generation without an external verification loop. On 240 held-out programs, CineDirector records 0.04 latent frames of detected cut deviation, 1.62 degrees of camera rotation error, and 0.716 cross-shot identity similarity. Local revision reduces physical violations by 62% at 1.27 times the generator evaluations, and distillation retains a 45% reduction with external verification disabled. These results support a unified approach to controlling when a shot changes, what survives the change, and how execution improves through feedback.

Primary area:
generative models
Keywords:
multi-shot video generation, controllable video generation, verification, preference learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_FAksOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

A multi-shot video system: an MLLM compiles a shot program, a causal Wan2.1 with cache resets executes it, a layered verifier triggers local re-generation of failed shots, and accepted repairs are distilled with DPO. It is evaluated on 240 programs against multi-shot and camera-control baselines; the appendix adds VBench, a 14B transfer and a human study.

Critical strengths

The verify-then-commit design with loop-off evaluation is sensible, and the verifier calibration (App. D) looks careful. Thresholds are set on a separate human-labelled pool with per-layer precision and recall, and F.3 openly reports a reward-hacking case where distillation suppressed motion.

Critical weaknesses
  1. The three claims that organize the paper are each supposed to be settled by a control that App. E.8 says has not been run: the comparisons "require actual generation runs and are not assigned numerical outcomes in this appendix" (Supp. p. 11, l. 586–589). What is reported instead is accounting rather than measurement. G.8 computes the final violation rate from the initial incidence, the physics recall and two conditional repair rates that the paper itself calls a check of "arithmetic consistency", "not independent validation" (Supp. p. 18, l. 932–938), and the abstract's "62%" reduction (p. 1, l. 023) is exactly that computation. The same applies to the flagged-shot breakdown in F.2 (Supp. p. 13, l. 670–671), whose layer rates are hard to reconcile with the loop-off violation rate the same appendix feeds into the identity. Nothing in the appendix distinguishes numbers produced by generation runs from numbers produced by the ledger, so the headline reduction cannot be read as an empirical result, and the claims that repair preserves the original contract and that distillation adds something beyond more supervision rest on controls that exist only as a specification.
  2. Evaluation appears partly circular. REC counts a re-entry as successful when "the ArcFace similarity ... is at least 0.50" (App. E.5, Supp. p. 10, l. 514–515), the same model used in the verifier, and camera error relies on the same VGGT "re-estimation" (App. D.3, p. 8, l. 404). The independent check uses "a different judge model and a re-trained detector seed" (App. F.3, p. 13, l. 701). Baselines receive programs through "one fixed template per system" (App. E.1, p. 8, l. 420), so the CDE gap of 1.46 vs 0.04 (main Table 1, p. 7, l. 333–335) favours scheduled resets by design. As a result, the identity, camera and physics gains may partly reflect the loop optimizing its own evaluators. The cut metrics mainly confirm that the scheduled switch works, rather than showing better generation quality.
  3. The paper's central efficiency argument is that targeted local revision is cheaper than resampling whole videos, but no matched-budget comparison supports it. Table 3 (p. 8, l. 378–386) contrasts local revision with "Best-of-3", "Refine ×3" and a semantic-only variant, none of which is defined in Appendices E or F, and none of which is per-shot resampling under the same verifier at a comparable budget. Table 4(c) (p. 9, l. 442–444) shows that replacing the single judge with the layered verifier already produces most of the reported improvement, so it is unclear whether the gain belongs to critique-guided editing or simply to rejecting more candidates, and the ablation of the loop therefore does not separate the two. Cost accounting is partial in the same way: verification is excluded from the reported evaluation counts although it runs on separate devices including a large judge, the relationship between the reported evaluation multiple and wall clock is stated to depend on a separately matched timing protocol (Supp. p. 7, l. 346–347), and the full ledger of model calls and hardware time is itself listed among the controls not yet run (Supp. p. 12, l. 634–639). The paper is also an assembly of existing parts (a compiler, a causal backbone, a verifier, DPO), and without per-component budgets and a matched-budget baseline the size of the contribution is hard to judge.
Questions for authors
  1. Table 3 (p. 8, l. 378–386): how are "Best-of-3", "Refine ×3" and "Semantic" defined, and what does per-shot resampling under the same layered verifier give at a comparable budget? The main-text claim that targeted revision is cheaper than "whole-video resampling" (p. 7, l. 375–377) needs that arm to be interpretable.
  2. Fig. 1 and Table 3 (p. 1, l. 041–049; p. 8, l. 410–412): which of the stage-comparison numbers come from the same sampling budget as the full loop, and what is the per-component cost of the compiler, verifier and revision passes? Without that split, it is difficult to distinguish the contribution of each component.
Reviewer_UHJhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CineDirector uses executable shot programs to control timing, camera motion, and character identity in multi-shot video generation. A verifier checks each shot and triggers local repairs without changing previously accepted footage. Director Distillation then trains the generator on successful repairs so that some improvements remain when verification is disabled.

Critical strengths
  • The separation between persistent cast memory and shot-local context is sensible. In particular, rejected repairs cannot update the committed identity bank.
  • Comparing preference pairs under the same program and memory snapshot avoids a real conditioning confound.
  • Table 4(a) provides useful evidence for the boundary objective: adding it reduces CDE from 0.29 to 0.04. The loop-off distillation results are also worth investigating further.
Critical weaknesses
  • The benefit of diagnosis-guided repair is not isolated. Table 3 lacks a control that resamples the same flagged shots with the same budget but without diagnostic instructions. The gains may largely come from selective local resampling.
  • Camera alignment can hide reduced motion. The similarity transform in Eq. (8) removes translation-scale differences, so lower camera error does not establish that repairs preserve the requested displacement.
  • The physics verifier is underspecified. The V-JEPA prediction setup, including masking and temporal context, is unclear. Calibration by camera-motion tier alone does not establish that its error separates physical violations from unusual but valid dynamics.
  • The contribution of critique remains unclear. Eq. (11) does not explicitly use critique; it enters through additional winner supervision. A matched comparison against DPO plus the same auxiliary loss with empty or shuffled critiques is needed.
  • Cut accuracy does not establish cross-shot coherence. Table 4(a) supports the boundary loss, but a scheduled per-shot concatenation baseline is still needed to separate precise timing from learned continuity across shots.
Questions for authors
  • How are the failure interval and diagnosis converted into condition edits and re-noising strength? How much is gained over diagnosis-free local resampling?
  • What fixes the scale in camera verification, especially when reduced-motion repairs are allowed?
  • Which V-JEPA checkpoint, masking pattern, and prediction horizon are used? How does it perform on controlled physical counterexamples?
  • Does critique still help against DPO plus matched winner supervision with empty or shuffled critiques? How is empty-critique inference supported during training?
Reviewer_hqDwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CineDirector turns a multi-shot video request into an executable “shot program” that fixes when each cut happens, how the camera moves, and which characters appear in each shot. A streaming generator follows this program directly. A layered verifier then checks each finished shot for semantics, identity, camera path, and physics. If a shot fails, only that shot is partially re-noised and regenerated, and it’s accepted only if it still meets the original request, so a fix can’t succeed by weakening the requested motion. Finally, the failed drafts and their accepted repairs are used as preference pairs to train the generator, so it makes fewer mistakes even with the verifier turned off. The main contribution is to connect these three steps (specify, repair locally, learn from repairs) around a single shared contract. Most of the underlying components, such as the streaming generator, camera conditioning, and DPO-style training, are drawn from prior work.

Critical strengths
  1. A fix is accepted only if it still meets the original camera and content targets, so the system can’t “fix” physics by quietly reducing motion.

  2. The model learns from its own repairs. Training on pairs of failed drafts and their fixes keeps a 45% drop in physical violations with the checker turned off. Plain fine-tuning, then preference learning, then preference learning with critiques, yields steadily better results, suggesting that the pairing itself matters.

Critical weaknesses
  1. The loop uses VGGT, ArcFace/DINO, and a physics detector, and these same tools appear to produce the evaluation scores.

  2. Missing and unfair baselines. Identity similarity to a reference image is compared with text-only models such as ShotStream and HoloCine. Reference-image generators like SkyReels-V3 (open-source), Vidu Q2, and Seedance 2.0 already existed and were not compared. CoAgent and physics-DPO methods are also missing.

  3. Novelty is overstated. The dual cache, the positional jump at cuts, and the cache reset are part of ShotStream’s design, while the condition mask resembles MultiShotMaster’s. ShotStream’s numbers in this paper also contradict ShotStream’s own paper, where it had the best cut control.

  4. Incomplete and inconsistent reporting. Several key analyses are described as “still needed” rather than run. There are no error bars despite three seeds, and no human-study numbers. Model labels conflict across tables, and the distilled loop-free model, which is the main selling point, never gets a full results row.

  5. The ethics statement doesn’t match the work. It claims no new data was collected, yet the paper builds a new 118K-scene dataset with face clusters. It claims no negative social impact for a system whose job is to preserve real-looking identities across shots.

Questions for authors
  1. Are your evaluation metrics (especially PVR) computed with the same tools the loop optimizes? Can you add an independent judge, like a held-out physics benchmark or a blind human study?

  2. Can you please compare against reference-image generators like SkyReels-V3, Vidu Q2, or Seedance 2.0? And why is ShotStream’s cut accuracy in Table 1 much worse than in its own paper?

  3. Can you give full results, with error bars, for the distilled model with the loop off? And the matched-data comparison showing the gain comes from the draft-versus-repair pairs, not just extra data?


My main justification for the score is:

1- Self-preferences. (weakness 1)

2- The core idea is good and useful, but the evidence doesn’t yet support the claims. Most gains are measured by the same tools the system optimizes, and the most relevant baselines are missing.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.