acceptodds
Under review as a conference paper at ICLR 2027

CANVAS: PLAN THE SHOOT, NOT JUST THE STORY, FOR LONG VIDEO

Abstract

Video generators synthesize striking short clips, but assembling them into coherent multi-shot films remains difficult. Existing approaches plan the story and shots, then render each shot with the same conditioning policy and compute budget. We argue that coherence hinges on two shot-level decisions: what visual context to inherit and how much computation to spend. We introduce CANVAS, a training-free framework that makes both decisions before generation. It compiles a story into an executable shooting plan: pixels carry over only within continuous action, while structured state carries identities, objects, and narrative history across cuts. A film level budget assigns computation according to shot difficulty, and a verifier directs revisions to specific failures. On ViMax-Bench, with the language model, image generator, and video generator held fixed, CANVAS raises global consistency from 0.528 for ViMax and 0.452 for MovieAgent to 0.694, with VBench aesthetic and imaging quality on par with ViMax. It spends 0.76 times the denoising of a uniform 25-step pass on the same 35 stories; generation takes 136 minutes per film, versus 129 for ViMax and MovieAgent. The consistency gains are supported by three evaluator families and a human study, and persist without frame chaining. We will release the code to the public.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
agents, agentic video generation, video generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_xM1COpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents CANVAS, a training-free agentic framework for multi-shot story videos with a frozen generator. A language model compiles a story overview into a validated shot plan, and before rendering each shot is assigned a boundary label (continue from the previous last frame or open from three-view identity packs), a selected text history, an object-action contract and a denoising tier set by a hand-weighted complexity score. A verifier gates frame quality and temporal consistency, and a reviser rewrites the prompt to address the failed check. With Qwen3.8-27B, FLUX.2-klein and MiniMax-H3 shared across systems, CANVAS reaches a global ViCLIP consistency of 0.694 on ViMax-Bench against 0.528 for ViMax and 0.452 for MovieAgent, at 0.76× the denoising of a uniform 25-step pass, and is preferred in 65–75% of human judgments.

Critical strengths

The consistency gains are large and agree across evaluators under a shared backend. The planner, image model, generator, resolution, frame rate, shot length and shot count are held fixed, and against ViMax CANVAS improves ViCLIP GC by 0.166, DINOv2 GC by 0.295 and ArcFace identity by 0.210, without loss in aesthetic or imaging quality. In a blinded study with 30 raters, CANVAS is preferred over HoloCine, ViMax and MovieAgent on all four criteria.

The routing principle is simple, transferable and candidly presented. Restricting pixel inheritance to one frame within continuous action reduces within-scene handoff change from 48.6 to 3.8, and transferring identity packs raises the baselines' GC by about 0.03. The authors score their 17 failed shots and explicitly report several negative findings.

Critical weaknesses

The headline gain is not attributed to the proposed mechanisms. On the ten ablation stories the full system scores GC 0.692, and removing chaining, identity packs, contracts or adaptive steps gives 0.659–0.688, while ViMax scores 0.527. Giving ViMax CANVAS's identity packs lifts it only to 0.563, which leaves about 0.13 GC unexplained, and the conclusion concedes that "the tested ablations do not explain the full global-consistency gap". The factors that have not been isolated are CANVAS's re-authored plan (5.3 scenes per film against 2–4 in the benchmark, to which ViMax is constrained), its prompt wording and the history selector H_i, which is central to Eq. 2 but never ablated; switching the planner LLM alone moves GC from 0.692 to 0.725. Mean pairwise similarity also rewards repeated sets, costumes and framing, and no shot-scale, camera-angle or location diversity is reported. In Fig. 8, the "median-gap" example, the five displayed CANVAS shots share one room and nearly identical wide framing (GC 0.859), while ViMax's vary in scale and setting.

Three listed contributions are contradicted by the appendix. Adaptive tiers versus uniform 25 steps change GC by +0.0013 [−0.0047, 0.0069], first-attempt pass rates by tier are 0.91, 0.88, 0.89, 0.88 and 0.96, and the authors write that "the tiers do not predict what fails"; a uniform schedule at the same mean spend is not reported, and the film-level budget is exceeded in 4 of 35 films. Object contracts "do not raise measured object-state fidelity" (0.743 vs 0.721). For typed revision, E1 records that of 47 repairs "2 fixed the failed check, 44 changed nothing, and 1 made it worse", yet Table 4 credits it with a GC gain of 0.047 over a one-take arm that differs on only about 6 of 88 shots. The verifier's scoring functions, thresholds and rewrite rules are not specified.

Baseline fidelity and missing records further limit the evidence. Both ported baselines score below their published GC, unmodified ViMax failed to parse about half its camera trees, and MovieAgent's keyframes come from its own StoryDiffusion module, so the image model is not fully held fixed. CANVAS uses MiniMax-H3's multi-image reference-to-video mode with up to three attempts per shot, while the baselines use single-keyframe image-to-video with one attempt. GroundShot, described as the closest work, is not compared. Tables 8, 9, 10 and 14 state that per-story records are unavailable, and the human study treats judgments from 30 raters on 24 stories as independent.

Questions for authors
  1. Can a plan-swap experiment be reported, running the CANVAS executor on the baselines' own plans and the baselines' executors on CANVAS plans, together with an ablation of H_i?

  2. What are the shot-scale, camera-angle and location diversity of each system, and what is the distribution of per-story GC gaps?

  3. What do a uniform schedule and a shuffled tier assignment at the same mean spend achieve?

  4. How does typed revision produce a 0.045–0.047 GC gain when about 9% of shots are re-rendered, and what are the verifier's exact definitions?

  5. Which timing figures and which no-chaining GC are correct, are the Fig. 1 films from ViMax-Bench, and can the missing per-story records be provided?

8 comments
AuthorsOpenReview ↗

Response to Reviewer xM1C (1/5)

We have uploaded a revised manuscript that addresses the concerns in this review; all revised text is shown in blue. The table at the end of part 5/5 lists each change and where it appears in the revised PDF.

We thank Reviewer xM1C for the detailed review. It led to most of the new experiments, and we agree with its central point: the submitted ablations did not explain the gain, and three contributions were overstated. The revision addresses both. Table, figure, section and appendix numbers refer to the revised PDF; quoted review text keeps the submitted numbers. The plan-swap grids are in our General Response.

Q1 (W1). The headline gain is not attributed to the proposed mechanisms

The factors that have not been isolated are CANVAS's re-authored plan […], its prompt wording and the history selector H_i, which is central to Eq. 2 but never ablated; switching the planner LLM alone moves GC from 0.692 to 0.725. Mean pairwise similarity also rewards repeated sets, costumes and framing […] In Fig. 8, the "median-gap" example, the five displayed CANVAS shots share one room and nearly identical wide framing (GC 0.859), while ViMax's vary in scale and setting.

A1. The new runs isolate the factors you listed:

  • Re-authored plan: swapping only the plan moves GC by 0.106 to 0.124 under either executor, the larger part of the gap (General Response). On the benchmark's own script, CANVAS still leads ViMax by +0.126 GC.
  • Prompt wording is part of what the plan swap moves: every CANVAS shot restates its setting and wardrobe in the same words (437 of 437 shots). With that restated text stripped from both plans, CANVAS's plans still lead ViMax's by +0.164 GC on all six paired stories (Table 30).
  • History selector Hᵢ: see A6.
  • Planner LLM: a stronger planner raises CANVAS's GC (0.692 to 0.702 or 0.725). This is consistent with the plan being the main factor, and every planner stays far above ViMax (0.527). We keep Figure 7(b) and label the Gemini and Claude rows as exploratory (Figure 7b, Tables 24 and 25).
  • Repeated sets and framing: on medium-to-long pairs from different locations, the lead is +0.135 GC and about +0.16 face identity (A7).

Figure 8. The revision describes Figure 8 as a multi-character story and adds a benchmark-script story at the median gap (Figure 9), with CANVAS's and ViMax's shots side by side, so the example no longer depends on CANVAS's own scene layout.

Revision. Appendix M attributes the gain factor by factor (grids in Table 4), and Section 5 replaces "the tested ablations do not explain the full global-consistency gap" with this attribution.

Q2 (W2). Three listed contributions are contradicted by the appendix

Adaptive tiers versus uniform 25 steps change GC by +0.0013 [−0.0047, 0.0069] […] a uniform schedule at the same mean spend is not reported, and the film-level budget is exceeded in 4 of 35 films. Object contracts "do not raise measured object-state fidelity" […] Table 4 credits it with a GC gain of 0.047 over a one-take arm that differs on only about 6 of 88 shots. The verifier's scoring functions, thresholds and rewrite rules are not specified.

A2. We agree, and the contribution list changes (General Response):

  • Tiers: uniform and shuffled schedules at similar planned spend give the same GC (A8), and 4 of 35 films exceed B through repairs. We report an average saving (0.76 of B) at similar GC. The fixed text-complexity rule describes how steps are assigned; we claim neither a performance advantage from this allocation rule nor a hard film-level budget.
  • Contracts: object-state fidelity is 0.743 with contracts versus 0.721 with prose (Appendix E); we describe their role as a pre-render check on object hand-offs.
  • Verification-triggered retries: both repair policies reduce delivered failures relative to one take; A9 distinguishes the R1 and R7 comparisons and gives the verifier's definitions.

Revision. Section 1's contribution list states these roles; Appendix B gives the verifier's formulas and thresholds and the reviser's rules (Table 6).

AuthorsOpenReview ↗

Response to Reviewer xM1C (2/5)

Q3 (W3). Baseline fidelity and input route

Both ported baselines score below their published GC, unmodified ViMax failed to parse about half its camera trees, and MovieAgent's keyframes come from its own StoryDiffusion module, so the image model is not fully held fixed. CANVAS uses MiniMax-H3's multi-image reference-to-video mode with up to three attempts per shot, while the baselines use single-keyframe image-to-video with one attempt.

A3.

  • Published GC uses Gemini-3-Flash with Wan2.2 or Veo 3.1. Here every system uses Qwen3.8-27B and H3, and on CANVAS's plans the ported executors reach 0.605 and 0.580 GC.
  • Camera trees: ViMax's call is constrained to its own response schema with one identical retry, so its trees parse.
  • StoryDiffusion: MovieAgent's executor, StoryDiffusion keyframes included, comes within 0.073 GC of CANVAS on CANVAS's plans. Replacing its keyframes with CANVAS's identity packs raises its GC by only 0.032 (Table 16).
  • Reference-to-video and attempts: CANVAS's plan text through plain text-to-video with one take and no references still leads ViMax by +0.185 GC; CANVAS with one take scores 0.645 against ViMax's 0.527 (Tables 7 and 16).

Revision. Section 4.1 states, per system, the keyframe source (FLUX.2-klein, or StoryDiffusion for MovieAgent), the input route and the attempts per shot; Appendix C lists ViMax's schema change, and Appendix M adds the text-to-video control.

Q4 (W3). GroundShot

GroundShot, described as the closest work, is not compared.

A4. GroundShot (arXiv 2606.20799) has released its core code, while its repository lists GroundBench, evaluation code and runnable examples as forthcoming. It reports results only on its own GroundBench (no ViMax-Bench numbers) with Vidu Q3, Kling or Wan2.1, and uses GPT-4o and GPT-4.1 for parsing, scheduling and verification, so a matched comparison needs adapting its pipeline to the shared planner and H3. The methods differ: GroundShot builds an entity memory from accepted shots and retrieves references for each shot, while CANVAS fixes each shot's text, boundary and steps in a plan compiled before rendering.

Revision. Section 2 and Appendix L consistently describe GroundShot as a closely related system and explain its different reference-memory design, benchmark and video models.

AuthorsOpenReview ↗

Response to Reviewer xM1C (3/5)

Q5 (W3). Per-story records and the human study

Tables 8, 9, 10 and 14 state that per-story records are unavailable, and the human study treats judgments from 30 raters on 24 stories as independent.

A5. Records. The revision adds per-story values for the earlier-route ablations and backbones (Tables 13 and 15, for revised Tables 12 and 14), the within-run controls (Table 18), the planners (Table 25), the repair re-runs (Tables 10, 11 and 31) and the baseline executors rendering CANVAS’s plans (Table 29) and text controls (Table 30). The single-seed no-chaining run (GC 0.676) is in Table 7. The printed per-story tables identify the experiments they cover.

Human study. Table 26 reports the recorded counts and shares (240 judgments per baseline and criterion). Table 33 (Appendix P) adds 95% percentile confidence intervals from 10,000 two-way product-weight (pigeonhole) cluster bootstrap repetitions (NumPy default_rng, seed 42), independently resampling stories (24 clusters) and raters (30 clusters). Each observed judgment receives the product of its story and rater resampling multiplicities as its weight, and the preference share is recomputed for each baseline and criterion. This accounts for both sources of dependence. All 12 intervals lie above 50%. As complementary evidence scored per story on all 35 stories, two VLM judges (Qwen3.8-27B, InternVL3.5-38B) compare storyboards of each pair under neutral names, in both orders; a story counts as a win when both orders agree and half a win when they disagree. Share of the 35 stories preferring CANVAS:

Judge and baselineVisual qualityText alignmentCross-scene consistencyCharacter consistency
vs. ViMax, Qwen3.8-27B0.540.630.640.64
vs. ViMax, InternVL3.5-38B0.510.610.610.60
vs. MovieAgent, Qwen3.8-27B0.560.800.860.89
vs. MovieAgent, InternVL3.5-38B0.530.710.730.73

Both judges prefer CANVAS on text alignment and on both consistency questions against both baselines; on visual quality they are near parity, consistent with Table 2.

Revision. Section 4.2 and Figure 7(a) report the human study as counts and shares (Table 26, Appendix H); Appendix H cites the clustered intervals, and Appendix P gives the two-way bootstrap method and all 12 intervals (Table 33). Section 4.2 summarizes the VLM check, with its protocol and order swap in Appendix H (Table 27) and the judge prompt in Appendix N.

Q6 (Q1). Plan swap and an ablation of Hᵢ

Can a plan-swap experiment be reported, running the CANVAS executor on the baselines' own plans and the baselines' executors on CANVAS plans, together with an ablation of H_i?

A6. Plan swap: we ran it in both directions with both baselines; the grids are in the General Response. Hᵢ: we re-authored five films with Hᵢ on and off and rendered both with the same text-to-video executor; removing Hᵢ lowers GC by 0.038 across the five stories (Table 30).

Revision. Table 4 (Appendix A.1) shows both grids; Appendix M gives the plan conversion, the ten-story executor swaps (Table 29) and the text and Hᵢ controls (Table 30).

Reviewer_zMfaOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CANVAS is a training-free framework for producing long videos. It compiles a story into an executable shot plan, choosing whether each shot continues from the preceding frame or starts from character reference images, while symbolic records carry identity, objects, and events. Hand-set shot complexity determines denoising steps, and typed checks trigger local retries. On 35 ViMax-Bench stories, comparisons sharing the language model and video backend improve global consistency from 0.528 for ViMax and 0.452 for MovieAgent to 0.694. DINOv2, ArcFace, and human preferences provide complementary support. Including retries, the default schedule averages 0.76 times the denoising of one uniform 25-step pass; this should not be read as an equivalent reduction in total system runtime.

Critical strengths

The most convincing result is the substantial system-level gap under shared backends: GC is 0.694 versus 0.528 for ViMax, while the DINOv2 difference is +0.295 with a paired story interval of [0.261, 0.330], and ArcFace also improves. The paper does not rely solely on within-scene similarity that can benefit mechanically from chaining: it includes near-duplicate filtering, a no-chaining ablation, and checks of motion and script alignment. A study with 24 stories and 30 raters further supports perceptual gains.

Critical weaknesses

Budget enforcement and the benefit of difficulty-based allocation are not established. Section 3.6 defines B as the reference cost of one uniform 25-step pass, while the introduction and contributions also describe a fixed film budget with reserved revision capacity. Equation (4) maps hand-set scores to steps and each shot permits two revisions. The execution procedure does not explain reallocation or stopping when total cost would exceed B. Figure 5(a) shows only 31 of 35 films below the uniform 25-step reference cost. The evidence therefore supports average savings, not a per-film hard constraint. Comparing against 25 steps and another difficulty-tier schedule also does not rule out similar performance from a lower uniform step count or a random permutation of the same allocations. First-attempt pass rates do not decrease with tier, but tiers already receive different compute, so this alone does not invalidate difficulty scoring; a fixed-compute or cost-matched test is needed.

Questions for authors

[Budget definition] Section 3.6 defines B as the uniform 25-step reference cost, while other passages describe a fixed budget with revision reserves inside it. Please reconcile these meanings and state whether a separate hard-budget rule includes all retries. How should the four films exceeding the reference in Figure 5(a) be interpreted?

[Compute attribution] With plans, seeds, verifier, and actual total denoising cost matched, does difficulty allocation outperform a uniform setting with a similar mean step count or a random permutation of step allocations? Please also reconcile the timing scope behind the abstract's 136/129 minutes, Figure 1's 20 stories, and Figure 5's 13 single-character stories.

4 comments
AuthorsOpenReview ↗

Response to Reviewer zMfa (1/2)

We have uploaded a revised manuscript that addresses the concerns in this review; all revised text is shown in blue. The table at the end of part 2/2 lists each change and where it appears in the revised PDF.

We thank Reviewer zMfa for the precise questions on the budget and timing. You were right that the evidence supports average savings rather than a per-film constraint or a benefit from difficulty scoring, and the revision says exactly that. Table, figure, section and appendix numbers refer to the revised PDF; quoted review text keeps the submitted numbers.

Q1 (Weakness). Budget enforcement and the benefit of difficulty-based allocation

Budget enforcement and the benefit of difficulty-based allocation are not established. […] The evidence therefore supports average savings, not a per-film hard constraint. Comparing against 25 steps and another difficulty-tier schedule also does not rule out similar performance from a lower uniform step count or a random permutation of the same allocations. […] a fixed-compute or cost-matched test is needed.

A1. We agree with both points. The reported runs enforce no per-film cap, so the result is an average saving: 0.76 of a uniform 25-step pass over 35 films. The schedule comparison you suggested gives similar GC with uniform steps and a random permutation of the tiers at similar planned spend, with actual costs reported in A3. The saving comes from H3 keeping its quality at fewer steps, not from difficulty scoring.

Revision. The abstract and Section 1 state the saving as an average against the reference cost B and remove the hard film-budget claim. Section 1 describes the fixed text-complexity rule and explicitly notes that a uniform schedule at similar planned spend performs equally well. Section 4.3 adds the similar-spend test, with actual cost ratios in Table 10.

Q2 (Question). Budget definition

Section 3.6 defines B as the uniform 25-step reference cost, while other passages describe a fixed budget with revision reserves inside it. Please reconcile these meanings and state whether a separate hard-budget rule includes all retries. How should the four films exceeding the reference in Figure 5(a) be interpreted?

A2. B is the cost of a uniform 25-step pass over the same shots (Section 3.6), and spend is reported against it. Steps are set per shot before rendering, each shot allows at most two repairs, and no separate film cap is applied, so retries are not bounded by B. The four films above B in Figure 5(a) (1.015 to 1.074) were planned below it (0.60 to 0.93) and went over through repairs. Pooled over 35 films, first attempts use 0.65 of B and all attempts 0.76.

Revision. Sections 1 and 3.6 and the captions of Figures 2 and 3 define B as this reference and remove the "revision reserve within the budget" wording. The caption of Figure 5(a) explains the four films and gives the pooled spend; Appendix D gives their planned and spent ratios.

Q3 (Question). Compute attribution at matched cost

With plans, seeds, verifier, and actual total denoising cost matched, does difficulty allocation outperform a uniform setting with a similar mean step count or a random permutation of step allocations?

A3. With plans, seeds and verifier fixed and planned spend matched, the three schedules give the same GC (six stories, three seeds); actual denoising including all attempts stays within 5% of the tiers':

Schedule at similar planned spendΔGC vs. difficulty tiersActual denoising / tiers
Uniform at the film's mean steps+0.0001.045
Random permutation of the film's own steps−0.0020.987

Difficulty allocation does not outperform either. The efficiency result stands on its own: tiers averaging 16.4 steps match 25 steps on every shot (0.694 vs. 0.693, 35 stories), while 8 to 13 steps drop to 0.678.

Revision. Section 4.3 summarizes this comparison; Table 10 (Appendix E) gives per-story values and the actual cost ratios.

AuthorsOpenReview ↗

Response to Reviewer zMfa (2/2)

Q4 (Question). Timing scope

Please also reconcile the timing scope behind the abstract's 136/129 minutes, Figure 1's 20 stories, and Figure 5's 13 single-character stories.

A4. We distinguish the 35-story modeled comparison from the 13-story timing estimates in Figures 1 and 5(b).

  • Abstract (136 vs. 129 minutes, 35 stories): H3 minutes per film, modeled from per-call timings on one RTX 6000 Ada, with repairs included for CANVAS and one 25-step call per shot for the baselines; since both baselines render the same number of equal-length shots, both get 129. Planning, image generation, verification and assembly are excluded. CANVAS uses 0.76 of the denoising yet slightly more minutes because its scene openings condition on several characters' reference pictures (up to nine), and these calls take longer than the baselines' single-keyframe calls (13.9 against 10.3 minutes per 25-step call on average), and CANVAS makes 512 calls for 437 shots.
  • Figure 1: its films are illustrative stories written for the figure, not ViMax-Bench films, and its time panel uses the 13 single-character stories of Figure 5(b). The caption said "ViMax-Bench" and "20 stories"; both are corrected.
  • Figure 5(b): all five bars now come from the same per-call model on the 13 single-character stories. Default CANVAS changes from 86 to 69 minutes and ViMax from 153 to 122 (MovieAgent stays at 122), because the submitted 86 and 153 did not come from this model (153 also charged ViMax re-renders that the evaluated films did not use). The 25-steps-on-every-shot and 8–13-step bars come out unchanged at 113 and 50, because the submitted panel had already computed them with this model. The 35-story per-call averages are cohort summaries rather than fixed rates for this subset.

Revision. The abstract and the captions of Figures 1 and 5(b) state the timing model and story set; Section 4.3 and Appendix D give the per-call model for both story sets and state the excluded stages.

On the human study

A study with 24 stories and 30 raters further supports perceptual gains.

Thank you for noting it. We now report 95% percentile confidence intervals from 10,000 two-way product-weight (pigeonhole) cluster bootstrap repetitions (NumPy default_rng, seed 42), independently resampling the 24 stories and 30 raters. Each judgment is weighted by the product of its story and rater resampling multiplicities, and each baseline–criterion preference share is recomputed. Table 33 (Appendix P) gives the intervals; all 12 lie above 50%. Table 26 retains the recorded counts and shares. See also our response to xM1C (A5).

Summary of manuscript changes for Reviewer zMfa

ConcernChange in the revised manuscriptWhere
Budget enforcementSaving stated as an average against the reference cost B; hard film-budget and "revision reserve within the budget" wording removed; four films above B explained (planned 0.60–0.93, spent 1.015–1.074)Abstract; Sec. 1; Sec. 3.6; Fig. 2, 3 and 5(a) captions; App. D
Benefit of difficulty-based allocationUniform and random-permutation schedules at similar planned spend, with actual denoising ratios; AUC and correlation of the score; allocation-benefit claim removedSec. 4.3; Table 10 (App. E); App. D
Budget definitionOne definition of B (uniform 25-step pass); retries not bounded by B; pooled spend 0.65 (first attempts) and 0.76 (all attempts)Sec. 3.6; Fig. 5(a) caption; App. D
Timing scope35-story modeled time (136 vs. 129 min) and 13-story estimates (Fig. 1, 5(b)) from one per-call model, with excluded stages stated; Figure 1 caption correctedAbstract; Sec. 4.3; Fig. 1 and 5(b) captions; App. D
Human studyRecorded counts and shares; two-way cluster-bootstrap 95% intervalsSec. 4.2; Fig. 7(a); Table 26 (App. H); Table 33 (App. P)

I appreciate the author’s detailed response. The clarification that BB represents a reference cost based on a uniform 25-step generation, rather than a hard, per-segment limit inclusive of retries. It resolves my questions regarding the definition of the budget. The explanation for why four of the videos exceeded BB also supports framing the 0.76× result as an average saving in denoising costs. Furthermore, the comparisons involving uniform step counts and randomized shuffling, along with the score calibration under a fixed number of steps, address my concerns regarding the attribution of computational effort.

It should be noted that these step-count comparisons are based on planned costs rather than actual total costs. When compared to the stratified approach, the actual cost ratios are 1.045 and 0.987; given that this evaluation covers only six stories and three random seeds, I do not believe a clear advantage for the allocation strategy has been demonstrated. While the response addresses my primary concerns, it does not definitively establish that the difficulty-based allocation itself provides a significant contribution. Consequently, I will maintain my score of 3 (Weak acceptance).

AuthorsOpenReview ↗

Response to Reviewer zMfa: Real-cost savings from per-shot compute planning

Thank you for the follow-up. We address the two remaining points.

Planned vs. actual cost. Only the matched-spend test (Table 10) matches planned cost. The efficiency results use actual cost: the 0.76 of BB counts every attempt, repairs included, and Figure 5(b) charges every executed H3 call.

SettingGC ↑ (35 stories, Table 1)H3 minutes per film ↓ (13 stories, Fig. 5b)
CANVAS, 25 steps on every shot0.693113
CANVAS, default tiers (13 to 25 steps)0.69469
CANVAS, 8 to 13 steps0.67850
ViMax0.528122
MovieAgent0.452122

Over all 35 stories, where scene openings use more reference pictures, the default tiers take 136 minutes and the baselines 129 (Section 4.3).

Difficulty-based allocation. We agree, and we do not claim that ranking shots by difficulty beats a uniform schedule with the same mean steps. Our claim is the measured saving. The default tiers match the GC of 25 steps on every shot at 0.76 of its denoising and about 40% less time. With 8 to 13 steps, CANVAS needs less than half the baselines' time and still reaches GC 0.678 against 0.528 and 0.452, with lower imaging quality (Table 2). The matched-spend test shows that this saving holds whether steps are ranked by difficulty or spread evenly.

Reviewer_nwDWOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

CANVAS is a training-free agent pipeline for multi-shot video that plans each shot before rendering, using frozen models. An LLM writes the shot plan and labels each shot as a continuation (starting from the previous shot’s last frame) or a cut (opening from three-view character references, with story state passed only as text). It also adds object-state contracts, hand-set per-shot denoising steps, and a verifier that revises failed shots. On ViMax-Bench with a shared backend, it raises global consistency from 0.528 (ViMax) and 0.452 (MovieAgent) to 0.694, with similar visual quality and 0.76× the denoising compute.

Critical strengths

1- Controlled comparison. All systems share the same planner, image model, video generator, resolution, and shot length. The gains are also assessed using ViCLIP, DINOv2, ArcFace, VBench, and a blinded human study, so they are not an artifact of a single metric.

2- Simple, cheap, and honest. “Pixels only inside continuous action, text state across cuts” is easy to adopt, training-free, and uses less compute. The paper also openly reports failed shots, weak components, and what the ablations cannot explain.

Additionally, I appreciate that the authors report all the details in the appendix, making it easily reproducible.

Critical weaknesses

1- Novelty. MovieAgent and ViMax already do hierarchical story → scene → shot planning, and the validate-and-correct loop is a small engineering addition. The explicit cut/continue rule is the clearest new idea, but it is simple film-grammar logic. The adaptivity claimed in the abstract and introduction is limited: compute allocation is a hand-set lexicon score with fixed thresholds (Appendix D), not learned, and context routing is a binary continue/cut label. Both are fixed before rendering and never updated from what was rendered. Identity packs are not novel as an idea, since reference banks exist (e.g., MovieAgent’s character bank), and the gain depends on H3 supporting reference-to-video. The typed verifier with prompt revision is close to ViMax’s VLM best-of-k selection and VideoAgent-style feedback loops.

2- The source of the gain is unclear. The gap over ViMax is 0.166 GC (35 stories), but the ablations (10-story subset) do not explain it. Removing chaining costs 0.016, identity packs 0.033, contracts 0.009, adaptive steps 0.004, and typed revision 0.047, about 0.11 in total. Giving ViMax the identity packs adds only 0.036, and the authors admit the gap is unexplained. A likely confound is that CANVAS writes its own scenes and shot content (5.3 scenes per film vs. 2–4 in the benchmark), while the baselines follow the benchmark script. This may produce more uniform content that ViCLIP scores as consistent.

3- Baseline fairness. Both baselines score below their published numbers on this backend (ViMax 0.528 vs. 0.532–0.557). ViMax needed code changes (about half its camera trees failed to parse), and MovieAgent’s keyframe module had to be wired in by hand. More importantly, the baselines use image-to-video from resized keyframes, while CANVAS uses H3’s reference-to-video path, so the setup favors CANVAS’s input route. The main results also use one seed per film.

4- Several contributions add little. Contracts are roughly equal to plain prose (GC 0.683 vs. 0.692, and the failed-shot difference has an interval that includes zero). Adaptive compute does not change consistency, and first-attempt pass rates are flat across tiers, so the difficulty score does not predict difficulty. For the reviser, only 2 of 47 repairs fixed the failed check on the earlier route (E1). On the main route, 23 of 40 failing shots passed after revision, but the reviser may also change the seed, so the cause is unclear. Yet typed revision vs. untargeted resampling differs by only 4 failed shots out of 88, while GC changes from 0.647 to 0.692 at equal time, which is hard to explain.

Questions for authors

1- If you give ViMax and MovieAgent the exact CANVAS shot plans and run CANVAS on the benchmark script directly, how much of the 0.166 GC gap is left? This would separate “better plan” from “better execution.”

2- The reviser fixed only 2 of 47 failures, and only about 40 of 437 shots fail at all. How can typed revision vs untargeted resampling change GC by 0.045? Can you show the per-shot breakdown?

3- Does the difficulty score predict anything? Please show the correlation between the score and per-shot failure or quality, and compare against random tier assignment at the same budget.

4- How often are the planner’s cut/continue labels wrong, and how often do the rendered videos actually satisfy the object contracts? Can you add a human or VLM check of story correctness, plus multiple seeds for the main table?

11 comments
AuthorsOpenReview ↗

Response to Reviewer nwDW (1/4)

We have uploaded a revised manuscript that addresses the concerns in this review; all revised text is shown in blue. The table at the end of part 4/4 lists each change and where it appears in the revised PDF.

We thank Reviewer nwDW for the plan-swap and benchmark-script suggestions. We ran both, and they changed how we present the paper. Table, figure, section and appendix numbers refer to the revised PDF; quoted review text keeps the submitted numbers. The plan-swap grids and the summary of revised claims are in our General Response.

Q1 (W1). Novelty relative to MovieAgent, ViMax and feedback loops

MovieAgent and ViMax already do hierarchical story → scene → shot planning, and the validate-and-correct loop is a small engineering addition. […] compute allocation is a hand-set lexicon score with fixed thresholds (Appendix D), not learned […] Identity packs are not novel as an idea […] The typed verifier with prompt revision is close to ViMax's VLM best-of-k selection and VideoAgent-style feedback loops.

A1. We agree that hierarchical planning, reference banks and verify-and-retry loops each exist, and Section 2 now says so. What is new, and now measured, is (a) a plan that fixes for every shot, before rendering, its boundary, its pixel input (at most one frame, never across a cut), the text state carried across cuts and its object hand-offs; and (b) evidence that this plan carries most of the gain: on CANVAS's plans, ViMax's and MovieAgent's own executors each gain 0.124 GC (General Response). Concretely, every CANVAS shot restates its setting and each character's wardrobe in the same words (437 of 437 shots).

You are right that both decisions are fixed before rendering and never updated from what was rendered; we replace "adaptive" with "per-shot" and identify online replanning as future work (Section 5). On reliance on H3's reference-to-video mode: CANVAS's plan text through plain text-to-video (no references, no chaining, one take) still leads ViMax by +0.185 GC (17 of 18 stories), so references mainly add face identity rather than GC. The verifier gates each take rather than ranking keyframe candidates. In the main R1 runs, the rewritten positive prompt never reached H3: the positive prompt was recomposed from the object plan and repairs operated through seed changes. Direct prompt rewriting is evaluated separately in R7; we present verification-triggered retries as an engineering component, not a core contribution.

Revision. Section 1 states the plan and its transfer as the first contribution; Section 2 positions each component against prior work as above; Section 5 states the limitations and future work.

Q2 (W2) and Q5 (Q1). The source of the gain, and plan versus execution

The gap over ViMax is 0.166 GC (35 stories), but the ablations (10-story subset) do not explain it. […] A likely confound is that CANVAS writes its own scenes and shot content (5.3 scenes per film vs. 2–4 in the benchmark), while the baselines follow the benchmark script. This may produce more uniform content that ViCLIP scores as consistent.

If you give ViMax and MovieAgent the exact CANVAS shot plans and run CANVAS on the benchmark script directly, how much of the 0.166 GC gap is left? This would separate "better plan" from "better execution."

A2. The plan swap separates the two: changing only the plan adds 0.106 to 0.124 GC under either executor, and changing only the executor adds 0.019 to 0.089 (grids in the General Response). With CANVAS's plans, ViMax's executor ends 0.019 GC below CANVAS and MovieAgent's 0.073 below. On the benchmark's own script (11 stories), CANVAS scores 0.559 GC against 0.433 for ViMax's own films and 0.521 for ViMax's executor on the same outline, and leads that executor by +0.261 face identity, +0.042 IC and +0.035 CC.

The gain also survives the uniform-content explanation. On medium-to-long shot pairs from different locations, CANVAS leads ViMax by +0.135 GC and about +0.16 face identity. A blind VLM story check finds that CANVAS shows the overview's events as often as ViMax (0.935 vs. 0.928) and keeps them in story order more often (0.943 vs. 0.829). CANVAS films are slightly less varied (4.56 distinct shot scales per film against 4.83 for ViMax, and 7.53 locations against 8.20; xM1C A7), and the controls above show that the gain does not depend on this.

Revision. Table 4 (Appendix A.1) gives the plan-swap grids. Appendix M reports the aggregate benchmark-script and like-for-like results and describes the plan conversion. Table 29 provides per-story GC for the baseline executors rendering CANVAS’s plans, alongside the baseline and CANVAS reference films; Table 30 provides per-story text controls and the history-selector summary. Appendix F gives the diversity counts (Table 22).

AuthorsOpenReview ↗

**Response to Reviewer nwDW (2/4)**

Q3 (W3). Baseline fairness

Both baselines score below their published numbers on this backend […] ViMax needed code changes […] the baselines use image-to-video from resized keyframes, while CANVAS uses H3's reference-to-video path, so the setup favors CANVAS's input route. The main results also use one seed per film.

A3. Published numbers use Gemini-3-Flash with Wan2.2 or Veo 3.1; here every system uses Qwen3.8-27B and H3. On CANVAS's plans the ported executors reach 0.605 and 0.580 GC, so they render consistent films when given consistent plans. Code changes: ViMax's camera-tree call is constrained to its own response schema with one identical retry, so its trees parse. Input route: giving the baselines CANVAS's identity packs through the same reference-to-video call raises their GC by only 0.032 to 0.036 (Table 16), and the plain text-to-video control above still leads by +0.185. Attempts: CANVAS with one take still scores 0.645 GC against ViMax's 0.527 on the same ten stories (Tables 7 and 16). Seeds: over three seeds of every system on five stories, CANVAS leads ViMax by +0.180 and MovieAgent by +0.195 GC.

Revision. Section 4.1 adds, per system, the planner, keyframe source, input route and attempts per shot; Table 1's caption names the published runs' planner and video models. Table 9(b) (Appendix E) reports three-seed means for every system on five stories.

Q4 (W4). Several contributions add little

Contracts are roughly equal to plain prose […] Adaptive compute does not change consistency, and first-attempt pass rates are flat across tiers […] For the reviser, only 2 of 47 repairs fixed the failed check on the earlier route (E1). […] typed revision vs. untargeted resampling differs by only 4 failed shots out of 88, while GC changes from 0.647 to 0.692 at equal time, which is hard to explain.

A4. We largely agree and revise the claims accordingly (General Response). Contracts: Appendix E reports object-state fidelity of 0.743 with contracts against 0.721 with prose, and GC of 0.692 against 0.683. We describe contracts as making object hand-offs checkable before rendering, without attributing the overall consistency gain to them. Step tiers save compute but do not beat a uniform schedule at similar spend (A7). Typed revision is addressed in A6.

AuthorsOpenReview ↗

**Response to Reviewer nwDW (3/4)**

Q6 (Q2). Typed revision and its per-shot breakdown

The reviser fixed only 2 of 47 failures, and only about 40 of 437 shots fail at all. How can typed revision vs untargeted resampling change GC by 0.045? Can you show the per-shot breakdown?

A6. The "2 of 47" count is from the earlier route E1, where identity was a gate. In R1, 23 of 40 failed first attempts passed after seed-based repairs (Appendix E). R1 recomposes the positive prompt from the object plan at every attempt, so the rewritten positive prompt never reaches H3. Table 7 retains the 0.645/0.647 GC rows as R1 seed-based controls; the newer R7 comparison passes typed rewrites directly to H3. The submitted default-versus-one-take contrast is 0.692 versus 0.645, a 0.047 GC gap. In the three-seed rerun, the seed-based-repair reference versus one take differs by 0.002 GC; typed revision versus one take differs by 0.004 GC. We therefore do not use the submitted gap as evidence of a GC gain from typed rewriting.

The arms also differ on more shots than the failed ones. A failed take that is kept becomes the first frame of the next continuation, so the next shots in that scene are rendered from a different input; Table 31 shows five shots that pass on their first attempt in every arm yet differ, each right after a changed take. This cascade is why first attempts differ across arms in the re-run below (14 vs. 15) even though plans and seeds are shared, and it is why the submitted Table 4's GC differences between revision arms cannot be read as the effect of the rewrite alone.

Per-shot breakdown from a three-seed re-run on six stories (51 shots per seed), with the same frame-quality and temporal checks in every arm:

Typed revisionUntargeted resamplingOne take
Failed first attempts ↓ (of 3 × 51)141415
Repaired shots that passed ↑ (at the same seed)12 (5)11 (0)0 (0)
Failed shots delivered ↓2315
H3 calls ↓174171153

Revision of either kind cuts delivered failures from 15 to 2 or 3. Typed revision and untargeted resampling deliver nearly the same number of failures; the rewrite's own contribution is the 5 shots it fixes at the same seed. We credit verification-triggered retries with fewer failed shots, without claiming superiority of typed rewriting over untargeted resampling or attributing the 0.692 vs. 0.647 GC difference to typed rewriting.

Revision. Section 4.4 summarizes the result and revises the claim; Table 3 (Appendix A.1) gives this table; Appendix E gives the cascade explanation, per-story counts for each seed (Table 11) and the E1 count, labeled as the earlier route; Table 31 (Appendix O) lists every shot whose delivered video differs between arms.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: revised manuscript uploaded; source of the gain, revised claims and new experiments

We have uploaded a revised manuscript that addresses the reviewers' concerns; all revised text is shown in blue. Each reviewer's reply ends with a table listing every change made for that reviewer and where it appears in the revised PDF.

Table, figure, section and appendix numbers refer to the revised PDF; quoted review text keeps the submitted numbers. Replies to each reviewer follow under their reviews.

We thank the reviewers. All three note that the comparison is controlled: every system shares the planner (Qwen3.8-27B), FLUX.2-klein portrait generation and the video generator (MiniMax-H3, "H3"); keyframe sources follow each system, including StoryDiffusion for MovieAgent (Section 4.1), and the gains agree across ViCLIP, DINOv2, ArcFace and a human study. The shared concerns were (i) where the 0.166 GC gain over ViMax comes from, (ii) whether the components listed as contributions are supported, and (iii) baseline fairness. We ran new experiments for each and revise the paper's claims to match what they show. GC is global ViCLIP consistency; face identity is the ArcFace measure.

What the new experiments show

  • The shot plan is the main source of the gain, and it transfers. Swapping only the plan raises GC by 0.106 to 0.124 under either executor; swapping only the executor adds 0.019 to 0.089.
  • The gain is not an artifact of repeated content. It holds on the benchmark's own script (+0.126 GC, 11 stories), on medium-to-long shot pairs from different locations (+0.135 GC), and over three seeds of every system on five stories (+0.180 over ViMax, +0.195 over MovieAgent).
  • CANVAS's executor adds face identity on the same plans: +0.079 to +0.184 over the baselines' executors.
  • The step tiers save 24% of the denoising at equal GC, but a uniform schedule at similar planned spend does equally well. The saving comes from H3 keeping its quality at about 16 steps, not from the difficulty score, and we revise the claim accordingly.

Plan swap. Each cell is the GC of one plan and executor pair, averaged over the stories that have all four films (four for ViMax; five for MovieAgent). The baseline references in these grids belong to the crossed-plan comparison; Table 29 uses the benchmark baseline references.

GC ↑ViMax executorCANVAS executor
ViMax plan0.4810.518
CANVAS plan0.6050.624
GC ↑MovieAgent executorCANVAS executor
MovieAgent plan0.4560.545
CANVAS plan0.5800.653

Down a column only the plan changes (+0.106 to +0.124). Across a row only the executor changes (+0.019 to +0.037 over ViMax's, +0.073 to +0.089 over MovieAgent's); CANVAS's executor also raises face identity (+0.079 to +0.184). On all ten stories where we ran the baselines' executors on CANVAS's plans, their GC rose in every story.

How the claims change (abstract, Section 1 contributions, Section 5)

  1. Main contribution: the compiled executable shot plan. Pixels pass only within continuous action; identity packs and text state carry identity and story across cuts. Appendix M attributes the gain; Table 4 (Appendix A.1) gives the grids.
  2. Step tiers: reported as an efficiency finding (0.76 of the reference cost B on average). The abstract, Section 1 and Section 3.6 consistently call this content-aware compute allocation: a fixed text-complexity rule sets each shot’s denoising steps. The paper explicitly states that uniform steps at similar planned spend perform equally well. We remove claims of an allocation advantage, online adaptivity, or a hard film-level budget.
  3. Object contracts: a pre-render validity check on object hand-offs, not a source of consistency.
  4. Verification-triggered retries: typed revision and untargeted resampling reduce delivered failures relative to one take, without an established advantage of typed rewriting over resampling (nwDW A6).

Other revisions.

  • Three seeds for every system on five stories (Table 9b).
  • The verifier's formulas (Appendix B).
  • One definition of B (Section 3.6).
  • A stated scope for every timing figure (Section 4.3, Appendix D).
  • GroundShot discussed in Section 2 and Appendix L.
  • Per-story repair, executor-swap and text-control results (Tables 10, 11, 29–31).
  • Per-story GC for all 35 stories (Table 32).
  • Clustered human-study intervals (Table 33, Appendix P).
  • Corrected Figure 1 caption and no-chaining value (Section 4.4, Table 7).
  • Limitations and future work (Section 5).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.