acceptodds
Under review as a conference paper at ICLR 2027

Découpage: Directorial Audio-Visual Control for Long Multi-Shot Video Generation

Abstract

In cinematic storytelling, sound and visual transitions do not always coincide in lockstep: dialogue frequently bridges across cuts to reveal reactions or introduces new scenes via J- and L-cuts, while deliberate silences pace the scene. Existing audio-visual generative frameworks, however, lack explicit mechanisms to coordinate independent visual and acoustic timing over extended horizons. As narrative duration and shot complexity scale, these systems struggle to maintain long-range dialogue pacing—often hallucinating speech during intended silences or erroneously driving mouth movements on silent listeners. To enable fine-grained cinematic control, we present Découpage, a screenplay-driven framework for long-form, multi-shot, and multi-character joint audio-video generation. Rather than coupling modality boundaries by default, Découpage uncouples generation into independently bounded visual and acoustic tracks governed by persistent character identities. Découpage builds on a joint audio-video diffusion model with dual-schedule temporal cross-attention, incorporating a directorial Lip-sync mechanism that selectively decouples speech conditioning from visual facial dynamics during off-screen dialogue. On multi-shot conversational benchmarks, Découpage achieves precise control over shot transitions, audio timing, and visibility-aware lip synchronization, directing 15- to 60-second screenplay-driven audio-video generation with multimodal coherence.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Audio-Visual Generation, AIGC

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ktHNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies joint audio-video generation of multi-shot, multi-character dialogue scenes, aiming to let visual cuts and dialogue timing be controlled independently. The screenplay is represented as a visual track and an acoustic track on a shared clock, and LTX-2.3 is fine-tuned with LoRA. Two modules are proposed: (1) dual-schedule temporal cross-attention and (2) a training-free directorial lip-sync mechanism. On self-constructed 15-second and 60-second screenplay benchmarks, the method outperforms LTX-2.3, MiniMax-H3, and MOVA in cut accuracy, dialogue timing, and silence preservation, and an ablation shows that the lip-sync mechanism reduces spurious lip coupling on listeners.

Critical strengths
  • A problem setting close to real film language. Independent cuts and dialogue, reaction shots, and deliberate silences are aspects that existing joint generators rarely control explicitly.
  • Clear timing-control gains with low overhead. Removing the dual-schedule masks lowers Cut F1 from 0.99 to 0.34 and Containment from 92.83% to 39.34%, and the lip-sync mechanism adds very little runtime.
Critical weaknesses
  • Restricting text cross-attention by shot interval is already common in multi-shot video generation (e.g., HoloCine, which the paper cites), and the paper mainly applies the same idea to the audio branch according to dialogue timestamps. MAVIN, also cited, already routes text cross-attention by both shot and dialogue intervals in a joint audio-video model, but the paper does not discuss this routing design in relation to it. The authors should discuss the relation to these methods and clarify the differences.
  • Simpler ways exist to keep a visible listener from lip-syncing to off-screen speech, such as stating in the prompt that the line is off-screen (as the official MiniMax-H3 prompting guide prescribes) or disabling audio-to-video cross-attention within that interval. The "Skip A2V" variant appears only as one qualitative example in the supplementary material, and Table 2 has no quantitative comparison with either alternative. In addition, the claim of eliminating spurious lip-sync on listeners rests only on the new TemporalBias metric, which is not reported for the baselines or real videos and is not validated by human evaluation.
  • The paper does not state where the training data and the 400 test screenplays come from, or whether the training and test sets are split by film. Moreover, the model is fine-tuned on the same timestamped screenplay format used at test time, whereas the baselines receive it zero-shot, and how the screenplay is converted into each baseline's input is not explained. It is therefore unclear how much of the large margin in Table 1 comes from the method itself.
Questions for authors
  1. Could the authors quantitatively compare the lip-sync mechanism with (a) stating in the prompt that the line is off-screen and the listener keeps listening, and (b) disabling A2V cross-attention within off-screen intervals (the Skip A2V variant)?
  2. Where do the training data and test screenplays come from, and are the training and test sets split by film?
  3. Could the authors report TemporalBias for the baselines and for real videos?
  4. In Table 2, is "w/o Temporal CA" a retrained model or the same checkpoint with masks removed at inference?
  5. How is the screenplay converted into each baseline's input? MiniMax-H3 outputs are only 14.375 seconds long; how is the missing part handled in evaluation?
4 comments

We sincerely thank Reviewer ktHN for appreciating our realistic film-language problem setting and recognizing the clear timing-control gains and minimal runtime cost of our dual-schedule and lip-sync mechanisms.

W1 — Relationship to HoloCine and concurrent work MAVIN.

Découpage is designed around the core grammar of film editing over extended 15- to 60-second horizons: treating the visual cut list and the acoustic timeline as two independently scheduled tracks on a shared clock to explicitly orchestrate J-cuts, L-cuts, reaction shots, and scripted silences.

  • Comparison with HoloCine: While we follow HoloCine's contiguous-shot sequence construction, HoloCine is video-only. It lacks an audio generation stream and cannot model an editable soundtrack, sound bridges across cuts, or audio-visual synchronization.
  • Comparison with MAVIN (Concurrent work): First, we note that MAVIN is a concurrent paper (released shortly before submission) that we cited for completeness. Second, Découpage goes significantly beyond MAVIN in three key aspects:
    1. Audio-Video Track Formulation & Explicit Silence Mode: MAVIN focuses on active dialogue routing without explicitly modeling scripted silences. In joint AV backbones, leaving non-dialogue intervals unconstrained causes hallucinated speech (18.69%–28.47% silence leakage in baselines, Table 1). By explicitly partitioning the acoustic track SA\mathcal{S}_A into character-tagged utterances and dedicated <NO SPEECH> intervals, Découpage reduces silence leakage to 1.12%.
    2. Directional Lip-Sync Mechanism: MAVIN only routes text-level conditioning, leaving internal Audio-to-Video (A2V) cross-attention symmetrically coupled across frames. As our ablations (w/o Mechanism and Prompt-only) show in the general response, text routing alone cannot prevent active speech features in A2V cross-attention from driving a visible listener's mouth during an L-cut or reaction shot. Découpage leverages its explicit <NO SPEECH> mode to construct a paired non-speech trajectory at inference, substituting the A2V residual whenever the active speaker is off screen.
    3. Scaling to Minute-Long (15s–60s) Multi-Shot Narratives: We explore and benchmark directorial audio-visual control up to full 60-second multi-shot conversations. Over minute-long horizons where baseline timing collapses, Découpage sustains 0.76 Cut F1, 91.12% containment, 4.46% leakage, and strong multi-turn voice consistency.

W2 and Q1 — Controlled comparison with Prompt-only and Skip A2V, plus human evaluation.

Holding the checkpoint, audio prompt, seeds, and sampling settings fixed on our evaluation subset, we evaluated both alternatives alongside a blinded user study (Tables A and B in the General Response):

  • (a) Why Prompt-only fails: Explicitly instructing the visible character to listen silently in the text prompt (while keeping native A2V guidance) only marginally changes TemporalBias from 1.78 (w/o directorial control) to 1.64. Because each DiT block applies A2V cross-attention immediately after video-text cross-attention (Fig. 3a), text-level instructions cannot block the strong speech conditioning signal in the audio branch from attending to and driving visible lip tokens.
  • (b) Why Skip A2V falls short: Disabling A2V updates during off-screen intervals (Skip A2V) lowers TemporalBias to 0.55, whereas our <NO SPEECH> residual reduces it further to 0.33. Simply skipping A2V completely severs cross-modal conditioning during off-screen intervals, perturbing latent continuity across boundaries and degrading control over natural listener reactions. In our blinded user study (Table B), raters preferred Découpage over both Skip A2V and w/o control by wide margins on plausible lip motion (66% vs. 12% and 7%) and natural listener reactions (58% vs. 12% and 12%). Comparison videos are included in the updated supplement.

Q3 — TemporalBias on real videos and why it is inapplicable to unaligned baselines.

Evaluating 200 randomly sampled real 15-second film clips from our curation pipeline, listener/reaction windows achieve a low TemporalBias of 0.47, reflecting that real listeners' facial dynamics are not time-locked to off-screen syllables. Our mechanism reduces TemporalBias from 1.78 down to 0.33, matching the low-coupling regime of real film clips (0.47). In contrast, computing script-window TemporalBias on the Table 1 baselines is ill-posed because the metric presupposes that the generator actually transitions to the listener shot and keeps the dialogue within the scheduled interval. Whereas Découpage satisfies both prerequisites (0.99 Cut F1, 93.01% audio containment), baselines fail to control shot cuts and dialogue timing, so target screenplay intervals capture missed cuts or drifted audio rather than true reaction shots.

W3, Q2, and Q4 — Data provenance, adaptation fairness, and “w/o Temporal CA”.

  • Data provenance and unseen test screenplays (Q2): We curated ~30,000 15s and ~30,000 60s training sequences by merging adjacent movie shots with native audio, using Faster-Whisper for speech timestamps and Gemini 3.0 Pro for dual-track annotations. Our evaluation benchmark comprises 400 screenplays in total — 200 15s and 200 60s sequences. Each duration subset consists of 100 pipeline-curated sequences randomly held out (never seen in training) and 100 LLM-authored screenplays (ChatGPT) that have zero source-film overlap with the training corpus by construction. Découpage performs consistently across both duration subsets.
  • Why gains stem from our mechanism rather than format adaptation (W3 & Q4): To answer Q4 directly: the “w/o Temporal CA” row in Table 2 uses a model retrained on the exact same 30,000 sequences and screenplay format, but without dual-schedule temporal masking. Despite full adaptation to our training data and prompt format, this retrained model achieves only 0.34 Cut F1 and 39.34% Containment — on par with the zero-shot baselines (0.18–0.28 Cut F1, 27.14%–52.68% Containment) — whereas adding Dual-Schedule Temporal CA boosts performance to 0.99 Cut F1 and 92.83% Containment. This confirms that the large margin in Table 1 comes from our proposed Dual-Schedule Cross-Attention mechanism rather than format fine-tuning.

Q5 — Baseline input conversion and MiniMax-H3 duration.

For baseline evaluation, each structured screenplay was converted into a chronological text prompt containing all visual shot descriptions, shot positions, timed dialogue lines, and sound cues for LTX-2.3 and MiniMax-H3 (and paired with the initial reference frame for MOVA I2VA). Regarding the 14.375 s duration of MiniMax-H3 in our initial submission: first, our benchmark metrics are inherently robust to minor tail-duration differences (0.625 s), as internal shot transitions (matched within a 0.5 s tolerance), scheduled dialogue turns, and scorable Sync-C face windows lie within the first 14.375 s. Second, we sincerely thank the reviewer for catching this frame-count configuration bug. We re-ran MiniMax-H3 under the exact same settings to generate full 15.0 s outputs across the 200-screenplay set and updated Table 1 in the paper and the comparison videos in the supplementary materials. As expected, the resulting scores remain virtually unchanged (e.g., H3 Cut F1 is 0.28 at full 15.0 s vs. 0.29 at 14.375 s, compared to our 0.99), confirming that this slight duration offset does not affect our conclusions.

Thank you for the detailed response and the additional experiments, which clarify the differences from HoloCine and MAVIN and the effect of the lip-sync mechanism. I agree that the proposed mechanisms are simple and lightweight, which is a practical advantage. However, I am not fully convinced that its technical contribution is sufficient, so I will keep my rating.

Thank you for considering our response and additional experiments, and for recognizing the practical advantages of our lightweight design. Our contribution centers on a dual-track formulation that makes the temporal organization and cross-modal relationships of an audiovisual narrative explicitly controllable. Independently timed visual and acoustic tracks, linked by a shared timeline and persistent character identities, allow dialogue to continue across visual transitions while the visible characters and their narrative roles change. This formulation exposes two complementary control requirements. The generator is required to follow the independently specified visual and acoustic schedules, and regulate how ongoing speech influences visible articulation as speaker visibility changes. We realize these requirements through dual-schedule temporal cross-attention and visibility-conditioned A2V residual substitution using a paired non-speech trajectory. The additional controlled comparisons and blinded human evaluation support the effectiveness of the visibility-conditioned intervention. We believe this formulation, its associated mechanisms, and the demonstrated directorial capabilities together constitute a meaningful contribution to controllable audio-visual generation. We appreciate your time and constructive feedback.

Reviewer_ocCmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes decoupage, a framework to address specific problems in joint audio-visual generators: Visual cuts and dialogue need independent timing. Decoupage consist of dual-track screenplay, dual-schedule temporal cross-attention and a training-free directional lip-sync mechanism. Evaluation on the author constructed benchmark yield consistent improvement and provided demos present better generation quality.yi

Critical strengths
  1. A clear, practically relevant problem with a clean formulation. J/L-cuts, reaction shots and scripted silences are real cinematic needs that current joint AV generators handle poorly. Splitting the screenplay into independently timed visual and acoustic tracks on a shared clock is a simple, intuitive interface for directorial control.

  2. Simple, lightweight mechanisms and clear gains: Proposed solution is clean and straightforward, yet shows large gains on controllability, especially on longer evaluation.

Critical weaknesses
  1. TemporalBias alone does not support the claim that natural listener behavior is preserved. A low score can also come from static or unnatural faces. It shows weaker audio coupling, but not that expressions and reactions are kept, which the paper claims (L287) without any human evaluation. A user study on unintended articulation, naturalness, and reaction appropriateness is needed.

  2. The lip-sync mechanism works at the shot level, not per character. By Eqs. (5)–(6), native A2V coupling is kept whenever the speaker is visible in the shot (L312–313), so a silent listener in a two-shot is not protected, even in plain turn-taking dialogue. This is separate from the overlapping-speech limitation. Conversely, replacing A2V residuals over the whole frame during off-screen speech may also suppress non-speech, sound-driven actions. Evaluations on shared-frame dialogue and such scenes could enhance the claim.

Questions for authors

See weakness above.

2 comments

We sincerely thank Reviewer ocCm for highlighting our clean dual-track formulation for J- and L-cuts, reaction shots, and scripted silences, as well as the simplicity and strong long-horizon controllability gains of our mechanisms.

W1 — TemporalBias scope and blinded user study on listener behavior.

To directly verify that reducing TemporalBias from 1.78 to 0.33 suppresses spurious articulation while preserving lively, contextually appropriate reactions (rather than producing static or unnatural faces), we conducted a blinded user study across matched reaction shots comparing w/o directorial control, Skip A2V, and Ours (Table B in the General Response). Raters evaluated (1) Plausible Listener Lip Motion (whether the listener avoids mouthing the off-screen dialogue while allowing natural non-speech mouth expressions such as laughter in Fig. 6) and (2) Natural Listener Reaction. Découpage was preferred in 66% of comparisons for plausible lip motion (vs. 12% for Skip A2V, 7% for w/o control, and 15% draws) and in 58% of comparisons for natural listener reactions (vs. 12% for Skip A2V and 12% for w/o control, and 18% draws).

W2 — Shared-frame two-shots and concurrent sound-driven actions.

  • Shared-frame turn-taking dialogue (two-shots): As noted, Eqs. (5)–(6) operate at the shot level (α(p)=0\alpha(p)=0 when V(p)=1V(p)=1) because spurious listener articulation primarily arises in isolated reaction shots (V(p)=0V(p)=0), where the active speaker is entirely off screen and the model's A2V attention falsely latches onto the only visible face. In contrast, when both the speaker and listener share a two-shot (V(p)=1V(p)=1), our dual-track text conditioning already binds actions and utterances to explicit character tags (e.g., the shot and audio tracks specify that [character1] speaks while [character2] listens), and the visible speaker's face directly absorbs the A2V alignment. As seen in the multi-character clips in our supplementary material, the model knows from the text which character is speaking, allowing the on-screen speaker to articulate cleanly while the co-visible listener remains naturally quiet.

  • Non-speech, sound-driven actions during off-screen speech: Replacing rnative(l)r_{\text{native}}^{(l)} with rns(l)r_{\text{ns}}^{(l)} during off-screen speech (α(p)=1\alpha(p)=1) neither suppresses visible actions nor loses their synchronized sound effects, for two architectural reasons: (i) on the A2V path, the auxiliary trajectory only substitutes the speech transcript with <NO SPEECH> while retaining the full [sound global caption] (encoding foley, ambient sounds, and acoustic events) and shared initial noise, while the visible action itself is directly guided by the shot caption ciVc_i^V; and (ii) on the V2A path, our intervention only modifies the A2V update sent to the video stream while the main audio branch retains its native bidirectional V2A cross-attention (Fig. 3a, Sec. 3.3). Consequently, visible actions performed by a listener during off-screen dialogue continue to attend to and trigger synchronized foley/sound effects in the main audio track (as shown in our supplementary examples).

Dear Reviewer ocCm, Thank you again for your comprehensive review and constructive suggestions. Following your suggestion, we added a blinded human evaluation of listener lip motion and natural reactions, with results reported in Table B of the General Response. We also clarified the shot-level scope of the intervention and discussed shared-frame dialogue and non-speech sound-driven actions, supported by additional supplementary examples. We would appreciate your assessment of whether this evidence and the clarifications address your concerns, and any feedback on remaining issues. Thank you for your time and suggestions.

Reviewer_oQmvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper describes two separate and actually independent skills: performing directorial cuts on both timeline/shots and character lip control on both visual and acoustic tracks. To achieve that, (1) they applied dual-track temporal cross-attention, where, for each modality, a timeline-controlled attention mask is applied to make sure video or audio can strictly follow the defined timeline; (2) they applied an "if-else" mechanism to enable train-free lip-sync control on the video generation based on whether the speaker is in the planned script or not.

The results are verified with video/audio quality scores, mostly focused on the consistency and alignment between video and audio, or between video/audio and the reference screenplay.

Critical strengths

The paper focused on an issue of cinematic video generation, which a lot of users will require during video generation, but was not well addressed in traditional video generation benchmarks. The authors identify an important part in the problem, which is the sync between audio and video, and between audio/video and the screenplay. Without significant training resources, they applied simple and straightforward methods to ensure the alignment and work out multiple metrics to evaluate it for other works in the future.

Critical weaknesses
  1. The ability of the alignment between video/audio and screenplay and the lip-sync control depends highly on the screenplay details with the timestamp and the character labels on the soundtrack. During training, it is generated by Gemini 3.0 Pro but it is hard for amateur end users to provide screenplay in such a format during inference. Lacking a module to provide a text-to-screenplay function, the current framework may be less user-friendly than expected.
  2. Something is lacking in the qualitative evaluation. (1) In the table 1, it shows that your model's lip-sync score is lower than Minimax baseline, but the qualitative examples lack any failure analysis targeting that. (2) It lacks cinematic scenes with "multi-character" generation, and in general, the examples in the paper are not complex enough to show the advantage of your model over the baseline. While the comparison shows your model can control video/audio within each boundary, it fails to show the baseline's flaws on lip-sync (except for MOVA) and acoustic track control(except for Minimax), and it is hard for readers to visually verify whether those baselines actually suffer from the "unintended listener articulation" or "hallucinated speech" problems that the paper claims to solve.
Questions for authors
  1. Is it possible to provide more qualitative examples based on the "Critical Weaknesses" section?
  2. The paper did show the ability to map the structured screenplay to cinematic-level camera/audio/timing control and transfer the intelligence of the screenplay text to the video generation result, but have the authors explored other areas to apply this method to cinematic storytelling, like camera/light or other aesthetic areas, other than just video-audio sync?
2 comments

We sincerely thank Reviewer oQmv for recognizing the practical importance of cinematic audio-video-screenplay alignment, our straightforward and resource-efficient design, and our evaluation metrics for future research.

W1 — From a short brief to an editable screenplay.

Rather than requiring users to write timestamps and character tags by hand, an LLM front end directly converts a short brief (e.g., “Cut to the witness while the detective finishes the question”) into our two-track format, placing the visual cut at 3.0 s while extending the detective's line to 4.5 s. In fact, 200 of our 400 evaluation screenplays were generated via an LLM (ChatGPT). Using these LLM-generated screenplays, amateur users can effortlessly generate multi-shot films while preserving an editable timeline for filmmakers to fine-tune cuts and dialogue overlaps.

W2 and Q1 — Lip synchronization and qualitative comparisons.

We acknowledge that as a proprietary commercial system, MiniMax-H3 delivers slightly sharper single-shot visual fidelity and stronger raw lip-sync confidence, whereas Découpage excels at multi-shot directorial control, achieving 0.99 vs. 0.28 Cut F1, 93.01% vs. 41.95% dialogue containment, and 1.12% vs. 27.28% silence leakage (Table 1). To transparently analyze the Sync-C gap, we provide side-by-side playback with audio for two jointly scorable speaking intervals where Découpage scores 3.37 and 3.98 versus 7.14 and 7.34 for H3. Visual inspection shows that Découpage produces natural, visually satisfying lip synchronization across these clips. H3 tends to generate crisp, frontal talking heads that heavily boost SyncNet confidence without executing requested cuts, whereas Découpage prioritizes multi-shot cinematic framing with more restrained, conversational mouth movement. Furthermore, to address W2(2), our updated supplementary archive includes complex multi-character conversational scenes (four characters alternating turns with J- and L-cuts across multiple listeners).

Q2 — Camera and lighting control.

Yes. Because our Dual-Schedule Cross-Attention (Eqs. 1–2) isolates visual shot intervals IiVI_i^V from acoustic intervals IjAI_j^A, filmmakers can direct camera framing, movement, and lighting per shot without disturbing ongoing dialogue. In Fig. 4 in the paper, for example, the scene maintains global green-tinted lighting while cutting from a close-up of the speaker to an over-the-shoulder view of the listener mid-sentence. Directors can likewise specify shot-level camera motion (e.g., a slow push-in) or a lighting shift at any cut while keeping the acoustic track continuous.

Dear Reviewer oQmv,

Thank you again for your constructive feedback. We would appreciate your thoughts on our response and updated supplementary material, which includes four-character conversational examples and side-by-side audiovisual comparisons addressing your qualitative evaluation concerns. We also clarified the use of LLM-generated screenplays to simplify the structured input.

As the discussion period draws to a close, could you let us know whether these additions address your concerns, and whether any remaining issues would benefit from further clarification?

Thank you for your time and consideration.

Comments on the paper

1 comment

We thank the reviewers for recognizing the core contribution of Découpage: enabling directorial timing and visibility-aware lip-sync control in multi-shot audio-video generation over 15- to 60-second horizons. Découpage pairs (1) Dual-Schedule Temporal Cross-Attention for shot and dialogue timing (0.99 Cut F1, 93.01% containment on 15s; 0.76 Cut F1, 91.12% containment on 60s) with (2) our training-free Directorial Lip-Sync Mechanism, which regulates audio-to-video (A2V) residuals so off-screen speech does not drive a visible listener's mouth. Below we summarize our new experiments and clarifications:

  1. Validating Directorial Lip-Sync Control via Ablations & Blinded User Study:

    To verify that our Directorial Lip-Sync Mechanism suppresses spurious off-screen lip-sync while preserving lively, contextually appropriate facial expressions, we conducted controlled mechanism ablations (Table A) alongside a blinded human evaluation (Table B) over matched reaction shots within the whole videos. Because baselines fail to follow the schedule, slicing their outputs at screenplay timestamps captures missed cuts or drifted audio rather than valid reaction shots.

    Under controlled schedules (holding checkpoint and seeds fixed), text-level Prompt-only instructions fail to stop downstream A2V speech features from driving visible lips (1.64 vs. 1.78 w/o control, Table A). At the feature level, Skip A2V removes cross-modal updates entirely (0.55), whereas our <NO SPEECH> trajectory supplies an in-distribution quiescent residual that achieves the lowest TemporalBias (Table A) and is strongly preferred in our blinded user study (Table B):

    Table A: Objective TemporalBias (↓) on controlled listener windows.

    Metricw/o directorial controlPrompt-onlySkip A2VOurs
    TemporalBias (Listener) ↓1.781.640.550.33

    Table B: Blinded user study win rates (↑) on matched reaction shots.

    Evaluation Criterionw/o directorial controlSkip A2VOursDraw
    Plausible Listener Lip Motion ↑7%12%66%15%
    Natural Listener Reaction ↑12%12%58%18%
  2. Dataset Provenance (30k 15s & 30k 60s) and Unseen Test Suite:

    We curated ~30,000 15s and ~30,000 60s training clips by merging adjacent movie shots annotated via Faster-Whisper and Gemini 3.0 Pro (Sec. 3.1). Our 400 test screenplays consist of 200 held-out curated sequences (never seen in training) and 200 LLM-authored screenplays (ChatGPT) that have zero source-film overlap and illustrate automated text-to-screenplay authoring. Retraining on the same 30k dataset without dual-schedule masks (“w/o Temporal CA”, Table 2) drops Cut F1 from 0.99 to 0.34, proving our gains come from the mechanism rather than format adaptation.

  3. Updated Supplementary Archive:

    Includes side-by-side videos for w/o directorial control, Prompt-only, Skip A2V, and Ours; full 15.0 s MiniMax-H3 comparisons with waveform/callout overlays; and additional multi-character results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.