acceptodds
Under review as a conference paper at ICLR 2027

BEAT: Rhythm-Elastic Alignment for Agentic Music-guided Movie Trailer Generation

Abstract

Automatic movie trailer generation selects shots from a full-length film and synchronizes them with chosen music. Existing methods either relegate music alignment to post-processing or enforce rigid one-to-one shot-music mappings, overlooking that professional editing rhythm is elastic: energetic passages favor rapid cuts, while quieter passages allow shots to span multiple bars. We introduce BEAT, a framework that addresses this gap with two core components: MuVA, a compact music-visual alignment encoder trained with Sinkhorn-regularized two-stage learning, and Bar-DP, an energy-adaptive dynamic programming algorithm that produces elastic many-to-one alignments. These components are integrated into a five-phase agentic pipeline that grounds the core alignment in learned cross-modal features while coordinating higher-level creative decisions through structured text signals. To support comprehensive evaluation, we also introduce TrailerArena, a benchmark with 20+ metrics across four complementary dimensions. On TrailerArena, BEAT achieves state-of-the-art performance across shot selection, ordering, and perceptual quality, while producing fully composed trailers end-to-end.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video Editing, Music-Visual Alignment, Movie Trailer Generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_rNi3OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces a novel framework for music-conditioned movie trailer generation, and a benchmark with more than 20 metrics. Using the benchmark, the authors compared results of existing approaches.

Critical strengths
  1. The metrics are comprehensive and the benchmark has been carefully constructed.
  2. The paper is well-written and the presentation is clear (including the motivation, algorithm)
  3. The suggested framework for trailer generation is novel.
Critical weaknesses
  1. In Reproducibility statement, it is unclear whether the authors will release code to reproduce metrics. Given that the benchmark is the authors' primary contribution, it seems essential.

  2. Given the instability of generative models' objective evaluation, a demo webpage looks essential. A subjective evaluation will also strengthen its claim.

Questions for authors
  1. Will the authors release the code to reproduce metrics?
  2. Can the authors provide demo?

The paper is solid and well-structured, but it is difficult to recommend acceptance without reproducible code for benchmark as well as subjectively convincing results, such as demo.

3 comments

Thank you for addressing my concerns thoroughly. I'll adjust my score and am more confident in recommending acceptance based on additional information provided by the authors.

Thank you for reviewing our response and updating your assessment. We are glad your concerns have been addressed and appreciate your helpful feedback. We will make sure the code and demo are included with the final version.

Thank you again for your willingness to adjust the score. As the discussion period is closing soon, we just wanted to gently mention that the rating in the review form still appears unchanged on our side, in case the update has not gone through yet. We truly appreciate your time and support!

Reviewer_LjftOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes BEAT, a framework for music-guided movie trailer generation that takes a full-length movie and a piece of trailer music, selects shots from the movie, and arranges them to fit the music. Its two core components are MuVA and Bar-DP. MuVA is a small Transformer with cross-attention that scores how well each music bar matches each movie shot from frozen CLAP audio features and ImageBind visual features. Bar-DP is a hand-designed beam search that combines these scores with per-bar RMS energy to produce many-to-one assignments. Both components sit inside a five-phase agentic pipeline covering music and shot analysis, spoiler filtering, shot selection, composition with generated text cards, and a VLM critic that triggers re-selection.

Critical strengths
  1. The paper is clearly presented and extensively evaluated, with retrieval experiments and ablations covering training choices and pipeline components. The appendix also reports unfavorable results and clarifies the evaluated configurations, making the findings easier to interpret.

  2. Separating learned shot compatibility from explicit timing control is a useful design. TrailerArena also extends evaluation beyond shot overlap to visual similarity, temporal structure, and perceptual quality.

Critical weaknesses
  1. The paper does not directly show how music affects the selected visual content. Learning from professional trailer pairings is reasonable, but a same-movie comparison with music of different moods is missing. It remains unclear whether music meaningfully changes scene selection beyond editing pace and visual energy.

  2. The shot-cropping procedure is underspecified. Bar-DP assigns a source shot and duration, but the paper does not explain which temporal window is retained. Cropping may omit key events, so the source-shot compatibility score may not represent the final excerpt. Shots shorter than an assigned bar also need clarification.

  3. The central elastic-selection contribution is not sufficiently isolated. The main comparison uses rigid greedy selection, while Bar-DP yields mixed results across metrics. The human study covers only two movies and does not directly compare Bar-DP with greedy selection under otherwise identical settings. Its perceptual benefit therefore remains uncertain.

Questions for authors
  1. For the same movie, does changing the music’s mood meaningfully change the selected scenes, beyond editing pace?
  2. How are shots cropped to fit their assigned duration? Does MuVA score the full shot or the retained excerpt?
  3. Which BEAT configuration was used in the human study? Was Bar-DP directly compared with greedy selection, keeping other components fixed?
  4. Can you provide more generated trailer examples across movies and music styles to show where BEAT works well and where it fails?
Reviewer_91wHOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

BEAT addresses music-guided movie trailer generation by selecting, ordering, and timing shots from full-length movies to create rhythmically synchronized, narratively coherent, and spoiler-free trailers. It proposes MuVA, a dual-stream music–visual alignment model trained with temporal alignment objectives and Sinkhorn regularization; Bar-DP, a beamsearch algorithm that jointly optimizes shot selection and variable-length shot durations to capture the elastic editing rhythm of professional trailers; and a five-phase agentic pipeline integrating shot analysis, spoiler filtering, alignment, trailer assembly, and iterative critique. The paper further introduces TrailerArena, a benchmark comprising 25 metrics across selection, ordering, composition, and perceptual quality. Experiments on Test-8 and Test-73 demonstrate competitive performance, with BEAT achieving the best results in shot selection, temporal composition (SDTW), and overall perceptual quality, alongside a user study showing significant improvements in rhythmic synchronization and engagement over five baselines.

Critical strengths
  1. MuVA learns bar–shot alignment through a dual-stream encoder that produces a music–visual compatibility matrix. It is trained in two stages, with Sinkhorn regularization in the second stage to encourage balanced alignment and prevent a few salient shots from dominating multiple bars.
  2. Bar-DP performs elastic many-to-one music–shot alignment via beam search, jointly optimizing shot selection and duration under editing constraints. An energy-adaptive cut bonus enables faster cuts in high-energy passages and longer shots in low-energy sections, capturing flexible trailer-editing rhythms with an inference time of under 0.3 seconds.
Critical weaknesses
  1. The paper's central hypothesis is that elastic many-to-one alignment better captures trailer editing rhythms than rigid one-to-one alignment. However, the main Table 2 results are obtained using MuVA with greedy, rigid one-to-one selection, rather than the proposed Bar-DP selector. More importantly, the independent beat-alignment evaluation in Appendix B.3 shows that MuVA-Greedy consistently achieves higher beat and downbeat F1 than both MuVA-BarDP and BEAT-Full, even outperforming competing baselines. While the user study reports preferences for the full pipeline, it does not isolate the contribution of elastic alignment, as other components such as text cards and transitions may account for the improvements.
  2. The perceptual comparison is potentially confounded by differences in trailer rendering. Metrics such as VL-Overall and AQ are computed on rendered trailers, but baselines such as IPOT, MMSC, SSMP, and PPBVAM are primarily shot-selection methods that do not inherently generate text cards or styled transitions. The paper does not clarify whether all methods are rendered using an identical assembly pipeline, making it difficult to attribute perceptual gains to the proposed alignment method.
  3. Several relevant agentic approaches, including TRAIL-DREAMS, CineAgents, and DIRECT, are discussed as motivation for BEAT but are not included in the experimental comparisons; M2T is also mentioned in Section 5.1 but omitted from Table 2. Moreover, despite criticizing prior agentic systems for their inference costs, the paper provides no end-to-end runtime comparison with these approaches.
Questions for authors
  1. Which configuration produces Table 2, and what does the full method score? Please confirm from the main text (not only Appendix A.4) that the BEAT row in Table 2 is MuVA + greedy rigid one-to-one selection with the critic and text cards off, and state whether MusicAnalyzer, NarrativeGuard, and energy-driven transitions are active.
  2. Appendix B.3 shows MuVA-Greedy beating MuVA-BarDP and BEAT-Full on every tracked beat/downbeat F1. How is this consistent with the paper's central claim? Please (a) report cut-density-normalized rhythm metrics, (b) report the distribution of selected span lengths k for Bar-DP versus the ground-truth trailers (do GT trailers actually show the hypothesized multi-bar spans at low energy?), and (c) run a human A/B in which rendering is identical and only the selection strategy (greedy vs. Bar-DP) varies, so that the rhythm preference can be attributed to elastic timing rather than to text cards or transitions.
  3. Please confirm movie-level disjointness between (i) the 870 Stage-2 fine-tuning pairs (472 MMSC + 398 CMTD), (ii) the ~4,500 V2M Stage-1 pairs, and (iii) Test-8 / Test-73; and confirm that all baselines were trained on their original training splits without access to the test movies.
  4. Were all methods' trailers rendered by the same assembler with the same transitions and no text cards? Please report results with the compositor disabled for all methods. Please also report VL-Overall variance across VLM seeds/temperatures, and re-score critic-enabled outputs with a different evaluator model (or with the human study) to decouple the critic from the metric.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Response to All Reviewers: Summary of Revisions

We are grateful to all reviewers for the constructive feedback. The questions and weaknesses raised in the reviews have improved the paper substantially.

We address every comment from each reviewer separately below. The revised version of the paper has been uploaded, and it mainly includes the following:

  • Clarifying the evaluation protocol, including the distinction between MuVA-Greedy, MuVA-BarDP, and the full BEAT pipeline, the modules enabled in each comparison, and the rendering procedures used for visual-quality evaluation.
  • Strengthening the analysis of elastic selection with matched-shot-budget controls and reference-free beat diagnostics, clarifying the trade-offs among shot coverage, cut density, alignment accuracy, and perceived rhythm.
  • Adding music-swap experiments that hold the movie fixed and vary the soundtrack, measuring changes in selected shot sets, bar-level assignments, and visual-feature distributions.
  • Adding independent evaluations using Qwen2.5-VL-7B and Gemma3-12B, together with a runtime breakdown that distinguishes cached-feature selection from full-pipeline generation.
  • Expanding and clarifying the human evaluation, including the 26-participant full-system study and a separate controlled comparison of Bar-DP and greedy selection with identical rendering and no text cards.
  • Improving the presentation of the evaluation metrics and experimental findings, bringing key diagnostics into the main text and expanding the discussion of component attribution, temporal evaluation limitations, and the generalizability of the human-study results.
  • Updating the anonymous project page with baseline comparisons, music-swap examples, paired elastic-versus-rigid edits, and representative failure cases, and clarifying our release plan for the TrailerArena evaluation code, MuVA, Bar-DP, the generation pipeline, and cached features to support reproducibility.

We have also refined the wording throughout the paper to align the claims more closely with the available evidence.

Dear Reviewers,

Thank you again for your time and thoughtful feedback. As the discussion period closes in less than 24 hours, we would like to kindly invite you to take a look at our responses. We have addressed every weakness and question raised in your reviews, and the revised paper reflects these changes.

If any concerns remain, or if further clarification or additional results would be helpful, please let us know and we will do our best to respond promptly before the deadline.

Best regards,

The Authors

AuthorsOpenReview ↗

Summary of Reviews and Author Responses

Dear Area Chair,

Thank you for handling our submission. As the discussion closes, we briefly summarize the reviews, our responses, and the discussion status.

Discussion status. The reviewers recognized the paper’s main contributions: the learned bar–shot alignment in MuVA and the elastic timing control in Bar-DP (Reviewers 91wH, Ljft), and the TrailerArena benchmark (Reviewers Ljft, rNi3). During the discussion, Reviewer rNi3 stated that our response addressed their concerns thoroughly and that they would raise their score, with greater confidence in recommending acceptance. Reviewer Ljft confirmed that the same-movie/different-music experiment addresses their main concern. The remaining reservation concerns the strength of the new evidence for Bar-DP’s benefit (8 test pairs and small gains in Gemma3 evaluation), and on this basis they maintain their score. Reviewer 91wH, whose initial rating was positive, has not yet joined the discussion.

Main concerns and our responses.

  • Isolating elastic selection (91wH, Ljft). Bar-DP’s lower F1 reflects the size of its selection rather than its accuracy. It selects 39.4 shots on average, against 87.2 for greedy selection, and does so with higher precision (0.282 vs. 0.217). When greedy selection is restricted to the same shot budget, its F1 falls below Bar-DP’s (0.190 vs. 0.217). To isolate the perceptual effect, we ran a controlled A/B study in which rendering is identical and only the selection strategy differs: 35 participants cast 216 of 280 votes (77.1%) for Bar-DP.
  • Effect of music on content (Ljft). Across 224 same-movie, different-music comparisons, the mean Jaccard overlap of selected shots is 0.456, showing that music changes the selected content and not only the pacing.
  • Evaluation protocol (91wH). We clarified the Table 2 configuration and that all methods share an identical assembly. Two evaluators independent of the critic (Qwen2.5-VL-7B, Gemma3-12B) both rank BEAT-Full first.
  • Reproducibility (rNi3, Ljft). We provided an anonymized repository with the evaluation code and a demo page with baseline comparisons, music-swap examples, and failure cases, and committed to a full code release.

Contributions. We would like to note that the paper’s contributions extend beyond Bar-DP. MuVA, the learned alignment model, accounts for the main quantitative gains: it leads shot selection metrics on both Test-8 and Test-73 using greedy selection alone. Bar-DP adds explicit, controllable elastic timing on top of it, with a moderate benefit in perceived rhythm. TrailerArena provides a 25-metric evaluation benchmark across four dimensions, which we will release with code to support future work on trailer generation.

We have revised the paper accordingly. We sincerely appreciate the reviewers’ feedback and your consideration.

Best wishes,

Authors of Submission 22617

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.