- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes StoryWeaver, a multi-shot audio-video generation framework built upon MOVA, aiming to improve cross-shot identity preservation, acoustic consistency, and audio-visual synchronization. The proposed Multi-Modal Sink Shot (MSS) mechanism combines windowed audio-visual cross-attention with sink-mediated inter-shot self-attention, while Audio-Visual Latent Stitching (AVLS) enables long-horizon continuation in the joint audio-video latent space. The model is trained on 350K multi-shot examples and evaluated on ST-Bench and the newly constructed WeaverBench, with component ablations, audio quality evaluations, long-horizon generation experiments, and human preference studies. The results demonstrate improvements in cross-shot consistency, identity preservation, and speaker consistency.
Critical strengths
- Task-driven architectural design for multi-shot audio-video generation. The proposed Multi-Modal Sink Shot (MSS) mechanism combines windowed audio-visual cross-attention for local synchronization with sink-mediated inter-shot self-attention for cross-shot consistency. This design enables the model to maintain character identity and acoustic consistency across shots while respecting shot boundaries and reducing the computational cost of dense global attention.
- Controlled ablation studies and comprehensive empirical evaluation. The paper provides controlled ablation studies of MSS and Audio-Visual Latent Stitching (AVLS) under matched backbone and training settings, demonstrating their contributions to cross-shot consistency and generation quality. Additional audio-specific evaluations, continuation strategy comparisons, long-horizon generation experiments, and human preference studies further support the effectiveness of the proposed framework.
Critical weaknesses
-
Limited comparison with strong audiovisual baselines. The main audiovisual evaluation compares only against StoryDiffusion + MOVA and IC-LoRA + MOVA. Although Appendix G includes LTX-2, it reports only two metrics and shows a modest synchronization improvement. Together with the 48.56% lip-sync preference against MOVA, this leaves the system’s advantages over strong contemporary audiovisual generators insufficiently established.
-
Insufficient qualitative comparisons. The supplementary videos showcase StoryWeaver without readily accessible, prompt-matched baseline outputs. This limits direct assessment of the claimed improvements in voice consistency and cross-shot transitions, which are difficult to judge from static figures and aggregate metrics alone.
My main concern is that comparisons with strong contemporary audiovisual generation systems are limited, making the method’s practical competitiveness difficult to assess. The small number of supplementary videos and lack of baseline videos further limit direct qualitative assessment. While the controlled MOVA experiments support the proposed components, evidence on other backbones remains preliminary, leaving it unclear whether these gains transfer reliably to stronger base models.
Questions for authors
- Could the authors clarify the baseline selection and the feasibility of comparison with stronger recent audiovisual generators, such as recent LTX variants or Seedance?
- Could the authors specify the LTX-2 checkpoint and adaptation budget, and report audio quality and speaker similarity to better assess generalization beyond MOVA?
- Given the modest synchronization gains, are the main benefits primarily cross-shot consistency? Could uncertainty estimates help clarify this distinction?
- Could the authors provide an anonymous comparison page with full, prompt-matched audiovisual outputs from StoryWeaver and the main baselines, including representative good examples and failures?