acceptodds
Under review as a conference paper at ICLR 2027

ReelOnce: Adaptive Reference Policy and Evaluation-Driven Hierarchical Refinement for Agentic Multi-Shot Story Video Generation

23 views

Abstract

Multi-shot story video generation requires shot-specific visual guidance while preserving consistency across characters, scenes, and narrative events, often requiring repeated manual revision. Existing workflows struggle to determine whether a problem originates in story planning, reference construction, or video generation. Fixed-stage retries are costly and may miss the root cause or disrupt content that is already correct. We present ReelOnce, an agentic framework that combines an adaptive reference policy with evaluation-driven hierarchical refinement. It selects an appropriate reference format for each shot, distinguishes the state depicted in a reference from the transition to be generated, and uses evaluation findings to identify the smallest necessary repair scope. Persistent story memory preserves narrative constraints and unaffected decisions throughout refinement. We further introduce Reconstructed Story Semantic Alignment (RSSA), which measures semantic consistency with the source story by reconstructing observable events from generated videos. On Pixar-style videos generated from 20 diverse stories, ReelOnce achieves strong RSSA performance while remaining competitive across VBench quality dimensions, demonstrating that the framework can balance story fidelity with visual quality.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
story-to-video generation, multi-shot video generation, agentic video generation, persistent story memory, adaptive reference policy, evaluation-driven hierarchical refinement

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_MJCFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper proposes a hierarchical repair framework for maintaining consistency in agent-based long video generation. It introduces an adaptive reference policy and an evaluation-driven refinement process to address narrative and visual inconsistencies without relying on costly, full-pipeline re-generation. Additionally, the authors propose a novel metric, Reconstructed Story Semantic Alignment (RSSA), to quantify the semantic fidelity of the generated videos against the source story.

Critical strengths
  1. Strong practical motivation & clear presentation: The paper addresses a highly relevant problem—repairing narrative and visual inconsistencies in long video generation efficiently. The proposed agentic pipeline is logically articulated, and the overall writing is of high quality.
Critical weaknesses
  1. Weak experimental validation for an engineering framework: The proposed hierarchical approach is primarily an engineering integration rather than a fundamental algorithmic innovation. As an engineering-focused system, it must strictly prove its practical value. However, despite claiming to solve the inefficiency of prior methods, it lacks quantitative comparisons of computational cost or runtime. Furthermore, the evaluation scale of merely 20 source stories is fundamentally too small to demonstrate generalization or practical reliability.

  2. Limitations in the RSSA evaluation design: While the motivation behind proposing the RSSA metric is meaningful, its specific design is inadequate. The metric's reliance on sparse frame extraction makes it difficult to guarantee that continuous temporal dynamics are accurately captured. This limitation significantly undermines the soundness of the narrative evaluation itself.

Questions for authors

Clarification Requests:

  1. Could you provide quantitative data comparing the computational cost and runtime against baseline methods to better understand the efficiency of your framework?

Suggestions for Future Work / Revisions:

  1. Expand the evaluation scale and test cross-model capabilities by using different foundation video models to better prove practical engineering value.
  2. Redesign the RSSA evaluation metric to better capture continuous temporal dynamics.
  3. Optimize the paper's layout.
Reviewer_s5AbOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents ReelOnce, an agentic framework for multi-shot story video generation. It selects reference formats for individual shots and uses evaluation feedback to revise planning, references, or generated media. Persistent story memory maintains shared constraints and tracks dependencies across these stages. The paper also introduces RSSA, which evaluates story fidelity by reconstructing events from sampled video frames. Experiments on 20 stories compare ReelOnce with existing systems using automatic metrics, human ratings, and ablations.

Critical strengths
  1. The paper addresses a practical problem: errors observed in a generated video may originate in planning or reference construction. Allowing feedback to trigger upstream changes, while preserving unaffected outputs, is a sensible design.

  2. The evaluation includes automatic metrics, human judgments, and qualitative repair cases. The complete system performs well against the reported baselines, and Figure 4 illustrates both targeted repairs and a rejected revision, making the workflow easier to understand.

Critical weaknesses
  1. Section 3.3 lists the factors used for reference selection, but does not explain how competing requirements are handled. The routing prompt and the distribution of selected modes are also missing. Comparisons with fixed-mode or simple rule-based policies, while keeping the rest of the pipeline unchanged, would help establish whether the routing decisions improve generation quality.

  2. The rationale for the four reference modes in Table 5 is unclear. Their functions overlap, which is not necessarily a problem, but the benefit of this particular grouping is not demonstrated. A useful baseline would let the same LLM construct valid reference inputs directly, without the four-director structure. The w/o Ref. Policy comparison in Table 1 also changes cross-shot media references, as described in Appendix B, so the difference cannot be attributed to adaptive routing alone.

Questions for authors

How does the Reference Expert resolve competing shot requirements, and how often is each mode selected? What evidence supports the four-mode design over a simpler policy or direct reference construction by the same LLM?

Can the reference-policy and memory ablations in Table 1 be repeated while keeping cross-shot media references unchanged? For memory, which parts of the shot contract contribute most compared with a fixed story plan or text summary?

Does hierarchical refinement outperform fixed-stage repair or repeated generation and selection under the same budget? Please report actual request counts, runtime, and costs so that better repair decisions can be distinguished from the benefit of additional generation attempts.

How often does the Doubao evaluator correctly identify defects under the eight-frame protocol in Appendix A.7? Beyond the examples in Figure 4, are there human-verified results on false alarms, missed defects, and revisions that degrade otherwise correct content?

How closely do RSSA scores agree with human judgments on the same videos? Can the ratings underlying Table 3 support a story-level correlation analysis, particularly for script faithfulness?

Reviewer_rBCiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper proposes ReelOnce, an agentic system for automated multi-shot story video generation from user prompt. To improve controllability and instruction following, its adaptive reference policy routes each shot to one of four specialized directors that provide composition images, identity assets, a combination of both, or ordered action panels. The system also introduces evaluation-driven hierarchical refinement to address errors in planning, reference construction, and media generation, aiming to improve cross-shot consistency and video quality. On ReelBench, the GPT-based configuration achieves the highest reported RSSA and LoCoT2V among the compared methods, at 78.70 and 0.8760, respectively. ReelOnce also achieves the highest mean scores across the seven reported VBench dimensions and human evaluation.

Critical strengths
  • ReelOnce provides a coherent and detailed system design for reference selection and targeted error correction. It uses shared story memory and reusable assets to maintain narrative constraints and visual references throughout generation. These components provides a promising approach for addressing challenges in maintaining consistency and correcting errors across multiple shots.
  • The complete system performs well across automatic and human evaluations. On the 20-story benchmark, ReelOnce (GPT) scores 78.70 on RSSA and 0.8760 on LoCoT2V, compared with 60.98 and 0.8227 for ViMax. Its mean across the seven reported VBench dimensions is also higher (87.90 versus 85.10). Human raters give it higher scores than ViMax for character consistency (4.03 versus 3.64) and script faithfulness (3.72 versus 3.12). Together, these results support the effectiveness of the complete system under the reported setup.
Critical weaknesses
  • Insufficient ablation analysis of hierarchical refinement. The w/o Repair ablation disables online quality evaluation and refinement, but does not isolate the contribution of the proposed hierarchical refinement mechanism. In particular, the paper does not investigate whether the four-level intervention design (D0–D3) is necessary or provides advantages over simpler refinement strategies, such as local media-level retries. Therefore, while the ablation demonstrates the overall benefit of evaluation-driven refinement, it does not sufficiently establish the effectiveness or necessity of the hierarchical design itself.
  • Limited reliability and discriminative power of the RSSA evaluation. RSSA relies on VLM-based judgments across seven sub-metrics, each scored on a 0–100 scale, raising concerns about the calibration and reliability of these scores. As shown in Tables 8 and 9, the sub-metrics do not consistently distinguish among the five ReelOnce configurations. Different evaluators also produce inconsistent rankings: for example, the GPT-5.6-luna and GPT-5.6 Sol evaluators disagree on the relative performance of several configurations in core-event coverage (CE) and event order (EO). Moreover, the three evaluators exhibit substantial systematic differences in their scoring scales, suggesting evaluator-dependent biases.
Questions for authors
  1. Ablation of hierarchical refinement. Have the authors conducted separate ablations of D1-only refinement and D1–D2 refinement to validate the necessity and effectiveness of the proposed hierarchical design?

  2. RSSA scoring calibration. Could the authors provide scoring anchors or examples for calibrating the 0–100 RSSA scores?

  3. Success rate under ablation. Why do ablations reduce the generation success rate from 100% to 90% (w/o Repair) and 85% (w/o Mem.)?

  4. Large drop in dynamic degree. Why does dynamic degree (DD) drop from 79.43 to 59.72 in w/o Ref. Policy compared with w/o Repair? How much of this drop is attributable to fixed references versus removing cross-shot media references?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.