acceptodds
Withdrawn from ICLR 2027

DramaDirector: Geometry-Guided Short Drama Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Short dramas combine rapid shot rhythms, dialogue-driven focus shifts, and frame-level spatial constraints that are often underspecified by text-only generation pipelines. We study plot-to-short-drama generation, where a global plot and local context are transformed into a structured storyboard and then realized as a multi-shot video. We propose DramaDirector, a geometry-guided framework that connects storyboard planning to visual realization through two coupled components. First, it retrieves appearance-reduced depth–pose references from real short-drama shots to provide reusable priors for spatial layout and body configuration. Second, it uses the learned retrievability of generated storyboard conditions as a GRPO training signal, encouraging the planner to produce descriptions that can be grounded by the downstream visual pipeline. Static visual and dynamic narrative conditions serve as the interface for first-frame generation and image-to-video synthesis. We also introduce DramaBoard, a benchmark built from 35 live-action dramas and 2,826 episodes with shot annotations (101,763 shot records); its Continuation subset contains 2,807 episodes and 81,283 shots, with structured storyboards and multi-dimensional evaluation protocols. In the reported benchmark tables, DramaDirector has higher or competitive values than representative agentic and video-generation baselines on several narrative-quality, instruction-following, and intrinsic-video-quality metrics; these comparisons are preserved from the submission record rather than newly rerun in the current artifact snapshot. Our code is anonymously released at: https://anonymous.4open.science/r/DramaDirector-7484/

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Short-Drama Generation, Text-Visual Alignment, Supervised Fine-Tuning, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_HJ6eOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DramaDirector, a framework for generating multi-shot short dramas from plot descriptions through geometry-guided storyboard planning. The method introduces a structured storyboard representation that separates static visual conditions from dynamic narrative instructions. The authors trained a text-visual alignment model to retrieve depth and pose references from a real short-drama database to guide first-frame generation. The authors further train the storyboard planner using multi-task SFT and GRPO, incorporating visual retrievability as a reward signal. They also introduce DramaBoard, a benchmark containing over 100K annotated shots from 35 short dramas. Experiments demonstrate improvements over existing methods in narrative quality, instruction following, and video generation quality.

Critical strengths
  1. The paper addresses an interesting and practical problem in multi-shot video generation. Using depth and pose references to bridge the gap between textual storyboard planning and visual generation is a reasonable approach.

  2. The authors construct a relatively large-scale short-drama benchmark with shot annotations, dialogue, and geometric information. The paper also provides extensive experiments and compares the method with recent baselines.

Critical weaknesses
  1. In Table 4, the proposed method achieves only 9.54% Recall@10 and 14.29% Recall@20. Although these results outperform the baseline methods, the absolute retrieval accuracy is quite low and it is unknown whether the retrieved depth/pose references actually provide suitable geometric guidance. Also, the paper does not sufficiently demonstrate how retrieval accuracy correlates with actual spatial correctness in the generated videos.

  2. The contribution of the retrieval-based GRPO reward seems relatively weak. In Table 13, incorporating the full retrieval reward only improves Recall@10 from 1.54% to 1.90% compared with SFT-only training. Moreover, the reward is computed by comparing generated storyboard descriptions against geometry from the target episode and a sampled negative episode, which does not necessarily reflect the geometric correctness or generation quality of individual shots.

  3. The evaluation seems not to sufficiently measure the geometry-related improvements in the generated videos.Metrics such as retrieval accuracy, first-frame quality and video consistency do not directly show whether the generated videos follow the retrieved poses, whether depth and spatial relationships are preserved, or whether the camera movement is accurate. The human evaluation also focuses on general perceptual quality rather than geometry-specific correctness. It would be great if the paper can include more direct quantitative and human evaluations of spatial controllability.

  4. Given the retriever is imperfect, and the short drama dataset has limited diversity, I'm wondering how the model behaves in such cases: for example, the storyboard requires a scenario that contains e.g. 4-5 people, but the retriever only obtained a depth map or pose video of e.g. 2 people. How will the generated video look like in such scenarios?

Questions for authors

See weaknesses

Reviewer_RjrkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes DramaDirector for plot-to-short-drama generation. A Qwen3-8B planner is first trained with multi-task storyboard SFT and then with GRPO. For each shot, it writes a static condition that describes the camera, background and subject layout, and a dynamic condition that describes the narrative, dialogue and emotion. An alignment model uses the static condition to retrieve a depth map and a pose map from a gallery of real short-drama keyframes. These maps guide Seedream-v5-lite in generating the first frame, which Vidu-q3-turbo then animates. The GRPO reward combines this retrieval score with an LLM judgment of video readiness and a format score. The authors also introduce DramaBoard, a dataset of 101,763 annotated shots from 35 dramas. When all systems share the same image and video backends, DramaDirector reports the best scores on most metrics against eight adapted baselines.

Critical strengths

The interface between the planner and the generator is concrete and inexpensive. Separate conditions handle layout and animation, which is a sensible design, and because the gallery embeddings are precomputed, a shot costs only about $0.30 to produce. The comparison is also reasonably controlled. All methods share Seedream and Vidu, and the agentic baselines even use a larger planner than DramaDirector. Under this setup, DramaDirector clearly leads on storyboard Relevance and Coherence.

DramaBoard is a useful resource for this domain. It provides keyframes, corrected transcripts, structured storyboards and depth and pose maps for more than 100k shots. A manual audit of 500 shots, each rated by three annotators, checks the annotation quality. The authors also report openly that the gallery is dominated by static, eye-level shots with one or two people.

Critical weaknesses

The central claim is that retrieved geometry drives the gains, but the paper does not show this. First, the retriever itself is weak. A gold static condition finds its own keyframe geometry among the top 10 results only 9.5% of the time, barely above the 9.1% of an existing retrieval baseline. Second, no experiment checks whether the generated first frames actually follow the retrieved depth or pose, and the human study explicitly leaves geometry out. Third, the paper never compares against a randomly chosen gallery reference or a retrieved RGB keyframe, so it remains unclear whether the specific retrieved structure matters. The generation ablation in Table 3 adds to this doubt. Consistency stays between 0.58 and 0.63 with text alone, depth alone or pose alone, yet rises to 0.90 when both maps are used. Moreover, the text-only variant scores below every baseline in Table 2 on Consistency and Faithfulness, even though the baselines also use the same backend without retrieved depth or pose maps.

The contribution of GRPO and the retrieval reward is not established either. In Table 5, the selected SFT checkpoint has exactly the same scores as the final model in Table 2. Fig. 3, however, reports clearly lower scores for the SFT-only variant, and Fig. 5 selects a separate GRPO checkpoint. As reported, the final system therefore cannot be distinguished from the SFT model. The direct effect of the reward is also small, since Table 13 shows that it raises Recall@10 on generated storyboards only from 0.015 to 0.019.

Finally, the evaluation does not support the narrative-quality claims. Relevance, Coherence and Engagement are defined as judgments of the storyboard text alone, and the judge is instructed not to use the video. Nevertheless, swapping only the video or image generator changes these scores by as much as DramaDirector's margin over the best baseline. For example, Relevance moves from 0.723 to 0.782 with Wan 2.6. The evaluation is also not independent of training, because Qwen3.5-Plus produces both the training annotations and the final scores, and Qwen3.5-Flash provides the GRPO reward. In addition, DramaDirector's planner is fine-tuned on DramaBoard while the baselines are only prompted. Retrieval excludes only the query episode, so other episodes of the same drama remain available as references.

Questions for authors
  1. Which checkpoint produced the DramaDirector row in Table 2, and what are the full Table 2 results of the SFT checkpoint and the GRPO checkpoint?

  2. Were the storyboards held fixed across the generator-swap rows in Table 2? If so, why do the text-only narrative scores change?

  3. How closely do the generated first frames follow the retrieved depth and pose, and how does the full system compare with a random gallery reference or a retrieved RGB keyframe?

Reviewer_9JdiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents DramaDirector, a framework for generating multi-shot short-drama videos from plot summaries and local narrative context. The method trains a storyboard planner through multi-task supervised fine-tuning and GRPO, retrieves depth and pose references from real drama shots to guide first-frame generation, and uses an image-to-video model to generate individual shots. Its central design is to use a learned text–geometry alignment model both for reference retrieval and for computing the planner’s reward. The authors also introduce DramaBoard, a structured storyboard dataset derived from 35 dramas.

Critical strengths

S1. Using retrieved depth–pose references to guide generation, together with an alignment-based reward for the planner, is a sensible way to connect storyboard planning with visual realization.

S2. The improvements are supported by ablations and human evaluation. DramaBoard is also a useful contribution, and the annotation audit adds confidence in its quality.And separating static visual conditions from dynamic narrative instructions gives the pipeline a clear structure, allowing geometry references to guide first-frame composition while narrative instructions drive the subsequent video generation.

Critical weaknesses

W1. The closest reference may still conflict with the prompt when the gallery lacks compatible structures. A comparison with and without references on low-compatibility queries would clarify when retrieval helps or hurts and whether a fallback is needed.

W2. Figure 4 provides useful first-frame evidence, but the current evaluation does not clearly establish whether the retrieval reward improves geometric correctness throughout the generated videos. Independently retrieved references may also conflict across shots in layout, eyelines, actions, or prop states. Targeted geometry and cross-shot continuity evaluations would strengthen the central claim

Questions for authors

Q1. When the gallery lacks compatible structures, does the system support retrieval abstention or generation without references? Does retrieval remain beneficial for low-compatibility queries?

Q2. How are references retrieved for different shots coordinated? Can the authors provide dedicated evaluations of cross-shot spatial consistency and continuity of actions and props?

Q3. What are the main advantages of real-shot retrieval over procedurally constructed structural references? Which advantages are supported by the current experiments?

Reviewer_EYwAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DramaDirector, a framework to address two challenges in short-drama video generation: (1) the distinctive storytelling conventions of short dramas (e.g rapid shot rhythms) that are not adequately captured by generic storyboard generation, and (2) the lack of frame-level spatial grounding in text-based storyboards.

To address these challenges, the authors propose retrieving depth and pose priors from a collection of real short-drama shots and optimizing a storyboard planner using GRPO to generate geometrically retrievable storyboards. The retriever is trained with a contrastive learning objective, while the planner is post-trained with GRPO using three rewards that assess storyboard format, geometric retrievability, and video-generation readiness.

The paper also introduces DramaBoard, a benchmark for training and evaluation of short drama video generation. DramaDirector is evaluated against multiple baselines on DramaBoard across three dimensions: storyboard narrative quality, instruction following, and intrinsic video quality. The proposed method achieves the best or second-best performance on most metrics, with notable improvements in relevance, coherence, and aesthetics. Ablation studies investigate the contributions of the retrieval and planning components.

Critical strengths

The incorporation of depth and pose priors provides explicit geometric guidance for video generation, which is particularly valuable for short dramas with rapid shot transitions, while remaining applicable to broader video-generation settings. A key strength (which interestingly can also limit the method's generalizability, as explained in the "Critical Weaknesses") is the dual use of the learned retriever, both to provide depth–pose priors for first-frame generation and to compute the retrievability reward for GRPO post-training of the storyboard planner. This creates a principled connection between storyboard planning and downstream visual generation. The experimental results demonstrate the benefits of depth–pose conditioning and GRPO-based planner optimization, and ablation studies provide evidence for the contributions of these components (if the questions raised in "Questions For Authors" are addressed).

Critical weaknesses

Insufficient evaluation of geometric faithfulness. The main contribution of the paper is geometry-guided video generation; however, the evaluation does not explicitly measure whether the generated videos satisfy the intended geometric constraints. Figure 4 evaluates spatial positioning and actions in generated first frames, but the overall video evaluation relies on general metrics such as Faithfulness and Consistency, which do not directly assess geometric accuracy. A more targeted evaluation of geometric measures like pose fidelity, spatial relationships, and depth consistency throughout the generated videos would be much stronger support for the central contribution.

Unexplored tradeoff between retrievability and narrative faithfulness. The proposed GRPO retrieval reward encourages the planner to generate storyboard descriptions that align with available geometric references. However, this may favor easily retrievable configurations over those most faithful to the intended narrative, particularly when the reference gallery has limited coverage. The GRPO reward computation uses coefficients to balance the two, but the paper does not investigate this potential tradeoff nor how retrieval-reward optimization affects narrative faithfulness specially when the pose-depth gallery coverage is limited.

Questions for authors
  1. Could the authors provide separate ablations for all three reward components R_{ret}, R_{vid}, and R_{fmt}? The current experiments only show the contribution of R_{ret} in Figure 3. A sensitivity analysis of the reward coefficients alpha and beta would also help clarify potential tradeoffs.

  2. Table 3 has interesting results that require further investigation. In particular, why does adding depth or pose individually to text conditioning worsen DIS, whereas combining both substantially improves it? A similar interaction appears in Consistency. Could the authors investigate this behavior and also report the Position and Action metrics from Figure 4 for these ablations?

  3. DramaDirector adds a retrieval layer to the pipeline. Could the authors compare the end-to-end inference latency and computational cost of DramaDirector against the baselines?

Minor:

  • Table 5 and Section 4.7 mainly report performance changes with additional training steps, without establishing convergence or meaningful tradeoffs. The absence of confidence intervals also makes it difficult to assess whether the smaller differences reflect genuine improvements or evaluation noise.
  • Figure 2 is dense, making the overall framework and interactions between its components difficult to follow.
  • Pages 8–9 appear to use different font sizes or formatting from the rest of the paper, deviating from the ICLR template.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.