acceptodds
Under review as a conference paper at ICLR 2027

DramaR²: Recursive Self-Evolution of Coherent Short-Drama Generation

Abstract

## Abstract Short-drama generation aims to tell a complete and coherent story within a few minutes. Recent methods use agentic systems to plan and organize short-drama production and long-video generation. However, multi-agent production spans multiple stages and can be unstable, leading to unexpected failures. A failure that appears local may therefore reveal a broader weakness in the production harness. Recursive self-improvement (RSI) offers a promising path for improving agentic short-drama generation. By learning from failures observed during production and updating the harness, RSI closes a higher-level loop from production to harness evolution and back to production, allowing the generator itself to improve across projects. We introduce DRAMAR², a self-evolving multi-agent framework for short-drama generation with an inner production loop and an outer RSI loop. The inner loop turns a story into an edited short drama. The outer loop reads the production trace, turns failures into harness updates, replays the original case, and compares the updated and current harnesses on paired test cases. In this way, RSI changes the target of improvement from one drama to the harness that produces later dramas. Accepted updates guide later productions, whose traces provide evidence for further improvement. We also introduce DRAMAR²-Bench with drama-level and pair-level tracks for complete short-drama evaluation. Experimental results show that DRAMAR² outperforms four existing agentic long-video generation systems. See our project page for video demos.

Primary area:
generative models
Keywords:
video generation, agentic video generation, recursive self-evolution

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_wUwpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes an agentic pipeline for generating multi-shot short dramas. The inner loop chains a Planner, an Asset module with gray-box layout, a Director with identity binding, a Render step, a Reviewer and an Editor. The outer loop performs recursive self-improvement: a Failure Packet is attributed to a pipeline stage, the harness is updated, the update is validated by frozen replay and a paired canary run, and a Promotion Gate decides whether to keep it. The authors also introduce DramaR²-Bench, consisting of 15 videos scored on 6 drama-level dimensions and 5 pairs scored on 2 pair-level dimensions, with Qwen3.5-Omni-Plus assigning 0-100 scores. Reported results are Total 90.20 versus 88.39 for ViMax, Drama Overall 89.49 versus 86.78, CSD 57.48 on ViStoryBench and 84.41 on VBench.

Critical strengths
  1. The Promotion Gate, which requires frozen replay and a paired canary before accepting a harness change, is a sensible guard against regression drift in self-modifying pipelines.
  2. The pair-level evaluation across episodes addresses cross-episode consistency, which existing video benchmarks do not cover.
Critical weaknesses
  1. The recursive self-improvement claim is not isolated by any experiment; the inner loop already contains five separate improvements (gray-box layout, multi-view references with identity binding, pre-render checks, scoped context, boundary-checked editing), and no inner-loop-only baseline or per-round curve is reported.
  2. The evaluation is 15 videos and 5 pairs with no variance or confidence intervals, scored by a single VLM with no human agreement rate, and Figure 2 is explicitly labeled as illustrative with no measurements.
  3. Figure 3 shows only static frames, which cannot support claims about temporal coherence.
Questions for authors
  1. What is the score of the inner loop alone, with the outer loop disabled?
  2. What is the agreement between Qwen3.5-Omni-Plus scores and human raters on the 15 videos?
4 comments
AuthorsOpenReview ↗

Response to Reviewer wUwp(1/2)

To Reviewer wUwp

Dear reviewer, thank you for your comments. We address each concern point by point below.

W1 & Q1. Measured RSI progress and production with updates disabled

With the outer RSI loop disabled, the initial harness runs the inner production loop with fixed rules. On DramaR²-Bench it obtains 88.51 Drama Overall, 63.30 Pair Overall, and 75.91 Total Avg. After RSI, the complete DramaR² system obtains 89.49, 90.90, and 90.20, respectively. The Pair gain is 27.60 points, with cross-causality and cross-continuity increasing by 24.80 and 30.40 points. Table 1 gives the dimension-level comparison.

The initial harness is the Baseline in Figure 4 of the paper. It retains story planning, asset management, rendering, review, local repair, and editing. The later checkpoints at Rounds 6, 9, and 12 measure production with the updated harnesses, with Round 12 corresponding to the complete system. Figure 2 illustrates the workflow, while Figure 4 reports the measured trajectory.

To examine one update separately from the cumulative results, we compare the frozen parent and updated harnesses on the same five episode-pair cases under a common base runtime and primary models. The parent harness obtains 89.63 Drama Overall, 84.03 Pair Overall, and 86.83 Total Avg. Its updated successor obtains 90.08, 91.93, and 91.00, respectively. Both harnesses remain fixed during production, and each result averages three fresh evaluations of the same completed videos.

The updated harness retains the parent rules, allocates shots to each intercut location, and explicitly stages successive emotional changes. On the 1–5 human rating scale, Drama averages change from 3.53 to 3.57, and Pair averages from 4.10 to 4.40. All five cases have positive mean automatic Pair gains. This comparison measures how a reusable update affects later productions.

Table 1. Inner-loop-only and post-RSI performance across DramaR²-Bench dimensions. (a) Drama-level.

SystemCausalityPaceStateSourceA/VVisualDrama Overall
Inner loop alone89.6087.6089.8786.4088.3388.6088.51
DramaR² after RSI91.8788.6089.5388.2088.6088.5389.49

(b) Pair-level and total performance.

SystemCross-CausalityCross-ContinuityPair OverallTotal Avg.
Inner loop alone68.4058.2063.3075.91
DramaR² after RSI93.2088.6090.9090.20

W2 & Q2. Evaluation scale, uncertainty, and human–automatic agreement

The 15 videos and five episode pairs are per method. Across five methods, the original evaluation covers 75 videos and 25 pair tasks from ten source cases. These cases comprise five standalone stories and five two-episode stories. Figure 2 explains the method, while Figure 4 in the paper reports measured checkpoint results.

We added case-cluster uncertainty estimates that keep related episodes together. The original Drama Overall difference from ViMax is +2.71 points, with a 95% interval of [1.08, 4.76]. In the new policy comparison, Drama and Pair differences are +0.45 [−0.67, 1.61] and +7.90 [3.37, 13.27]; exact two-sided sign-flip tests give p = 0.5625 and p = 0.0625. Repeated judgments are averaged within cases, so cases remain the independent analysis units.

The follow-up human study uses a 1–5 scale, with ten ratings per task. The ten evaluators have backgrounds in film aesthetics, regular short-drama viewing, or short-drama production. DramaR² has the highest reported human mean across all eight dimensions, with Drama and Pair averages of approximately 3.73 and 4.40.

Using these expanded human ratings, we compare human and Qwen3.5-Omni-Plus Total Avg. rankings across the five methods. The method-level Spearman correlation is ρ = 0.90, with an exact two-sided permutation p = 0.0833. Human and automatic evaluations agree on 90% of pairwise method rankings and both rank DramaR² first.

For the same-video rater check, ten evaluators rated the first episode of A Marriage of Convenience and then the ordered episode pair. For each dimension, we compare pairs of raters and count score differences of at most one point. This percentage is 96.30% across the six episode-level dimensions and 100% across the two pair-level dimensions. These values describe agreement among raters on this example; the correlation above measures human–automatic agreement at the method level.

AuthorsOpenReview ↗

Response to Reviewer wUwp(2/2)

W3. Evidence for temporal coherence

The complete videos corresponding to Figure 3 in the paper are available in Section 1 of the anonymous demo page (https://anonymous.4open.science/w/bsjfdjhdf/), under Three Sunsets. This section presents the same story from all five methods, so reviewers can inspect shot transitions, character continuity, dialogue, and story progression directly. It is the ocean-liner story in which a florist and an aristocratic passenger promise to share three sunsets before their voyage ends.

The new human comparison evaluates cross-episode causality and continuity after watching the sequences. Their means increase from 4.0 to 4.5 and 4.2 to 4.3 after the update. Automatic Pair scores also improve across all five cases when the three repeated evaluations are averaged. Section 3 of the demo page (https://anonymous.4open.science/w/bsjfdjhdf/) presents the production of The First Drop, including local segment repair, a one-second crossfade between segments, and timed subtitles. The completed episode can be watched directly in the same section.

E1. Character provenance and AI-content disclosure

All character faces in the experiments are AI-generated. No real-person face photographs were used as inputs or reference portraits, and the videos depict fictional characters. The experiments therefore do not involve unauthorized use of real individuals’ facial images or portraits, addressing the concern about consent and likeness use.

AuthorsOpenReview ↗

A gentle discussion-phase reminder

Dear Reviewer wUwp,

Thank you very much for your great efforts in reviewing this paper and for your recognition of our work!

As the end of the discussion period is approaching, we are eagerly anticipating your feedback to ascertain if our responses have adequately addressed your concerns. Your feedback will be instrumental in enhancing the quality of our manuscript. We would appreciate the opportunity to address any additional concerns you may have before the discussion period ends.

Thank you once again for your time and consideration!

Sincerely,

The Submission 1777 Authors

AuthorsOpenReview ↗

A Gentle Reminder Before the Discussion Deadline

Dear Reviewer wUwp,

Thank you again for your careful review and constructive suggestions.

As the discussion deadline approaches, we would greatly appreciate your feedback on our responses. We would be happy to clarify any remaining questions.

Thank you for your time and consideration.

Best regards,

The Submission 1777 Authors

Reviewer_nKFnOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

DramaR² combines a complete short-drama production pipeline with persistent harness updates selected through failure replay and paired canaries. The final system leads aggregate automatic and human evaluations, and the appendix documents an actual update-and-reuse cycle. However, cumulative checkpoint gains do not fully establish the recursive procedure's additional generalizable benefit.

Critical strengths
  • The system separates identity, mutable state, audience knowledge, and spatial layout, addressing complementary sources of narrative inconsistency.
  • Appendix B.2 gives concrete failure attribution, a reusable policy change, automatic promotion conditions, and subsequent policy reuse. It would be inaccurate to describe RSI as merely a conceptual diagram.
  • The appendices disclose evaluator prompts, aggregation, missing-audio treatment, human rating procedures, and the method-level scope of human–automatic correlations.
Critical weaknesses
  • The central attribution remains incomplete. Checkpoints jointly introduce state transfer, shared assets, multi-view/spatial inputs, and source-coverage/runtime rules (Section 4.3; Appendix B). These changes are legitimate outputs of harness improvement, but their cumulative benefit does not by itself establish the recursive procedure's additional value over comparable nonrecursive improvement. Evidence from a frozen harness on independent stories or an already performed matched-budget search control would strengthen this claim.
  • Selection and final-test independence need explicit documentation. Appendix B.2 specifies iteration and held-out stories, and Appendix B.3 distinguishes the outer-loop test set from checkpoint benchmarking. Nevertheless, a repeatedly used promotion canary is a selection set. The relationship between those story families, production failures, final benchmarks, and benchmark-informed updates remains unclear. This is a request for the actual split chronology, not an allegation of leakage.
  • Reliability and evaluator independence are limited. The study uses a small set of source stories, with correlated episode, dimension, and pair scores (Appendix C.1). Human evaluation provides useful independent support, but rating allocation, inter-rater reliability, and story-level uncertainty are not reported. The aggregate human–automatic correlation of 0.90 uses five method means (Appendix D.3, p. 23), not item-level accuracy. The same audiovisual model also serves production, candidate, and benchmark evaluation roles (Table 3).
  • Full resource matching and search reproducibility are incomplete. Shared primary models do not match auxiliary asset generation, review, retries, and offline search. The candidate generation-work cap of 1.25 (Appendix B.2, p. 16) is not a total search-cost comparison. Complete candidate counts, acceptance rates, intervention boundaries, and executable configurations should be disclosed. External code/log availability was not independently checked in this review.
Questions for authors
  • How are failure, iteration, canary, and final-test story families separated, and when is the harness frozen?
  • What do existing candidate logs show about promotion, human intervention, and later reuse?
  • Can existing item/rater results and complete offline/online costs be supplied?
4 comments
AuthorsOpenReview ↗

Response to Reviewer nKFn(1/2)

To Reviewer nKFn

Dear reviewer, thank you for your comments. We address each concern point by point below.

W1 & Q2. Reusable updates, automatic promotion, and later production

We address attribution through a frozen-harness comparison. The parent harness and its actual updated successor are fixed before evaluation and used on the same five episode-pair cases under a common base runtime and primary models. The successor retains the parent rules and adds a reusable directing rule: allocate shots to each intercut location and explicitly stage successive emotional changes. This tests whether an update learned from earlier production improves later stories, alongside the cumulative results in Figure 4 of the paper.

Harness promotion is automatic. The outer loop compares the candidate with the current harness and promotes it only after the required production, quality, and test-set checks pass. The configured criteria require a mean gain of at least three points, no per-story decrease, a preference win, and the video-generation work cap, followed by the reserved-story acceptance check. The update is then activated for later production. The later production request contains the updated policy, showing its reuse beyond the failure that prompted it.

Across three fresh evaluations of the same fixed outputs, Drama Overall changes from 89.63 to 90.08, Pair Overall from 84.03 to 91.93, and Total Avg. from 86.83 to 91.00. All five cases have positive mean Pair gains. Human Drama averages are 3.53 and 3.57, and Pair averages are 4.10 and 4.40. Human cross-causality increases from 4.0 to 4.5, cross-continuity from 4.2 to 4.3, and source adherence from 3.5 to 3.9. These measurements connect the inherited harness update to improved source coverage and cross-episode coherence.

Harness promotion is separate from the acceptance of individual production outputs. In the follow-up comparison, both the original and updated harnesses remain frozen. Output acceptance does not select or modify either harness, and the outer loop retains its fixed automatic promotion rule.

W2 & Q1. Story separation and harness freeze points

The update campaign uses two development stories for failure analysis and candidate comparison. Repeated canary tests guide candidate selection, and a third story is reserved for acceptance after the candidate is fixed. Benchmark scores are then computed with the harness frozen.

The five episode-pair cases in the new policy comparison are based on stories different from the two used to develop the update. Neither episode of these cases comes from those two stories. Both policies are fixed before the new evaluation scores are obtained and remain unchanged throughout evaluation. This separates development of the update from the subsequent evaluation of its use in later productions.

W3. Case-level statistics and human evaluation

The analysis treats each source case as a cluster, keeping its two episodes and pair score together. Repeated judge calls and dimensions stay within that cluster. The original Drama Overall difference from ViMax is +2.71 [1.08, 4.76]. In the frozen-policy comparison, Drama and Pair differences are +0.45 [−0.67, 1.61] and +7.90 [3.37, 13.27], with exact paired sign-flip p-values of 0.5625 and 0.0625. This explicitly accounts for the dependence among observations from the same story.

We expanded the human panel to ten raters, with ten ratings per task. Their backgrounds include film aesthetics, regular short-drama viewing, and short-drama production. DramaR² obtains the highest reported human mean across all eight dimensions. For the frozen update comparison, human Pair averages increase from 4.10 to 4.40. These human judgments provide a separate assessment of the same production outcomes.

We additionally examined ten ratings of the same episode from A Marriage of Convenience, together with ratings of its episode pair. Across the six episode-level dimensions, 96.30% of within-dimension rater pairs differ by at most one point; across the two pair-level dimensions, this is 100%. State and Source have zero between-rater standard deviation. These results directly describe agreement on this example. The paper’s ρ = 0.90 is reported separately as agreement across five method means, with exact two-sided p = 0.0833.

The automatic repeatability analysis uses three Qwen evaluations of the same videos, with the videos and scoring protocol held fixed. The resulting Pair gain is positive in each repetition, ranging from 3.30 to 16.40 points. Repeated scoring measures evaluator variation; case-cluster analysis measures variation across stories, and human ratings provide a separate assessment.

AuthorsOpenReview ↗

Response to Reviewer nKFn(2/2)

W4 & Q3. Evaluation records, costs, and planned release

The 1.25 cap constrains submitted video-generation work, measured in 720P-equivalent seconds and including retries. We report API costs separately, covering both video production and the wider research experiments.

Evaluation cost. A complete-episode evaluation costs approximately USD 0.07, and an episode-pair evaluation approximately USD 0.15. One scoring pass over 20 episodes and ten episode pairs costs approximately USD 2.91.

Production and research cost. One complete episode costs approximately USD 52 before RSI and USD 50 after RSI. The estimated research API expenditure is approximately USD 7,600, covering harness development and search, benchmark production, evaluation, and the follow-up experiments.

We will release executable configurations, harness updates and decision records, per-video and per-pair scores, the three repeated evaluations, and the cost summary. Section 3 of the anonymous demo page (https://anonymous.4open.science/w/bsjfdjhdf/) presents The First Drop as a complete production walkthrough, from story planning and reference assets through rendering, review, local repair, and editing. It also illustrates the outer RSI workflow: using production evidence to propose a candidate harness rule, followed by replay, paired tests, and automatic promotion.

AuthorsOpenReview ↗

A gentle discussion-phase reminder

Dear Reviewer nKFn,

Thank you very much for your great efforts in reviewing this paper and for your recognition of our work!

As the end of the discussion period is approaching, we are eagerly anticipating your feedback to ascertain if our responses have adequately addressed your concerns. Your feedback will be instrumental in enhancing the quality of our manuscript. We would appreciate the opportunity to address any additional concerns you may have before the discussion period ends.

Thank you once again for your time and consideration!

Sincerely,

The Submission 1777 Authors

AuthorsOpenReview ↗

A Gentle Reminder Before the Discussion Deadline

Dear Reviewer nKFn,

Thank you again for your careful review and constructive suggestions.

As the discussion deadline approaches, we would greatly appreciate your feedback on our responses. We would be happy to clarify any remaining questions.

Thank you for your time and consideration.

Best regards,

The Submission 1777 Authors

Reviewer_76x5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the system is designed so that, when producing relatively long dramas, the AI improves the production team’s “manual” (the agent harness) rather than only fixing individual outputs. The inner loop generates scenes from the story and edits them into a drama, while the outer loop collects production failures and proposes a revised version of the harness. A revised manual is adopted only when three conditions are met at once: it repairs the scene that originally failed, no protected metric gets worse than under the old manual, and at least one target metric improves. The new manual is then used from the next production onward, and when new failures appear, another round of improvement follows — this is how recursive self-improvement (RSI) is carried out.

Critical strengths

(1) The paper reframes the improvement target from the individual failed scene to the reusable agent harness, and brings recursive self-improvement — previously studied mainly in coding and terminal-use settings — into multimodal video production. (2) The promotion procedure is appropriately conservative: a candidate harness must repair the original failure under frozen replay, show no regression on protected metrics in paired canary tests, and improve at least one target metric before it is adopted. This guards against the regressions that make self-modifying systems risky in practice. (3) The engineering choices are well motivated by concrete failure modes: multi-view identity references with pre-render ID binding address character swaps, gray-box opening layouts address spatial inconsistency, and per-segment scoped context keeps parallel generation tractable.

Critical weaknesses

(1) The RSI trajectory is reported at only three checkpoints (Rounds 6, 9, 12) from a single run; without repeated runs or intermediate rounds, it is unclear whether the gains are stable or partly due to chance, and the drops in Source and A/V at intermediate rounds are not analyzed. (2) The evaluation set is small — ten source stories yielding 15 videos and only five episode pairs — which is especially concerning because the largest reported gains are on the pair-level metrics. (3) All scores are reported as single means with no variance, confidence intervals, or significance tests, even though generation is stochastic; the 1.8–2.7 point margins over ViMax may fall within run-to-run variation. (4) The human study uses only five raters, and the human–automatic agreement (Spearman ρ = 0.90) is computed over just five method-level averages, which is too few to establish statistical significance.

Questions for authors

The paper appears to be oriented toward a practical system rather than an academic contribution, yet the number of experiments is relatively small, so its effectiveness has not been adequately validated.

Is there a particular reason the number of human participants was kept so small?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 76x5 (1/2)

Thank you for your comments. We address each concern below.

W1 & Q1. Research contribution, RSI progress, and supporting evidence

Our contribution is to make the reusable agent harness itself an object of improvement in video generation. DramaR² links production failures to the decisions that caused them, turns this experience into reusable rules, tests the revised harness, and carries accepted updates into later productions. This creates a recursive loop in which each production can improve the system used for the next one. DramaR²-Bench measures the resulting quality within dramas and across episodes.

We extend Figure 4 of the paper with Rounds 7, 8, 10, and 11. Round 7 adds multi-view asset references and a spatial opening image. Round 8 adds checks that align the planned duration and segment totals with the delivery window. Round 10 ties duration to story content, maps source details to segments, preserves speech delivery modes, and carries approved assets across episodes. Round 11 allows dialogue adaptation while checking that important facts reach the audience through visible action or speech, and separates planning records from the video-generation request.

For a matched intermediate comparison, we evaluate three existing versions of the same episode at Rounds 8, 10, and 11. The episode follows a woman who returns to the past, cancels a harmful marriage, and tests whether her memories can change her future. Each complete video is scored three times against the same reference using the fixed Qwen rubric. Mean Overall scores are 87.86, 89.20, and 89.28, respectively. The table gives the six drama-level dimensions and scoring variation.

We also trace the intermediate A/V and Source changes to the original case reports. The lower A/V scores at Round 6 are associated with unclear speech, low dialogue volume, and speech masked by music. The Source change at Round 9 is concentrated in two stories whose reports identify missing setup, decisions, and connecting events. The later detail ledger, audience-information checks, and explicit speech delivery rules address these failure types. At Round 12, Source reaches 88.20 and A/V 88.60. Candidate promotion uses the outer loop’s own test set, while Figure 4 reports benchmark performance.

We added a controlled comparison of a frozen parent harness and its updated successor on the same five episode-pair cases, with a common base runtime and primary models. The updated harness inherits the parent rules and adds explicit coverage of intercut scenes and successive emotional changes. Across 20 complete episodes, three fresh evaluations of the same fixed videos give 89.63 → 90.08 for Drama Overall and 84.03 → 91.93 for Pair Overall. All five cases have positive Pair gains after averaging the three evaluations. Removing any one case leaves a positive gain of 5.25–9.38 points, and each scoring repetition also gives a positive Pair gain.

The ten-rater human evaluation supports the same pattern: Pair averages increase from 4.10 to 4.40, while Drama averages are 3.53 and 3.57. Together, the checkpoint trajectory, frozen-harness comparison, and human results show how reusable updates improve cross-episode production while maintaining within-drama quality.

Table 1. Intermediate checkpoints on the same complete episode (three ratings per video).

RoundCausalityPaceStateSourceA/VVisualOverall ± SD
885.3386.3391.6789.0088.3388.0087.86 ± 2.53
1090.0088.6789.3389.0089.0089.0089.20 ± 2.02
1189.6787.6791.3389.0088.3389.0089.28 ± 1.70

W2. Benchmark coverage and case-level robustness

Each story tests several narrative requirements, allowing a modest number of cases to cover different story structures and continuity challenges. We use a compact case set to keep the cost of generating and evaluating complete, multi-minute dramas manageable for other researchers.DramaR²-Bench contains ten source cases: five standalone stories and five two-episode stories, yielding 15 videos and five episode pairs per method. It covers six Chinese and four English inputs, six portrait and four landscape formats, and narrative settings including fantasy, romance, family identity reversals, revenge, and future-call mysteries. The paired stories test character and object states, changing relationships, unresolved events, and facts already known to the audience.

To check whether one case drives the overall gain, we remove each case in turn and recompute the mean. All five cases show Pair gains, and the mean gain across the other four is 5.25–9.38 points, indicating gains across stories. W3 reports story-level confidence intervals. Related full-story evaluations also use compact sets: MoviePrompts uses ten script prompts (Wu et al., 2025), and FilmEval uses fifteen source novels (Zuo et al., 2026).

AuthorsOpenReview ↗

Response to Reviewer 76x5 (2/2)

W3. Uncertainty and statistical analysis

We now quantify uncertainty across source cases and repeatability across scoring calls separately. In the original Drama Overall comparison, DramaR² scores 2.71 points higher than ViMax. The 95% confidence interval is [1.08, 4.76], estimated using 50,000 paired case-cluster bootstrap samples. Both episodes of a paired case stay together, and standalone and paired cases are resampled within their respective groups. Removing any one case leaves a positive difference of +1.71 to +3.34 points.

For the new policy comparison, we average the three fresh judgments per case before analysis. The Drama and Pair differences are +0.45 [−0.67, 1.61] and +7.90 [3.37, 13.27], respectively. Exact two-sided paired sign-flip tests give p = 0.5625 and p = 0.0625.

These analyses support two complementary findings. DramaR²'s advantage over ViMax remains positive under case removal, and the frozen-policy update produces consistent cross-episode gains across all five cases. The smaller within-drama difference shows that the main benefit of this update is cross-episode coherence. Source cases remain the statistical units, while the three scoring repetitions assess evaluator repeatability on the same generated videos.

Section 4 of the anonymous demo page (https://anonymous.4open.science/w/bsjfdjhdf/) examines consistency across random seeds using an actual Wan production request from our framework. We keep the complete five-shot prompt, six image references, two voice references, and generation settings fixed, and generate three 25-second outputs with different seeds. All three outputs preserve the same characters, core dialogue, event order, and ending. Differences occur in camera framing and action timing, while the central narrative remains stable across the three seeds. The page displays the original input, shared references, and all three outputs side by side.

W4 & Q2. Rater recruitment, expanded human evaluation, and agreement

The follow-up panel includes ten evaluators, with ten ratings per task. These evaluators have a foundation in film aesthetics, regularly watch short dramas, or have experience producing short dramas themselves. The initial study recruited five professional evaluators familiar with short dramas and film aesthetics. Each task requires watching a complete multi-minute drama or an ordered episode pair, so the initial panel size reflected both the availability of evaluators with relevant backgrounds and the time needed for full-video review.

Human–automatic agreement. Using the expanded ten-rater results in Table 2, we compare human and automatic Total Avg. rankings across the five methods. The Spearman rank correlation is ρ = 0.90, with exact two-sided permutation p = 0.0833. Human and automatic evaluations achieve 90% agreement in pairwise method rankings, and both rank DramaR² first. They also agree on the direction of improvement in the frozen-harness comparison: automatic Pair Overall rises from 84.03 to 91.93, while human Pair averages rise from 4.10 to 4.40.

On the 1–5 scale, DramaR² has the highest reported mean in all eight dimensions, with human Drama and Pair averages of approximately 3.73 and 4.40, compared with 2.60 and 3.70 for the highest baseline averages at the respective levels.

We also report direct agreement among the ten raters on the same example, A Marriage of Convenience. The first six dimensions concern its first episode; the final two concern continuity after watching both episodes. State and Source receive identical scores from all ten raters. Across the six episode-level dimensions, 96.30% of the 270 within-dimension rater pairs differ by at most one point; for the two pair-level dimensions, this is 100% of 90 comparisons. Table 3 gives the mean and standard deviation for each dimension. This is a descriptive agreement result for this example, separate from the method-level human–automatic correlation.

Table 2. Follow-up human ratings (1–5; ten ratings per task).

MethodDrama Avg.Pair Avg.Total Avg.
MovieAgent2.602.852.73
VGoT1.151.251.20
ViMax2.553.703.13
VideoClaw2.133.402.77
DramaR²3.734.404.07

Table 3. Ten-rater agreement on one episode and its episode pair (1–5).

DimensionMean ± SDRangeWithin 1 point
Causality3.10 ± 0.572–495.56%
Pace2.40 ± 0.522–3100.00%
State4.00 ± 0.004–4100.00%
Source5.00 ± 0.005–5100.00%
A/V3.90 ± 0.323–4100.00%
Visual3.30 ± 0.673–582.22%
Cross-Causality4.20 ± 0.424–5100.00%
Cross-Continuity4.20 ± 0.424–5100.00%
AuthorsOpenReview ↗

A gentle discussion-phase reminder

Dear Reviewer 76x5,

Thank you very much for your great efforts in reviewing this paper and for your recognition of our work!

As the end of the discussion period is approaching, we are eagerly anticipating your feedback to ascertain if our responses have adequately addressed your concerns. Your feedback will be instrumental in enhancing the quality of our manuscript. We would appreciate the opportunity to address any additional concerns you may have before the discussion period ends.

Thank you once again for your time and consideration!

Sincerely,

The Submission 1777 Authors

AuthorsOpenReview ↗

A Gentle Reminder Before the Discussion Deadline

Dear Reviewer 76x5,

Thank you again for your careful review and constructive suggestions.

As the discussion deadline approaches, we would greatly appreciate your feedback on our responses. We would be happy to clarify any remaining questions.

Thank you for your time and consideration.

Best regards,

The Submission 1777 Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.