acceptodds
Withdrawn from ICLR 2027

Scripty: An External Script Supervisor for Continuity in Story-Level Video Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Story-level video generation runs a model clip by clip inside a harness that writes prompts and picks reference images, yet a detail set in one clip, such as a cut braid, often reverts in the next: retaining event history does not specify the current state, and correct text can still conflict with an outdated reference image. The harness must know which states still apply, and both the prompt and the reference images must show them. We present Scripty, a continuity module that works like a film set's script supervisor, attached via host-specific adapters without modifying host code or models. From frozen storyboards, asset lists, and reference images it builds a state record, each change citing its storyboard line, and it keeps each state until the storyboard changes it, injecting states into prompts and guiding reference-variant generation. Generated prompts and clips never serve as state evidence. We introduce StateHold, a benchmark that asks whether a later clip depicts the state the storyboard specified at that point. On a fixed host across 42 stories, Scripty reaches 48.8% cross-unit fact pass rate versus 26.0% for native recap (+22.8 pp); adding reference-image variants improves this further, with some within-unit losses. These results support improved storyboard-state conformance but do not establish better viewer-perceived continuity or reliable state expiry.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multi-shot video generation, state persistence, agentic video generation, structured state tracking, reference image conditioning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_xBtkOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Scripty, an external continuity module that maintains storyboard-grounded state records and guides video generation through text injection (L0) and reference-image adaptation (L1). On the proposed StateHold benchmark, Scripty improves cross-unit state conformance from 26.0% to 48.8%.

Critical strengths
  • Important problem with a practical solution. Scripty addresses state inconsistencies across generated video clips, particularly conflicts between textual prompts and outdated reference images.

  • Strong empirical evaluation. The +22.8 pp improvement over native recap is substantial. The L0/L1 ablations, paired-seed rendering experiments, and robustness analyses provide convincing evidence of the method's effectiveness.

Critical weaknesses
  • Limited methodological novelty. Prior works such as SEAM, CANVAS, and FilmWorld already explore structured state tracking and state-aware generation. Scripty's main contribution appears to be their integration into an external module. The necessity of evidence-linked state records is not directly validated, and the improvements may partly reflect additional information access and computation.

  • Limited evidence for general continuity improvements. StateHold primarily measures whether later clips depict storyboard-specified states, rather than whether states are consistently maintained across generated clips. Although Joint CUC partially addresses this issue, its absolute performance remains relatively low (31.5%). Human validation and long-gap evaluations are also limited.

Questions for authors
  • What are the essential methodological differences from FilmWorld and CANVAS? Could the authors ablate evidence binding or provide a compute- and information-matched baseline?

  • Could the authors provide stronger human validation of Joint CUC and additional long-gap evaluations?

Reviewer_eT5COpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Scripty is an external continuity module for story-level video generation. In practical terms, the proposed method is a training-free harness framework for long-form, story-level video generation. It constructs an evidence-linked persistent-state record and uses this record to constrain prompt writing (L0) and reference-image selection or editing (L1). The paper also introduces StateHold to evaluate whether later clips depict the states specified by the storyboard.

Critical strengths

Figures 2 and 3 show how the same evidence-linked state record can guide both textual instructions and reference images. This directly addresses a concrete failure mode: an outdated reference image can undermine even a correct prompt. Restricting state evidence to upstream artifacts also avoids treating generated errors as authoritative state. The adapter-based design is a useful systems contribution that does not require training a new video model.

Critical weaknesses

The authors do not improve the underlying video model. Instead, they introduce a relatively simple harness intended to make the story-level video generation pipeline more stable. I remain skeptical that the scope of this work is sufficient for ICLR. The computational costs are reported, but the source of the gains remains unresolved. The paper reports 11–15 times as many model calls per unit and approximately 1.9–2.6 times the story cost of native recap. These are substantial increases. Thus, Table 1 demonstrates that a more expensive pipeline improves results, but does not sufficiently isolate whether the gains come from the explicit state representation, fuller access to upstream information, or additional inference and checking. In addition, the limited human checks do not establish the reliability of the judgment pipeline across the evaluated state types and methods.

Questions for authors

1/ In my own trials, general-purpose agent tools such as Codex have already produced outcomes similar to those of the authors' framework. What is distinctive about this framework, and what value does it add? Should the experimental evaluation include comparisons with general-purpose agent tools such as Codex to demonstrate which capabilities are beyond those of a general-purpose coding agent? Moreover, the proposed harness is entirely inference-based, involves no training, and appears relatively simple. In my view, it may fall short of the scope and depth of contribution expected for acceptance at ICLR.

2/ If the intended main contribution is the empirical analysis rather than the harness, I also find the analysis limited. For example, regarding gain attribution and inference budget, which reported comparison most closely matches Scripty's access to upstream information and its inference and checking budget? Beyond prompt length, what evidence separates the value of the state representation from fuller information access and additional computation? Please state which factors remain uncontrolled and which conclusions they limit.

3/ For the L1–L0 comparison in Table 2, how many evaluated cases use a successfully validated reference variant, retain a previous reference image, or retain a variant that remains flagged after checking? How do the CUC and WUC changes distribute across these categories? This would clarify when the additional image-channel workload produces gains and when it introduces losses.

4/ For Table 1, which fields are used for final scoring, retry decisions, and answer selection? Please report the number of affected facts by method. If the mismatch is logging-only, explain why it cannot affect these steps. Otherwise, please state whether reconstructing judgments with consistent answers, rationales, and attempt identifiers changes the main paired CUC and Joint CUC effects and their uncertainty.

Comments on Writing and Presentation are following:

a/ Some figure labels should begin with a capital letter rather than a lowercase letter, such as "necklace established" in Figure 1. The meaning of each label should also be explained clearly. Although I can infer what "necklace established" means from the context, it is initially confusing. Similar issues occur in Figures 2 and 3.

b/ Figure references should not always be placed in parentheses. For example, instead of "The typical harness (Figure 2) runs a planner," I suggest "As shown in Figure 2, the typical harness runs a planner." Integrating the reference into the sentence would reduce the excessive use of parentheses throughout the paper.

c/ The Related Work section is too brief. It does not develop a coherent narrative or clearly establish the significance and importance of the proposed work.

d/ All symbols should be explicitly defined, including F in Equation (1). Equation (1) could also be split into two separately numbered equations that can be cited individually, rather than grouping both expressions together.

Reviewer_5mRAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Scripty, an external continuity module for story-level video-generation pipelines. Scripty constructs a storyboard-grounded, shot-indexed state record from upstream storyboards, asset metadata, and reference images. The record is used through two channels: textual state constraints injected into the host writer (L0), and generated reference-image variants for states not represented by existing assets (L1). The module attaches through host-specific adapters without modifying the host source code or the underlying video model. The paper also introduces StateHold, a benchmark for testing whether a later generation unit depicts a state established earlier in the storyboard. On 42 stories and 127 cross-unit facts, Scripty L1 obtains a 48.8% pass rate compared with 26.0% for the native recap baseline. L0 obtains 39.4%, and adding the reference-image channel improves L1 over L0 by 9.4 percentage points. Within-unit performance is broadly maintained, although the L1 channel can cause regressions in some settings. Experiments on two public hosts are small and exploratory. A human viewing-preference study does not show a significant perceptual continuity benefit.

Critical strengths
  1. A clear and practically motivated decomposition of continuity control. Scripty makes persistent storyboard state explicit and uses it to control both text prompts and reference images. The adapter-based design is attractive because it can be added to an existing generation harness without modifying the host or video model.

  2. Careful experimental disclosure and limitation analysis. The authors provide frozen question banks, model and prompt provenance, sensitivity analyses, cost estimates, and extensive discussion of failure modes. The paper is appropriately cautious about not claiming improved viewer-perceived continuity, reliable state expiry, or clean causal attribution when the experiments do not establish them.

Critical weaknesses
  1. The main comparison does not isolate the effect of the proposed state representation. Scripty has access to a richer and more structured information pipeline and uses substantially more model calls, checking, and retries than the native recap baseline. The length-matched history-summary control does not match information access, image access, checker behavior, or compute. Therefore, the 22.8-point improvement may reflect additional upstream information, verification, and computation rather than the specific state-record mechanism.

  2. The primary evaluation is heavily dependent on a synthetic benchmark and an incompletely validated automatic judge. StateHold facts and questions are author/model-produced and may be aligned with the representation that Scripty explicitly maintains. The judge calibration covers only baseline arms, uses earlier question wording, and has a small human sample. The paper also reports retry-provenance defects and substantial judge instability on repeated passes. Moreover, the stricter joint metric falls to 31.5% for Scripty versus 13.4% for recap, and the human viewing-preference study finds no significant benefit. Thus, the benchmark result does not yet establish robust end-to-end or perceptual continuity improvement.

Questions for authors
  1. Can the authors provide an ablation in which the native recap baseline receives the same full storyboard, reference-image information, model-call budget, and prompt-checking budget as Scripty, while varying only the state representation and injection format? This is necessary to separate the state-record contribution from information and computation advantages.

  2. How do the main conclusions change when all final frozen question wording is independently human-rated for both baseline and Scripty outputs, with retry answers and reasons stored in a way that guarantees final-answer provenance?

  3. Can the authors report results on a benchmark whose facts are authored independently of Scripty’s state schema, including more long-gap cases and explicit state-expiry cases, and release enough host/media material for independent end-to-end reproduction?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.