acceptodds
Withdrawn from ICLR 2027

MAniCast: Multi-Shot Animation Creation with Structured Character-Bound Scripts

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Creating controllable multi-shot animations remains difficult for non-expert users because existing approaches typically require dense frame-level sketches, poses, or keyframes produced with professional drawing and animation skills. A more intuitive alternative is to create animations using only text descriptions and reference images. However, integrating characters defined by text descriptions and reference images with textual narratives remains challenging, particularly in multi-character, multi-shot animation creation. To address this problem, we present MAniCast, the first framework to combine modality-unified character casting with explicit identity-to-role binding for multi-shot animation generation. Structured Character-Bound Scripts combine story- and shot-level instructions with a shared character bank in which each identity is defined by either a detailed text description or a reference image, while explicit placeholders assign selected identities to roles and actions across shots. To realize these assignments at the network level, we introduce Character-Bound Attention-over-Attention (AoA), which derives identity features from textual or visual character definitions and injects them into the corresponding placeholders in the narrative. We further construct MAniScript, a multi-shot animation dataset annotated with Structured Character-Bound Scripts, each of which contains a character bank of text- or image-defined identities. Experiments demonstrate that MAniCast outperforms prior text-controlled and reference-based baselines in casting text- and image-defined characters, executing identity-bound actions, and generating complex interactions across shots. Our model and dataset will be publicly released upon acceptance.

Primary area:
generative models
Keywords:
Multi-Shot Animation, Character Casting, Multi-modal Conditioning, Diffusion Transformers

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_cQSFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MAniCast addresses multi-shot animation generation with characters defined by either text descriptions or reference images. It uses a shared character bank and explicit placeholders in shot-level scripts. The main technical component, Character-Bound Attention-over-Attention (AoA), injects query-dependent character features into the cross-attention values at these placeholders. The model is finetuned from Wan 2.2 on 105,906 multi-shot animation videos. Evaluation includes comparisons on 1,000 held-out sequences, AoA ablations, and a user study.

Critical strengths
  1. Mixing text-defined and image-defined characters within the same story is a useful setting. The shared character bank and explicit placeholders provide a straightforward way to specify character appearances and assign actions across shots.
  2. The AoA ablations show improvements on relevant task metrics. Text AoA improves attribute fidelity and event accuracy, while adding Image AoA further improves casting and cross-shot identity scores. These results support the usefulness of the module within the proposed system.
Critical weaknesses
  1. The novelty is limited relative to existing work. Several methods discussed in Section 2 already provide closely related components. Aura and Script-a-Video separate character definitions from shot narratives, while HunyuanCustom and PolyVivid insert image features into textual conditions. The main addition appears to be query-dependent value injection for both text-defined and image-defined characters. The AoA ablations show that the module helps within MAniCast, but do not show that query-dependent value injection is more effective than existing ways of linking character descriptions or reference images to their corresponding roles in the prompt. The comparisons also leave gaps. MultiShotMaster and AnimeShooterGen are discussed but omitted from Table 1 without explanation. MAVIN and UnityShots are not discussed, even though both support reference-based character customization in multi-shot generation. The paper therefore does not clearly establish what additional capability is provided by its mixed text/image casting setup.
  2. Important dataset and evaluation details are missing. MAniScript is a main contribution, yet Section 5 provides little information about its sources, annotation procedure, or identity verification across shots. The reference-image repainting mentioned in the AI use statement is also unexplained. It is therefore difficult to assess the reliability of the character annotations. The paper does not explain how each baseline receives equivalent character information, particularly text-defined characters, or how single-shot methods are used in the multi-shot evaluation. The five task-specific metrics are described in lines 344–357, but the VLM prompts and temporal sampling protocol are not provided, and the agreement of the resulting automatic judgments with human annotations is not reported. The separate preference study does not validate these automatic judgments. Moreover, its overall-quality result is nearly tied, with 171 votes for MAniCast and 169 for Kaleido in Figure 5.
Questions for authors
  1. How was MAniScript constructed and checked? In particular, how were identities matched across shots, and how were the repainted reference images verified?
  2. What inputs were provided to each baseline for text-defined and image-defined characters? How were the single-shot methods used to produce the multi-shot results?
  3. How was the VLM-assisted task-specific evaluation described in lines 344–357 implemented and validated? In particular, what prompts and temporal sampling protocol were used, and how well do the automatic judgments agree with human annotations?
Reviewer_kxubOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents MAniCast, a framework for controllable multi-shot animation generation using Structured Character-Bound Scripts. The input consists of story- and shot-level instructions together with a shared character bank, in which characters can be defined by either text descriptions or reference images. Explicit placeholders bind these identities to narrative roles and actions. To implement this binding, the paper proposes Character-Bound Attention-over-Attention (AoA), which derives identity-specific textual or visual features and injects them into the corresponding narrative placeholders. The paper also introduces MAniScript, a dataset of multi-shot animation videos with structured character-bound annotations. Experiments on 1,000 animation sequences report improvements in character casting, identity-bound event accuracy, and cross-shot identity consistency over several text- and reference-controlled baselines. Ablations and a user study provide additional evidence for the proposed design.

Critical strengths
  1. Clear Motivation.

  2. Useful formulation of character control. The shared character bank and explicit role placeholders make a difficult multi-character, multi-shot control problem concrete and usable. The formulation allows users to define an identity once, reuse it across shots, and explicitly specify which character should perform each action.

Critical weaknesses
  1. Comparison fairness is insufficiently established. MAniCast is fully fine-tuned on a large animation corpus, while comparable domain adaptation is not documented for the external baselines. The baselines also differ in their support for character banks, placeholders, reference images, and multi-shot generation. These differences make it difficult to attribute the reported gains specifically to AoA. Moreover, MAniCast underperforms some baselines in imaging quality, aesthetic quality, and reference similarity, suggesting a possible trade-off between controllability and visual fidelity.

  2. The reliability of the task-specific evaluation remains unclear. Several important details are unspecified, for example:

  • How are “observable events” defined?
  • If an action is unobservable because the required character is missing, is it counted as a failure?
  • Are samples with failed identity confirmation excluded from Matched Reference Similarity?
  • Are the exclusion rates consistent across methods?
  1. The novelty appears limited.
Questions for authors
  1. After removing AoA, are the character images and the identity-to-placeholder associations provided to the model through another pathway? If not, the ablation conflates access to character information with the effectiveness of the proposed routing mechanism, weakening the evidence specifically supporting AoA?

  2. How are the character bank, textual definitions, reference images, and shot instructions adapted for each external baseline, especially for systems without native placeholders or multi-shot generation?

  3. Could the authors clarify the following details of the Task-Specific Metrics? (refer to Critical Weaknesses 2.)

Reviewer_mrncOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a framework for generating multi-shot animations with consistent character identities and accurate character-action assignments. Specifically, it introduces a structured representation and an attention mechanism to improve tne character consistency across shots and actions. Experiment results show that the proposed method improves character casting, identity-bound action accuracy, and cross-shot identity consistency over existing baselines.

Critical strengths

+) Intuitive and practical formulation. The proposed structured scripts method provides a clean separation across character, shots, and story, making the problem easy to tackle.

+) Comprehensive evaluation. The proposed method was compared to many baseline benchmarks with different metrics to show the effectiveness of the method.

Critical weaknesses

-) Novelty might be limited. Structured scripts are already widely adopted in multi-shot video generation. The paper should more clearly distinguish its contribution from prior structured representations.

-) Human evaluation scale is limited. Only 25 video cases, and the preference margins are relatively small.

Questions for authors

a) How do you maintain the original model's generation capability given the attention mechanism was changed? Assuming the data used to finetune the model can hardly be the same quality as training the WAN model?

b) What makes the proposed AoA mechanism more effective than simpler alternatives, e.g, injecting character embeddings into placeholder tokens or using conventional cross-attention with character-specific tokens?

c) Clarification of difference of the proposed structured scripts against existing methods.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.