acceptodds
Withdrawn from ICLR 2027

Manga2Animate: An Agentic Framework for Faithful Manga Animation with Script and Storyboard Planning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Adapting existing manga into coherent animation that stays faithful to the source is a valuable yet challenging task. Existing methods either access the manga mainly through text, losing its characters and art style, or feed manga pages directly into video generation models, which misinterpret the manga's structure and produce incorrect storylines. In this paper, we propose **Manga2Animate**, a multi-agent framework that grounds every stage of generation in the source manga. We decompose the task into two intermediate representations, an animation script and a visual storyboard. Each representation is produced by a creator agent and reviewed against the source panels by a paired reviewer agent. We tailor representation-specific selection and dual-level feedback to progressively refine both before synthesizing the final animation. Extensive experiments and a user study demonstrate that Manga2Animate produces animations more faithful to the source manga and more consistent across shots than state-of-the-art approaches.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Manga-to-Animation, Multi-Agent Framework, Visual Storytelling, Storyboard Generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_QSQeOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focuses on the manga-to-animation task. They find that text-based pipelines lose visual fidelity, while direct video generation misinterprets panel order and dialogue. Thus, they propose Manga2Animate, a multi-agent framework that iteratively refines animation scripts and storyboards through creator-reviewer collaboration and dual-level feedback. To validate their method, they conduct several experiments, including baseline comparisons, component ablations, comparisons across generators, and a 37-participant user study. The experimental results demonstrate that Manga2Animate better preserves the source plot, character identities, and art style while improving consistency across shots.

Critical strengths
  1. This paper proposes Manga2Animate, which is the first automated framework that turns existing manga into animation.

  2. This paper establishes an evaluation protocol for manga-to-animation generation, which paves the way for future studies.

  3. They test their method on the proposed benchmark and further validate their results through a user study.

Critical weaknesses
  1. The introduction lacks citations when characterizing prior methods. Its broad claim that current video models cannot understand manga also needs clearer qualification. For example, Figure 2 illustrates specific failures but does not establish this broader conclusion. The authors should cite the relevant methods and clarify the scope and evidence behind these claims.

  2. Table 2 supports the overall benefit of the reviewer-equipped pipeline (Sec. 4.7). Removing a reviewer also reduces the candidate counts. A comparison with equal candidate counts would clarify the added value of reviewer selection.

  3. Section 3.4 adopts two refinement rounds while motivating this choice by API cost control. Table A4 only compares one and two rounds. No analysis of quality versus API cost explains why refinement stops after two rounds.

  4. Minor suggestion: Specific agent models are listed in the Appendix. Briefly naming them in the main text would make the system's external model dependencies clearer.

  5. The paper claims that their method can faithfully turn existing manga into animation. However, the video evaluation lacks separate assessments of source-grounded action and expression fidelity. Motion metrics and frame aesthetics do not establish faithful enactment of the manga. Figure 7 reports broad preferences without dedicated action, expression, or timing criteria.

Questions for authors
  1. Could the authors review and revise the introduction, adding appropriate citations to claims that require supporting references? In particular, please check statements about prior methods and their limitations.

  2. Could the authors compare reviewer selection with random selection using the same number of candidates?

  3. Could the authors compare generation quality and API cost across different refinement-round counts? Please explain the rationale for choosing N = 2.

  4. Could the authors briefly list the LLM backends for the creator and reviewer agents in the main text?

  5. Could the authors separately evaluate source-grounded action and expression fidelity, alongside motion timing and naturalness? Task-specific metrics or targeted human ratings would help.

Reviewer_gD6QOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper tackles manga-to-animation, where existing routes fail in complementary ways: storytelling pipelines access the manga mainly through text and lose its art style and character identity, while feeding manga pages directly into a video model misreads panel order and speech balloons, producing wrong storylines or panels pasted as frames. It proposes Manga2Animate, a multi-agent framework that inserts two intermediate representations between the manga and the video model — an animation script describing what happens per panel and a visual storyboard showing how each shot should look — each built by a creator agent, checked against the source panels by a paired reviewer agent, and refined with dual-level feedback, so the video model only has to animate. The authors also establish an evaluation protocol for this task, under which Manga2Animate outperforms storytelling and direct-generation baselines even when they are given the same script, with a user study selecting its outputs best overall in over 90% of responses.

Critical strengths
  1. The paper identifies two failure modes of existing routes: manga→text→video loses the art style and character identity, while manga→video misreads panel order and speech balloons. The proposed Manga2Animate answers both: a script fixes plot and dialogue, a storyboard fixes appearance and style, both checked against the source panels, so the video model only needs to animate and never has to understand the manga.

  2. The framework is model-agnostic and evaluated under multiple image/video combinations (Gemini + Seedance 2.0, FLUX.2 + MiniMax-H3), so the improvement can be attributed to planning rather than to any particular backbone.

Critical weaknesses
  1. The limitations note that outputs vary across runs, yet the evaluation set contains only eight stories of six panels, far too small to support reliable conclusions. None of the results reported variance. More importantly, the improvement is not consistent across metrics. On several of them (e.g., motion smoothness and Waifu-v3), the method performs no better or even worse than the baselines, which makes the overall claim fragile.

  2. Most ablations (Section 4.7) are relegated to the appendix, which undermines the self-consistency and readability of the main text.

Questions for authors
  1. In Table 1, why is the proposed method's motion smoothness lower than AniMaker's and ViMax's, and why does its Waifu-v3 aesthetic score fall below those of the direct manga-to-video baselines? Are these regressions an accepted cost of stricter alignment to the source panels?

  2. Your setting costs about 25 minutes and 30 VLM calls versus under 4 minutes for direct manga-to-video. Could you add a budget-matched baseline, such as direct generation with N samples (Best-of-N), to isolate the gain from planning rather than from extra compute?

  3. The framework is presented only for manga-to-animation. Do you expect the proposed framework to transfer to other structured visual inputs?

Reviewer_9EfnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Manga2Animate turns existing manga pages into faithful multi-shot animation by inserting two source-grounded intermediate representations — an animation script and a visual storyboard — each generated by a creator agent and reviewed against the source panels by a paired reviewer, refined over two rounds. Its distinctive choices are compositional clip-wise script selection (reassembling the best clip description per panel) and routing storyboard-level feedback back to the script. It outperforms storyboard and video baselines on self-adapted fidelity metrics. However, the quantitative case rests on eight stories in single runs with no uncertainty, and the key ablations have small or sign-inverted margins.

Critical strengths
  1. Clean, source-grounded decomposition with explicit traceability. The script + storyboard decomposition, with a one-to-one manga panel → script clip → storyboard panel correspondence, turns an otherwise loose "be faithful" goal into components that can be individually checked against the source. This is a thoughtful framing of the task.
  2. Compositional clip-wise selection is a concrete design contribution. Exploiting the locality of script errors — different candidates fail on different panels — is a non-obvious and practically useful choice. Experimental results shows clip-wise selection lifting Waifu-v3 from 4.423 to 5.090 and DD from 5.951 to 9.457 relative to script-wise selection.
  3. Independent user study reports strong preferences and the fidelity advantage holds for both an open-source (FLUX.2) and a closed-source (Gemini) image backend.
Critical weaknesses
  1. Single-run, eight stories, no uncertainty. The whole quantitative case rests on eight stories of six panels (Sec. 4.1), and Tables 1, 2, A1–A5 report point estimates with no SD, CI, seed, or statistical test. The paper itself concedes outputs "may vary across runs" (Limitations, p. 9). A 0.025 I2VS gap (0.791 full vs. 0.766 w/o Script Reviewer, Table 2 p. 9) is thus not yet evidence of a causal contribution versus noise — a fundamental gap, not a cosmetic omission.

  2. The "smoking-gun" ablations have small, sign-inconsistent margins. Removing the Script Reviewer raises MS (0.963 vs. 0.960) and Shadow-v2 (0.523 vs. 0.518) (Table 2 p. 9); removing feedback raises OCCM (72.6 vs. 72.2) and Waifu-v3 (3.813 vs. 3.798) (Table A3 p. 20). The "complementary feedback" claim rests on CSD 0.605 → 0.597 / 0.588 (0.008–0.017). With one stochastic run and no error bars, these margins are consistent with run-to-run variance, not the causal necessity of any reviewer. Reverse reading (per this repository's rigor rule): the "largest I2VS drop" may only mean I2VS is most sensitive to script fidelity, not that the reviewer is essential relative to SOTA.

  3. Self-defined metrics; aesthetics point the other way. MangaEval is author-adapted (DINO→DINOv3, binary→continuous DD, CLIP→SigLIP, GPT-4.1 PA). The win is on fidelity metrics (I2VS, I2VB, CIDS, PA, OCCM), but the baselines win the anime aesthetic predictors: Story-Iter Waifu-v3 6.308, StoryDiffusion 5.789 vs. Ours 3.798 (Table A1 p. 19). The win is thus concentrated in metrics the authors redefined, with the user study as the main independent corroboration.

  4. "Over 90% for both storyboards and videos" is imprecise. Fig. 7 (p. 8) shows three of five video criteria below 90%: Character Identity 88.5%, Manga Plot 89.5%, Absence of AI Hallucination 86.8%; only video Overall hits 90.2%. If "best overall" means the Overall criterion specifically, it is true (storyboard 92.9%, video 90.2%), but the phrasing reads as all criteria > 90%.

Questions for authors
  • For each configuration in Tables 1, 2, A1–A5, are multiple runs available? Please report per-story or per-metric standard deviations / confidence intervals, the random seed, and the number of runs. This is the single question most likely to change my assessment.
  • In Table 2 and Table A3, several ablation variants score higher than the full model on MS, Shadow-v2, OCCM, or Waifu-v3. Are these margins within the observed run-to-run variance? If so, the ablation conclusions should be stated as trends, not causal contributions.
  • Does "over 90% of responses, for both storyboards and videos" refer specifically to the Overall criterion (storyboard 92.9%, video 90.2%), or to all criteria? Please align the abstract with Fig. 7, where video Character Identity (88.5%), Manga Plot (89.5%), and Absence of AI Hallucination (86.8%) are below 90%.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.