acceptodds
Under review as a conference paper at ICLR 2027

AnimateBanana: Transforming Process Diagrams into Animated Narratives

45 views

Abstract

Diagrams are often used to visually communicate scientific processes. However, conveying process dynamics and explanatory flow from a static diagram is challenging. Although animated narratives improve comprehension and engagement, creating them from a diagram is a manual and time-consuming process. We present **AnimateBanana, a framework for automatically transforming process-oriented scientific diagrams into animated narratives**. Given a diagram and a user-specified presentation style, AnimateBanana first constructs an editable, animation-aware programmatic representation of the diagram. Specialized multimodal agents progressively reason over the source diagram and structured intermediate representations to determine presentation structure, narration, and animation. Critic-mediated feedback is used to refine intermediate outputs. **The resulting narratives are faithful to the source diagram, controllable, and deployable across multiple presentation formats (PDF, SVG, GIF, MP4, and PPTX). To support this new task, we introduce AnimateBench, a curated benchmark of 5000 process-oriented diagrams with verified animated narratives** spanning diverse diagram families, presentation styles, and levels of structural complexity, including expert-derived reference narratives where available. We further introduce task-specific metrics for evaluating final narratives. Automated evaluation and controlled user studies show that AnimateBanana substantially outperforms competing baselines across animation validity and narrative quality. Ablations further support explicit intermediate representations, source diagram grounding, and stage-wise critic feedback as important design choices for reliable animated narrative generation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Scientific Diagrams, Animated Narratives, Multimodal Agents, Programmatic Representations, Diagram Understanding, Science communication

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_NEDLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces AnimateBanana, a framework for transforming static, process-oriented scientific diagrams into animated and narrated presentations. Given a diagram and a user-specified animation style, the method first constructs an editable, animation-aware programmatic representation, then uses specialized multimodal agents to plan the presentation sequence and narration and generate the final animation. Critic-mediated feedback is used to refine intermediate outputs. The resulting narratives can be exported to formats including SVG, GIF, MP4, PPTX, and animated PDF.

The paper also introduces AnimateBench, comprising 5,000 curated diagrams with verified animated narratives and 500 human-authored presentations extracted from related conference talks, together with task-specific metrics for animation validity and narrative quality. The method is evaluated through MLLM-based automatic evaluation, comparisons with direct-generation and alternative-backbone baselines, component ablations, and a user study. The authors report substantial improvements over the evaluated baselines.

Critical strengths
  1. Novel and practically relevant task formulation. While prior work has explored paper-to-slide generation, narrated figure walkthroughs, and SVG animation, this paper focuses on converting process-oriented scientific diagrams into editable, narrated animations with controllable styles and multiple export formats. The task addresses a real and labor-intensive scientific communication workflow.

  2. Structured and multi-format generation. The method explicitly represents diagram elements, presentation order, and step-level narration before rendering the final animation. This design supports controllable animation styles and export to multiple presentation formats rather than only producing a raster video.

Critical weaknesses

1. Questionable benchmark independence and insufficient annotation transparency. AnimateBench is bootstrapped using AnimateBanana itself, which may introduce method-dependent biases into the reference narratives and task formulation. The paper also provides almost no information about the human verification process, including the number and expertise of verifiers, the number of verifiers per sample, cross-checking or adjudication, modification rates, and annotation workload. Given the claimed scale of 5,000 samples, these omissions make the quality and independence of the benchmark difficult to assess.

2. Insufficiently controlled and reproducible evaluation. The large performance gap between AnimateBanana and strong closed-source models is difficult to interpret. The paper provides almost no details about how the closed-source direct-generation baselines were implemented. Missing information includes, but is not limited to, the prompts, model and API versions, provided inputs and context, expected output format, and, where applicable, tool access, interaction procedure, rendering process, and treatment of failed generations. Moreover, the complete MLLM-judge prompts and implementation are unavailable, while the human quality evaluation covers only 25 diagrams per system. Together, these issues substantially weaken the reliability of the main empirical conclusions.

Questions for authors
  1. How was the human verification of AnimateBench conducted? In particular, how many verifiers were involved, what was their expertise, how many verifiers examined each sample, was any cross-checking or adjudication performed, and what fraction of the automatically generated narratives required modification?

  2. What role do the verified animated narratives in AnimateBench play in the reported evaluation? Are they used as reference annotations by any of the main metrics, and if not, what is their purpose in the benchmark?

  3. What exact protocol was used for the closed-source direct-generation baselines? Please provide the exact prompts, model/API versions, and generation and rendering code used for the closed-source direct-generation baselines. These details are necessary to assess and reproduce the comparison. Please also provide complete outputs for several representative end-to-end baseline examples, rather than only extracted frames and judge results.

  4. What was the complete MLLM-judge protocol, including the prompts, model version and settings, frame-sampling procedure, presentation-step extraction, and input format?

  5. How should the reported frame/step inconsistencies be interpreted?In data/evaluation_runs.json, the sample Claude Code with Opus 5/CVPR_2025_arch00012 has 15 extracted frames but 20 judged steps, with the last five frames set to null while still contributing to the score. Please clarify this inconsistency and whether it affects the reported results.

  6. The MLLM-judge validation covers only 50 outputs for cross-judge agreement and 75 human-matched outputs. How were these sample sizes and samples determined, and what evidence shows that the reported agreement is reliable and representative?

  7. The human study compares AnimateBanana, AnimateBanana with a Qwen backbone, and Gemini 3.1 Pro direct generation. What is the rationale for mixing backbone and generation-method comparisons, and what conclusion is this design intended to support?

Reviewer_qwwVOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Scientific process diagrams have complex workflows, architectures, and causal relationships in a static visual form. Manually producing animated explanations is time-consuming and requires specialized skills. Therefore, AnimateBanana proposes a training-free, multi-agent framework that transforms a static scientific diagram into an editable, narrated animation. Specifically, it has three stages. Stage 1 (diagram transmuter) reconstructs diagrams as animation-aware SVG code with ids, hierarchy, and preserved raster content. Stage 2 (animation planner) parses the representation to structured XML, finds a coherent presentation sequence, and generates in-sync narration. Stage 3 (diagram animator) then produces the requested animation with iterative visual and structural feedback.

The main contribution is an integrated diagram to animation pipeline, AnimateBench (5000 scientific diagrams plus 500 additional presentation references), and task specific evaluation metrics such as visual fidelity (VF), style compliance (ASC) and selection sensibility, granularity and pacing, and narration alignment, all combined as animated narrative (AN).

For experiments, the paper uses Gemini-3.7 Flash as the primary LLM, and compares against five direct-generation models, and includes alternative LLMs and component ablations. On AnimateBench, AnimateBanana achieves 0.877 AN, compared with 0.131 for the strongest direct-generation baseline. The paper also has human evaluation which involves 38 participants reporting an AN score of 0.855 and a 61.6% overall preference win rate.

Critical strengths

The task is well motivated and the validity-gated metric is a good design for it. Requiring animations to pass visual fidelity and style compliance before scoring narrative quality is, I believe, appropriate for this task. The per-step scoring is straightforward to interpret and could be useful for future work. The supplement also provides criterion-level judge rationales, making the evaluation more transparent and allowing individual scoring decisions to be inspected. The website is also helpful and provides additional information.

Critical weaknesses
  1. It is unclear whether the comparisons with the baselines is fair. The large performance gap in Table 1 may partly come from how the evaluation handles raster images, rather than the proposed multi-stage design. In the supplement’s evaluation walkthrough (the zip file that was provided), direct baselines fail the VF gate mostly by replacing photographic regions with vector icons, and 83.1% of AnimateBench diagrams contain rasters. AnimateBanana’s own VF rises from like mid-0.9s to 0.988 on raster-free diagrams (supplement, stratified results). Additionally, the supplement’s evaluation walkthrough (evaluation_runs.json, five samples per system) shows the direct baseline runs with no requested style and empty narration at every step, yet ASC and NA scores are reported for them. Requires clarification please. I think the paper also lacks a baseline that uses an existing diagram-to-SVG method (StarVector, DeTikZify, VFig, all cited) with an existing SVG animation method (Keyframer, LogoMotion, VAnim, all cited). Without these comparisons, it is difficult to determine how much improvement actually comes from the proposed architecture.

  2. Also, important experimental results are not explained completely. Table 1 shows that several large models, including Qwen3-VL-235B and DeepSeek-V4 Pro, perform extremely poorly within the pipeline and scoring below 0.04 AN while a 27B Qwen scores 0.367. The paper sees this as scale being insufficient, but does not go in detail where or why they fail. There's a natural question about whether the method generalizes across models or depends on LLM-specific prompting. Without that breakdown, the LLM rows cannot support the paper’s generality claim.

Questions for authors
  1. What are the direct baselines' VF and AN scores on diagrams without raster content? Section 6 states baselines received the same diagram and auxiliary context. Did they also receive the same animation-style and narration instructions?
  2. Why is there no comparison with a pipeline combining existing diagram-to-SVG and SVG-animation methods?
  3. Why do several larger models achieve near-zero scores in Table 1? At which stage do they fail, and are these failures caused by output formatting, execution errors, or poor animation quality?
  4. Section 4 says each candidate narrative was inspected. How many of the 5,000 narratives were manually checked, how many annotators were involved, and what percentage required corrections? Was the reported 88.2% improvement from verification tested blindly?
Reviewer_8X37OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

AnimateBanana converts scientific process diagrams into narrated animations through SVG reconstruction, sequence planning, and iterative refinement. The paper introduces AnimateBench and evaluates the pipeline using automatic metrics, ablations, and human studies.

Critical strengths
  1. The pipeline addresses a practical task through a coherent combination of structured SVGs and narrative planning.

  2. AnimateBench, ablations, and human studies provide useful evidence of effectiveness within the evaluated setting.

Critical weaknesses
  1. Evidence of generalization remains limited. The within-benchmark results are encouraging, but more representative cases with unfamiliar diagram structures would help establish broader applicability. Source-level separation between development and evaluation data also needs clarification.

  2. Semantic reliability needs stronger validation. Visual validity and user preference provide limited evidence of factual correctness on unfamiliar inputs. Aggregate scores also combine rendering validity with narrative quality, obscuring distinct failure modes.

Questions for authors
  1. How are development and evaluation sources separated? Could you provide a small, diverse set of independently collected diagrams evaluated with fixed prompts and settings, including successes and failures?

  2. Can independent assessors check factual grounding and explanation order against the source material and report agreement with the automatic evaluator?

If the authors can address my main concerns and answer my questions well, I will be very happy to raise my rating.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Closing Statement and Summary

We sincerely thank all the reviewers for their time and thoughtful feedback. We are encouraged that the reviews have recognised the novelty and practical relevance of our work and found our evaluation idea to be useful for future work.

During the discussion, we addressed all the reviewers' concerns through additional analyses, experiments, and clarifications. For ease of reference, we summarise the key points below.

ReviewerWhat the reviewer appreciatedMain concernsHow we addressed them
8X37Practicality of the pipeline; human studies providing evidence of effectiveness.Generalisation, source separation, and clarification on the user study.Added evaluation on 27 unfamiliar diagrams spanning diverse domains, clarified source-level separation between development and evaluation sets, and provided further evidence of grounding and narrative-quality assessment.
qwwVWell-motivated task, appropriate evaluation design, and useful supplementary website.Baseline fairness and suggested experiment to assess pipeline contribution.Provided additional raster/non-raster comparisons, clarified baseline settings and backbone differences, and conducted the suggested agentic harness experiment. The experiment shows that stage descriptions as invocable skills/tools alone do not lead agents to independently discover our proposed workflow or reconstruct the structured representations and information dependencies underlying AnimateBanana.
NEDLNovel and practically relevant task; structured representation of pipeline intermediates, support for controllable animation styles, and export to multiple presentation formats.Benchmark transparency and evaluation validity.Detailed the annotation and human-verification process, clarified the role and value of human-verified sequences, narratives, and diagram XML, provided complete baseline outputs, prompts, and detailed evaluation protocols, and addressed clarifications regarding the user-study results.

We have incorporated additional requested analyses into the supplementary material and companion website, with representative visual examples to make the results easier to inspect.

We hope that our responses and the additional evidence address the reviewers' concerns and provide a clearer picture of the contributions of our work. We respectfully request that these be taken into consideration in the final assessment of our submission.

Sincerely,

Authors of Submission 4633

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.