acceptodds
Under review as a conference paper at ICLR 2027

DyCraft: An Agentic System for Long-Horizon 4D Scene Generation

20 views

Abstract

While frontier coding agents can generate high-quality 3D scenes and articulated assets, they struggle to create coherent long-horizon animations. When equipped with general-purpose harnesses, they frequently produce outputs that are physically implausible, generating objects that lack physical support, interpenetrate, or move unnaturally. Such bare agents struggle to repair these errors because they default to scripting the full animation in one pass, making it difficult to disentangle interdependent temporal and spatial constraints. Drawing on the practice of human animators, we introduce DyCraft, a system for 4D scene generation that breaks animation into steps that coding agents can more reliably solve. DyCraft first expands the user's prompt into a detailed plan and builds a static scene. It then creates a 3D storyboard of key animation poses, where each frame is a static 3D scene that the agent can build and debug in isolation. The storyboard decomposes the problem into pairs of verified start and end states, allowing the agent to author local motion between them with a collection of motion authoring tools. We evaluate DyCraft on 16 long-horizon, multi-shot animation tasks covering humanoid robots, human and animal characters, and physical mechanisms. Compared with a native GPT-6 agent in Codex and prior agentic baselines, DyCraft substantially reduces physical violations and generates higher-quality 4D scenes that are more often preferred by human raters.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
4D scene generation, Text-to-4D scene, Agentic framework, Animation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_hwf9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a workflow for long-horizon animation generation. Given a text prompt, the system first generates 3D key poses, then interpolates between them to produce frame-by-frame motion, performs verification, and finally generates a Blender animation.

Critical strengths
  1. The paper is logically well-structured, and the proposed workflow is clear and easy to follow.

  2. The idea of first generating key poses and then interpolating between them is intuitive and straightforward, and the experimental results demonstrate its effectiveness.

Critical weaknesses
  1. Conventional video generation models typically produce photorealistic and natural-looking results. In contrast, the Blender animations generated by DyCraft often appear cartoonish and lack visual realism, as many objects are constructed from basic geometric primitives.

  2. How does DyCraft determine the appropriate 3D key poses for different types of motion? Are these key poses generated directly by an LLM-based agent?

  3. Based on my understanding, DyCraft mainly integrates existing agents and tools, and it has a new workflow that first generates 3D key poses and then interpolates between them to produce continuous motion. However, even this workflow appears to be easily achievable through prompt engineering: instead of instructing the LLM to generate the entire animation directly, the prompt is modified to encourage key-pose generation followed by interpolation using some heuristics. Although the results are good, I find the algorithmic and framework-level novelty rather limited. The proposed approach seems more like a carefully designed prompting strategy than a fundamentally new method or framework.

  4. The ablation studies are limited to qualitative comparisons, without quantitative results to demonstrate the contribution of each component. Quantitative ablations would be necessary to better understand the effectiveness of the proposed workflow and individual modules.

  5. The evaluation is conducted on only 16 text prompts, which I consider insufficient to demonstrate the robustness and generalizability of the proposed method.

  6. I believe the comparison between DyCraft and the bare-agent baseline is not entirely fair. DyCraft has access to various additional tools, such as the CAD authoring interface and humanoid motion skills, which are not equally provided to the bare agent. A more convincing comparison would equip the bare agent with exactly the same tools but without explicitly instructing it to generate key poses before producing continuous motion. Instead, the agent should be allowed to explore how to use these tools independently. This would help distinguish whether the improvements primarily come from the proposed workflow or from the additional tools. Moreover, I believe that current frontier LLMs, such as GPT-6-Astra, are already capable of reasoning about how to effectively use these tools when provided with appropriate instructions.

  7. The authors should also report and compare the API costs and wall-clock generation times of different methods. Since DyCraft introduces additional planning, verification, and refinement steps, it is important to understand the computational overhead associated with the proposed workflow.

Questions for authors

Please see the "Critical Weaknesses" section

2 comments

Dear Reviewer hwf9,

Thank you for the detailed review and constructive feedback. Because of its length, our response spans two comments.

1. (W1) Visual Realism

DyCraft does not just produce video animations. It writes an entire editable Blender project, and its films are renders of that Blender project through the authored cameras. This enables far more downstream applications compared to generated videos alone, including the ability to easily re-pose the scene and render from arbitrary viewpoints. Besides, the animated assets could also be imported into simulators or game engines, the generated kinematic motions could be retargeted onto real robot embodiments, and the Blender assets directly provide synthetic data such as depth, segmentation, and 3D trajectories. As the paper argues, video generative models produce realistic footage, but they are very difficult to control precisely and often contain geometric, temporal, and physical inconsistencies in their outputs.

Our anonymous results page now provides an interactive 4D viewer for all delivered scenes, enabling real-time free-viewpoint control directly in your browser: https://dycraft-anonymous.github.io/#viewer.

2. (W2) How Key Poses Are Chosen

Yes, the agent automatically chooses key poses for each brief without human intervention. In our motion prompt rewriter (Appendix A.1), a transition between key poses is defined as a change in what holds, supports, or moves a body: for example when a robot regrasps a tray, sets a load down partway, or shifts its stance. The same rule covers characters, robots, and mechanisms.

As stated in our method section, the agent first authors each key pose in Blender, renders it, then revises it from the renders and verifier feedback. Before the agent can write the motion between two states, both of those states must pass their verifiers and the agent must inspect their renders (Sec. 3.3).

3. (W3) Prompt Engineering

We directly test this alternative. A fresh-context LLM translated our paper into a detailed workflow prompt (W), which we gave to the bare agent without our tools. We ran the bare agent, W, and Full DyCraft on 8 random briefs with GPT-6-Astra at medium effort, the same code, and an 8-hour budget, and asked raters to compare the W and Full DyCraft films for each brief. Every film is at https://dycraft-anonymous.github.io/#ablations

Bare agentWorkflow prompt (W)Full DyCraft
Preferred by raters ↑—34%66%
Penetration ↓55.8%20.7%1.1%

Raters preferred Full DyCraft in 23 of 35 judgments. The workflow prompt leaves penetration at 20.7%, far from Full DyCraft's 1.1%, which is lower on 6 of 8 briefs.

A prompt can describe key poses and checks, but the agent must then build the checks itself. Without our verifiers, the W agent wrote its own collision checks. On 5 of the 8 briefs, it reported that every frame passed them, yet those films still had 4.4% to 72.0% penetration when scored with our final evaluator.

DyCraft instead supplies verifiers that check every key state and every frame of motion.

4. (W4) Quantitative Ablations

We also ran an add-one ablation on an independently chosen set of 8 random briefs. Three independent arms each add prompt rewriting, the verifiers, or 3D storyboarding to the same bare GPT-6-Astra agent, with the model, effort, and time budget fixed across the arms of each brief. The 3D storyboarding arm also includes prompt rewriting and the verifiers, which it depends on. Raters compared each arm's film with the bare agent's film for the same brief.

ArmPreferred over bare agent ↑Penetration ↓
Bare agent—72.1%
Prompt rewriting67% (20 of 30)67.5%
Verifiers62% (16 of 26)34.3%
3D storyboarding58% (23 of 40)24.0%

Raters preferred each arm over the bare agent. The verifiers lower penetration from 72.1% to 34.3%, and the 3D storyboarding arm reaches 24.0%. We plan to extend this ablation to more briefs for the final version.

5. (W5) Benchmark Size

The 16 prompts of the submission already span diverse scenarios and three categories of embodiments. They cover humanoid robots, humans/animals (e.g. a barista, dog, and macaque), and physical mechanisms (e.g. a marble run or boat drawbridge). Each prompt is a complete multi-shot film, while the 4D generation methods we cite evaluate single-shot clips of at most a few hundred frames. The 16 films contain 201 shots and 440 planned key states, with 232 characters and objects in motion. We prioritized coverage across task categories and depth per sequence over raw task count.

In direct response to this concern, we expanded the benchmark to 24 briefs (33.9 minutes of animation in total), balancing the three embodiment categories at 8 briefs each. We show the final DyCraft film for each: https://dycraft-anonymous.github.io/#gallery

In a new user study on all 24 briefs, 109 raters on Prolific compared films in randomly ordered pairs and answered three questions per pair. The table gives the share of judgments preferring DyCraft. Raters preferred DyCraft over VIGA on 23 of the 24 briefs and over the bare agent on 20.

QuestionDyCraft vs. VIGADyCraft vs. Bare agent
Overall 4D realism92.0%75.6%
Physical plausibility88.5%70.1%
Motion quality88.5%71.0%
Pooled over the three questions89.7%72.3%
MetricDyCraftBare agentVIGA
Penetration ↓13.0%80.1%74.9%
Floating ↓2.7%15.7%26.1%

6. (W6) A Bare Agent with the Same Tools

We ran the tool-matched baseline you describe. T gives the bare agent every DyCraft tool, with the same reference documentation for each tool that Full DyCraft's agent receives, and leaves the agent free to explore and use them. T ran with W and Full DyCraft on the same 8 briefs and settings as in Section 3.

Bare agentAll tools, optional (T)Full DyCraft
Preferred by raters ↑—44%56%
Penetration ↓55.8%59.3%1.1%

Raters preferred Full DyCraft in 24 of 43 judgments. With all tools available, T stays at 59.3% penetration, no better than the bare agent's 55.8% and far from Full DyCraft's 1.1%, which is lower on 7 of 8 briefs.

7. (W7) API Cost and Wall-Clock Time

All methods use GPT-6-Astra. We count the tokens of every agent role and price them at standard API rates ($10, $1, and $50 per million uncached input, cached input, and output tokens). Active time includes the turns of the main agent and leaves out idle gaps. The table gives medians over the 24 briefs.

MeasureDyCraftBare agentVIGA
Active time per film (h)6.81.03.3
Cost per film ($)2161953

The bare agent is not cut off by a time limit. It declares completion on its own, often without planning, inspecting, or verifying its work. DyCraft spends its additional time on these steps, which yield better scenes and fewer geometric violations. In a median DyCraft run, the harness's own jobs take 51.8% of the active time, led by rendering at 12.6%, verification at 10.5%, CAD building at 7.4%, and storyboarding at 6.7%. The rest is the agent's own turns, in which it reasons over renders and check results and writes the scene and motion code.

Reviewer_CvYkOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces DyCraft, a system for long-horizon 4D scene generation. In contrast to bare agents, that try to generate the whole animation in one pass, DyCraft breaks the process into steps that coding agents can solve in a better way. Initially, DyCraft expands a user’s prompt into a detailed scene and motion plan and builds a scene in CAD. It then plans key poses and object states as a 3D storyboard, reviews these states and authors the intermediate motion between those states. Lastly, it has a verification stage where it checks that every frame is valid and does not violate basic physical constraints. When compared with other baselines, such as VIGA and a bare agent, on a benchmark of 16 long horizon multi-shot animation briefs, DYCraft achieves superior performance since it creates high quality 4D scenes and reduce physical violations.

Critical strengths
  1. DyCraft generates high quality 4D scenes that do not violate physical constraints. It is clear from the videos in the supplementary material and the qualitative results in Figure 7 that DYCraft is superior compared to VIGA and bare agent.

  2. The paper is well written, easy to follow and has a nice structure.

  3. The authors provide a thorough ablation study in Figure 8 where they show the effect of prompt rewriting, 3D storyboarding, domain specific motion authoring tools and motion verifiers. Thus, they show how each component reduces the physical plausibility violations.

Critical weaknesses
  1. The paper shows only success cases. What happens when DyCraft fails? Could they add a section where they discuss about the limitations?
  2. The whole pipeline seems computationally expensive. The authors include a timeline in Figure 3 where they mention the total time from rewriting stage to final delivery of one scene, but they don’t mention anything about the computation cost and generation time compared to the baselines (VIGA and bare agent) per scene.
  3. Could the authors add more baselines apart from GPT-6-Astra and VIGA? For example, can they evaluate with Gemini 4 Argon or Claude Fable 5.1?
Questions for authors

See the questions in the Critical Weaknesses section. Apart from these could the authors please answer the following questions:

  1. How many iterations do the motion authoring and verification stages usually do? Do the authors define a max number of attempts for these two stages?

  2. What is the maximum duration of a video it can generate? Does the performance drop for longer videos?

1 comment
AuthorsOpenReview ↗

Response to Reviewer CvYk

Dear Reviewer CvYk,

Thank you for the positive assessment and detailed review.

1. (W1) Failure Cases

Thank you for pointing this out. We will add a Limitations section and a failure-case figure to the final version.

A failing verifier blocks clean completion of the task (Sec. 3.4). When the agent is under external time constraints, it may not be able to resolve every failing verifier in the allotted time. In these cases, DyCraft delivers the film with unresolved checks recorded, including remaining small penetrations.

Our penetration and floating metrics measure geometry, and do not yet measure accurate dynamics. Raters judge physical plausibility and motion quality in our user study.

2. (W2) Computation Cost and Generation Time

All methods use GPT-6-Astra. We count the tokens of every agent role and price them at standard API rates. Active time includes the turns of the main agent and leaves out idle gaps. The table gives medians over the 24 briefs.

MeasureDyCraftBare agentVIGA
Active time per film (h)6.81.03.3
Cost per film ($)2161953

When we run the bare agent, we do not enforce a time limit cutoff. The bare agent declares completion on its own, often without planning, inspecting, or verifying its work. DyCraft spends its additional time on these steps, which yield better scenes and fewer geometric violations. In a median DyCraft run, the harness's own jobs take 51.8% of the active time, led by rendering at 12.6%, verification at 10.5%, CAD building at 7.4%, and storyboarding at 6.7%. The rest is the agent's own turns, in which it reasons over renders and check results and writes the scene and motion code.

3. (W3) Other Backbones

We ran bare agents from two other model families as additional baselines on 8 of our briefs: Claude Fable 5.1, and Gemini 4 Carbon (the latest version of Gemini 4 Argon). The table gives their penetration on these briefs. Full DyCraft is lower than the Fable bare agent on 7 of the 8 briefs and lower than the Gemini bare agent on all 8.

GPT-6-Astra bare agentClaude Fable 5.1 bare agentGemini 4 Carbon bare agentFull DyCraft
Penetration ↓55.8%46.4%63.8%1.1%

Their films are on our anonymous results page, next to the GPT-6-Astra bare agent and Full DyCraft: https://dycraft-anonymous.github.io/#backbones

4. (Q1) Iterations and Attempt Limits

Over the 24 briefs, the median DyCraft run made 14 storyboard staging calls, 5.5 storyboard acceptance calls, and 13 verification calls.

DyCraft does not set any per-stage attempt limit. Nothing in the code limits the number of storyboard, verification, or review calls. The agent decides how to spend its time budget, which is stated in the prompt and counts down in each harness reply.

5. (Q2) Maximum Duration and Longer Films

DyCraft sets no hard limit on film length. Its prompt states an expected length but does not require it, and the delivered films range from 24 to 281 seconds, with a median of 74 seconds. The longest shows a humanoid robot cooking scrambled eggs in 281 seconds (6,755 frames at 24 fps), and a robot mopping a break room is 249 seconds. Both are on our anonymous results page: https://dycraft-anonymous.github.io/#gallery

Raters preferred DyCraft more on the longer films. On the 12 briefs with the longest DyCraft films, raters preferred DyCraft over the bare agent in 80.6% of judgments and over VIGA in 94.7%, against 64.0% and 84.6% on the other 12 shortest briefs. The two longest films were preferred over the bare agent in 90.9% and 81.5% of judgments.

Reviewer_R6CLOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

DyCraft is an agentic system for generating editable, long-horizon 4D scenes. The contribution of the paper is a harness system that augments a coding agent with scene and motion prompt rewriting, CAD-based scene construction, 3D storyboarding, domain-specific motion authoring guidance, and geometric verification. The agent first builds and verifies key static poses, then generates continuous motion between them and checks the resulting animation. To demonstrate the effectiveness of the approach, the paper also introduces DyBench, comprising 16 prompts covering humanoid robots, human and animal characters, and physical mechanisms. Using GPT-6-Astra, DyCraft obtains preference rates of 64.9% against the bare agent and 86.9% against VIGA. Its penetration and floating rates are 13.3% and 2.8%, compared with 76.5% and 17.8% for the bare agent. Qualitative ablations illustrate the contribution of the main workflow components.

Critical strengths
  1. The harness addresses the difficulty in long-horizon animation generation. The 3D state verfications and CAD construction and native simulation helps physics realism and photo realism.

  2. The complete system shows substantial gains using Astra. The reductions in geometric violations are large, and human judgments favor DyCraft over both baselines (Tables 1 and 2).

Critical weaknesses

The evidence for individual components is much weaker than the evidence for the full system, i.e, I think component-wise ablations are limited. A lot of the design choices are intuitively sound, but it is not justified. Section 4.5 provides qualitative ablations on the barista example, without aggregate scores or repeated generations across the benchmark. Prompt expansion, CAD tools, motion guidance, verification, and storyboarding change together in the main comparison.

Questions for authors
  1. Can you quantify the importance of storyboarding, rather than only showing barista failures? Which component accounts for the largest change in preference and geometric violation rates?

  2. How do the authoring verifiers differ from the final evaluation checks? Please explain why delivered scenes retain counted penetrations despite completion-blocking verification.

1 comment
AuthorsOpenReview ↗

Response to Reviewer R6CL

Dear Reviewer R6CL,

Thank you for the detailed review. Following your suggestion, we now report a quantitative ablation of each component in Section 1. Section 2 answers your question about the verifiers. In response to Reviewer hwf9, we also expanded our benchmark from 16 to 24 briefs and provide additional results for DyCraft and its baselines. Please see the general response.

1. (W1, Q1) Component Contributions and the 3D Storyboard

To measure the contribution of each component, we ran an add-one ablation on 8 random briefs: 2 humanoid robots, 2 characters, and 4 physical mechanisms. Three independent arms each add prompt rewriting, the verifiers, or 3D storyboarding to the same GPT-6-Astra bare agent, with the model, effort, and time budget fixed across the arms of each brief. The 3D storyboarding arm also includes prompt rewriting and the verifiers, which it depends on. In a user study, 20 raters on Prolific compared each arm's film with the bare agent's film for the same brief, answering one preference question per film pair. Every film is on our anonymous results page: https://dycraft-anonymous.github.io/#ablations

ArmPreferred over bare agent ↑Penetration ↓
Bare agent—72.1%
Prompt rewriting67% (20 of 30)67.5%
Verifiers62% (16 of 26)34.3%
3D storyboarding58% (23 of 40)24.0%

Raters preferred each arm over the bare agent. Prompt rewriting adds the richer scene and motion detail shown in the ablation films of the paper. The verifiers lower penetration from 72.1% to 34.3%. With 3D storyboarding, penetration is 24.0%, below the verifiers alone on 6 of 8 briefs. We plan to extend this ablation to more briefs for the final version.

2. (Q2) Authoring Verifiers and the Final Evaluator

The final evaluator is separate code from the verifiers and was authored independently. It only reuses the harness's geometry export and mesh intersection test functions (since whether two meshes intersect is a deterministic fact). The two differ in what they allow or excuse. The agent can declare a penetration as intended, such as a wheel seated in its rim. These agent-proposed allowances are saved with the film and accepted by the verifiers, but are ignored by the final evaluator.

Delivered films keep counted penetration for two reasons.

  • The evaluator ignores agent-proposed contact allowances. In the 13 films that finished with every check passing, all counted penetration comes from contacts that the agent declared as intended.
  • Some collisions flagged by the verifier may remain unresolved. A failing verifier blocks clean completion of the task (Sec. 3.4), but under operator time constraints a run can end before the agent resolves every verifier-flagged collision. DyCraft then delivers the film with these collisions recorded.

We traced the frames with penetration across all 24 delivered films. 59% fall under the agent's declarations, and 41% are unresolved collisions. None of the traced frames are collisions that the verifiers missed.

Comments on the paper

1 comment

Dear Reviewers and Area Chairs,

We thank the reviewers for their constructive feedback.

DyCraft is an agentic system for long-horizon 4D scene generation. It produces high-quality minute-long Blender animations with fewer physical errors than bare GPT-6-Astra agents, and is preferred by human raters in over 70% of pairwise comparisons. Our films, ablation studies, and an interactive 4D scene viewer are visible on our anonymous results page: https://dycraft-anonymous.github.io/

Reviewers found that our harness generates high-quality 4D scenes with large reductions in physical violations (R6CL, CvYk), provides experimental results that demonstrate the idea's effectiveness (hwf9), and has clear presentation and workflow (R6CL, CvYk, hwf9).

We summarize the new evidence from our rebuttal responses, and include full results in each reviewer thread.

Summary of New Evidence

  • R6CL and hwf9 asked for quantitative evidence for the impact of each component. We ran an add-one ablation on 8 random briefs, in which three independent arms each add prompt rewriting, the verifiers, or 3D storyboarding to the same bare GPT-6-Astra agent. 20 raters on Prolific compared each arm with the bare agent and preferred the arm with prompt rewriting in 67% of judgments, the arm with the verifiers in 62%, and the arm with 3D storyboarding in 58%. Penetration falls from 72.1% for the bare agent to 34.3% with the verifiers and 24.0% with 3D storyboarding. The films are at https://dycraft-anonymous.github.io/#ablations.
  • hwf9 asked whether prompt engineering alone, or the bare agent with the same tools, would match DyCraft's results. On an independently chosen set of 8 random briefs with matched settings, raters preferred Full DyCraft in 66% of judgments over a workflow prompt written from our paper, and in 56% over the bare agent with all DyCraft tools available but optional. On these briefs, Full DyCraft has 1.1% penetration, against 20.7% with the workflow prompt, 59.3% with all DyCraft tools, and 55.8% for the bare agent. Neither alternative comes close to Full DyCraft in penetration.
  • CvYk and hwf9 asked about API cost and runtime of our system. We report both for each method, with a breakdown by stage for DyCraft.

Results on All 24 Briefs

In response to hwf9, we expanded the benchmark from 16 to 24 briefs to increase evaluation diversity, balancing the three embodiment types at 8 briefs each. We show the final DyCraft film for each brief. The films in total contain 33.9 minutes of DyCraft animation: https://dycraft-anonymous.github.io/#gallery

In a new user study on all 24 briefs, 109 raters on Prolific compared films in randomly ordered pairs and answered three questions per pair. The table below shows the percentage of judgments preferring DyCraft. Raters preferred DyCraft over VIGA on 23 of the 24 briefs and over the bare agent on 20.

QuestionDyCraft vs. VIGADyCraft vs. Bare agent
Overall 4D realism92.0%75.6%
Physical plausibility88.5%70.1%
Motion quality88.5%71.0%
Pooled over the three questions89.7%72.3%
MetricDyCraftBare agentVIGA
Penetration ↓13.0%80.1%74.9%
Floating ↓2.7%15.7%26.1%

In the per-reviewer responses, we also clarify how the verifiers differ from the final evaluator, how key poses are chosen, iteration counts and attempt limits, film length, how unresolved checks are handled, and visual realism. We will add these results to the final version of the paper.

Best regards,

Submission 19602 Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.