- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a framework, WorldWeave, that separates persistent 3D world construction from video generation. The method first uses a pretrained image editing model to complete terrain based on neighboring regions. Subsequently, an agent hierarchically plans region roles, district layouts, and asset relations. Deterministic geometry tools perform 3D asset placement, constraint checks, and local revisions, and validated additions are saved to the world state. During video generation, the system queries this world along a specified camera trajectory and renders depth sequences to guide the video model in generating frames. Across 120 tasks involving exploration and revisitation, the method demonstrates better structural consistency and camera compliance than the evaluated baselines.
Critical strengths
-
The work constructs a relatively complete world construction and generation system, and demonstrates stronger consistency than existing world models on the benchmark. In addition, the visualizations also demonstrate the advantages of this agent-guided world construction method, especially its consistency in long-horizon tasks.
-
Using an image generation model as a terrain generator, and then leveraging the capabilities of a VLM agent to construct a complete world, is a reasonable and novel idea. The experimental results also verify that this approach is feasible and effective. This coarse-to-fine world design is insightful.
Critical weaknesses
-
The generated worlds still appear coarse and insufficiently detailed. In addition, since the agent system selects buildings, vegetation, rivers, and other assets to place from a predefined asset catalog, the generation diversity is limited by the asset catalog, and the system cannot generate complex scenes in the way that video generation methods can. In addition, the time cost of using an agent to construct such scenes is likely to be relatively high.
-
During video rendering, only depth information is provided, which leads to the loss of texture and material details of the world scene. Specifically, when an object disappears from view for a relatively long period and then reappears, the consistency of its details cannot be guaranteed.
Questions for authors
What are the computational and time costs of agent-guided scene construction, and what is its success rate? How does the system handle failed construction cases, such as interpenetrating objects or poorly arranged layouts?
Additional Human Evaluation with Newly Randomly Sampled Cases
Thank you for raising the question of case selection and representativeness in our user study. To further assess the robustness of the findings to case selection, we randomly sampled 20 scene–trajectory cases from the full set used in the main experiments and evaluated WorldWeave alongside six baseline methods, covering 140 videos. More than 42 participants took part in this evaluation. Each video received three valid evaluations, yielding 420 video evaluations and 1,260 individual criterion scores.
We retained the original three criteria—camera consistency, object consistency, and structure/texture consistency—using the same 1–5 rating scale. We first averaged the ratings for each video and then averaged across the 20 cases with equal weight. The results are shown below.
With newly randomly sampled cases and a new round of human evaluation, WorldWeave again achieves the highest average score on all three criteria, consistent with the main findings of the original study. At the case level, WorldWeave achieves the highest or joint-highest score in 15/20 cases for camera consistency, 17/20 for object consistency, and 18/20 for structure/texture consistency. These results show that the observed advantages extend across most evaluated cases.
This additional experiment reproduces the main human-evaluation findings under renewed random case selection, providing further evidence of their robustness. We hope these new results address your concern about case selection and representativeness, and we would welcome your feedback on the additional evidence.