acceptodds
Withdrawn from ICLR 2027

StateFlow: Building, Evolving, and Accessing 3D World States for Previsualization

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Previsualization bridges ideas and production in film, games, architecture, and urban design by letting creators iteratively refine scenes, actions, cameras, and spatial-temporal dynamics. Existing generative methods instead jointly control these factors through one-shot image or video synthesis, providing limited support for local editing, long-horizon evolution, and scene revisitation. Revisiting and selectively editing interacting elements requires an explicit, persistent world state that separates scene structure, evolution, and observation. We present **StateFlow**, a state-centric framework built around an editable 3D world with three components. State Construction generates complementary front-view and bird's-eye references, matches their object evidence, and lifts the matched objects into a spatially grounded initial 3D state under geometric and physical priors. State Evolution converts user intent into selective state transitions while preserving unaffected content and accumulated world history. State Access closes the loop between camera proposals and rendered feedback to obtain feasible views of the evolving world. Experiments on complex dynamic scenes demonstrate spatial-temporal consistency across long sequences and multi-object interactions, together with high-quality video creation and game-like prototyping.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Previsualization, 3D World Generation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_mHAZOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

StateFlow is a framework built around an editable 3D world with three components: State Construction, State Evolution, and State Access. Current image and video generation methods all directly output the results from the prompt, which provides limited support for local editing, long-horizon evolution, and scene revisitation. StateFlow can support accurate control for generation. It splits the whole generation process into state generation and rendering, which makes sense.

Experiments show that StateFlow achieves high scores in scene reconstruction, object consistency, camera control, and background stability. The qualitative results are amazing.

Critical strengths

The paper proposes a complete framework for future visual generation, providing accurate control and convenience for creation. For the visualization, the qualitative results are amazing.

The workload involved in this work is substantial.

Critical weaknesses

I have to say that this paper seems more like an engineering report than a research paper. The core idea of StateFlow is to maintain an editable 3D scene representation, but such an idea has already been explored in many previous works and is relatively common in some industrial workflows.

The 3D state representation of each object in this paper only includes its geometry, position, orientation, and semantic attributes. Therefore, StateFlow cannot adequately represent physical attributes such as density. The representation mainly consists of static geometry together with time-varying poses and discrete asset replacements, rather than a full physical or 4D world model. As stated in the appendix, the next state for non-rigid motion is also generated as a new asset, while the original text-to-video model is likewise responsible for generating the transition. I therefore do not clearly understand why the proposed approach should be superior to direct text-to-video generation in this setting. Because the separate generation of different states' images is prior not to obey the physical rule, as it lacks temporal constraints.

Questions for authors
  1. Among the three components, I think State Evolution is the most distinctive part of StateFlow. The authors state in the related-work section that previous methods mainly focus on initial scene construction or fixed dynamic sequences. However, selectively editing a designated object while preserving other objects can already be implemented in conventional 3D software such as Blender. Although the ablation study in the paper shows that removing the state table leads to performance degradation, I still find this design relatively trivial. Therefore, the authors should clarify the specific technical contribution of State Evolution beyond standard object-level editing.
  2. The experiments should more directly evaluate the effectiveness of State Evolution in multi-round revision, as the paper claims the ability. In particular, they should demonstrate whether the world state remains stable after multiple rounds of revision, including the preservation of object identity, background content, spatial relationships, and previous edits.
  3. The complete pipeline consists of many components, including image generation, VLM-based matching, 3D asset generation, state updates, rendering, and video generation. These stages may introduce and accumulate errors. How does StateFlow detect, prevent, or correct such accumulated errors? The experiments should report the end-to-end success rate.
  4. A valuable baseline would be an agent-controlled workflow based on simple white-box or proxy geometry. This would help determine whether detailed 3D assets are actually necessary for layout and camera control. GEN3C should also be included as a relevant baseline, since it uses explicit 3D geometry for world-consistent video generation and camera control.
  5. The reported computation time is as high as 17 minutes for a representative scene, which also raises unavoidable concerns about the practical usability of the system. Is such a pipeline truly suitable for an interactive and creator-friendly workflow, given its high latency and potentially high cost?
  6. In Conflict-Aware Dual-View Initialization, the two generated views may contain inconsistent evidence, which could lead to ambiguous or inaccurate scene construction. I am also not convinced that a VLM can reliably match all object instances across the two views, especially when the scene contains multiple visually similar objects.
Reviewer_TPHqOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

StateFlow presents a framework for generative previsualization built around a persistent, editable 3D scene. It constructs object-level assets using front-view appearance references and bird’s-eye-view layouts, then refines their placement through consistency and physical checks. User instructions are translated into selective updates to object records, while camera trajectories are proposed and revised using rendered feedback. The resulting scene can be rendered directly or used to condition a video model for visual enhancement. Experiments evaluate scene construction, edit consistency, camera planning, and downstream video quality.

Critical strengths
  1. The framework addresses a practical need in previsualization: revising scene elements and camera choices while preserving previously accepted content. Its explicit object-level representation provides a useful basis for repeated editing and scene revisitation.

  2. The evaluation examines the construction, editing, and camera-planning stages separately, alongside downstream video results. The component ablations and user study provide evidence for the practical benefits of the integrated workflow, particularly layout control and camera feasibility.

Critical weaknesses
  1. The contribution beyond integrating existing components is not sufficiently established. Persistent object records, selective editing, and render-based camera refinement are useful design choices, but the paper does not clearly identify which introduces a substantive methodological advance. The full-regeneration and direct-video baselines demonstrate the benefits of explicit 3D authoring, but do not isolate the contribution of StateFlow’s specific design.

  2. Closer comparisons with existing scene-generation and editing systems are needed. Holodeck 2.0 [1] already combines VLM-guided asset generation, spatial layout constraints, and targeted object editing, while Imaginarium [2] uses generated visual references and scene-graph optimization for layout generation. These approaches appear more directly relevant to assessing the construction and editing contributions than the current baselines. Similarly, ChatCam [3] already uses rendered-image feedback for camera-anchor refinement, so the distinction of StateFlow’s trajectory-level feedback loop should be articulated and evaluated more explicitly.

[1] Bian, Zixuan, et al. "Holodeck 2.0: Vision-language-guided 3d world generation with editing." arXiv preprint arXiv:2508.05899 (2025). [2] Zhu, Xiaoming, et al. "Imaginarium: Vision-guided high-quality 3d scene layout generation." ACM Transactions on Graphics (TOG) 44.6 (2025): 1-24. [3] Liu, Xinhang, Yu-Wing Tai, and Chi-Keung Tang. "Chatcam: Empowering camera control through conversational ai." Advances in Neural Information Processing Systems 37 (2024): 54483-54506.

Questions for authors
  1. Beyond the integrated workflow, what do the authors consider the main methodological contribution of StateFlow? How does it differ from the construction and targeted-editing capabilities of closely related systems such as Holodeck 2.0 and Imaginarium?

  2. Could the authors provide more insight into the render-feedback camera-planning process? For example, which types of errors are most often corrected, and how much improvement is typically obtained through successive feedback rounds?

Reviewer_oYd6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents StateFlow, a generative system for previsualization that organizes scene construction, state evolution, and camera access around a persistent, editable, object-centric 3D world state. It reframes the challenge of long-term consistency in video world models as a problem of maintaining an explicit 3D world state. Specifically, StateFlow introduces a training-free workflow that uses advanced multimodal large language models and various generative models to construct an explicit 3D world, with subsequent editing, object motion, and camera movement grounded in this shared scene state. The framework aims to mitigate identity drift, unintended changes to unrelated content during local edits, and inconsistencies when revisiting scenes in direct video generation.

Critical strengths

As follows:

  1. The overall technical approach is sound, and the results demonstrate the value of StateFlow as a previsualization system.
  2. The work represents a substantial engineering effort, effectively integrating a range of advanced methods. Both the individual module designs and their integration are well conceived.
  3. The experimental evaluation is solid, providing evidence from multiple perspectives, including scene construction, video quality, camera motion planning, intent alignment, and user studies.
Critical weaknesses
  1. Gemini is used for both planning within the workflow and evaluating its outputs, which may introduce evaluation bias because the same model family is involved in both stages. Although Table 7 reports an evaluation using GPT-5.4, the evaluation criteria are overly broad. I recommend extending Table 8 to include evaluations by at least two LLMs from different model families, both distinct from Gemini, to better substantiate the method’s effectiveness.

  2. StateFlow offers practical engineering value, but its theoretical and technical advances over existing work are not sufficiently clear. For example, StoryBlender already uses persistent continuity memory and shared assets to organize 3D storyboards, and employs engine verification and rendering feedback to refine scenes and cameras. Its camera module performs closed-loop refinement through discrete commands. WorldDirector already uses an LLM to plan object and camera trajectories, which subsequently serve as conditioning signals for video generation. These works differ from StateFlow in their task settings or implementations, so their existence does not, by itself, negate the novelty of this paper. Nevertheless, the authors should at least discuss these related works to help readers understand the paper’s distinct contributions.

  3. The paper presents a complex engineering system, but several key implementation details are insufficiently described, potentially affecting reproducibility and the assessment of its technical contributions.

Questions for authors
  1. Several steps that rely on large models remain underspecified. It is unclear whether they are constrained by predefined rules or output formats, or simply rely on the models’ existing capabilities. For example, the implementations of cross-view object matching, conflict checking, and textual diagnosis and repair are not adequately explained.

  2. How exactly are candidate scenes assembled?

  3. How is the physical checker implemented, and how are responsibilities divided between this checker and the VLM during conflict checking?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.