acceptodds
Withdrawn from ICLR 2027

Layout2Video: Persistent 3D Layout Control for Indoor Video Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Text- and image-conditioned video diffusion models produce visually compelling videos, but their inputs do not explicitly specify scene-wide 3D object arrangements. Maintaining these arrangements as the camera reveals new regions therefore remains challenging. We present Layout2Video, a framework for generating indoor videos from 3D semantic layouts and text prompts without task-specific training. The framework first renders view-specific semantic and geometric conditions from the layout. A pretrained multi-view diffusion model uses these conditions and an initial reference image to generate candidate views, from which sparse anchors are selected. A video diffusion model then synthesizes segments between consecutive anchors, and the segments are concatenated into a single video. All generative components remain frozen. We also construct a benchmark containing 555 room layouts and introduce an instance-level evaluation protocol for object presence, layout adherence, and appearance consistency. Extensive experiments demonstrate that our method effectively preserves the specified object arrangement and appearance throughout generated videos, particularly as previously unseen regions become visible.

Primary area:
generative models
Keywords:
video generation, 3D semantic layout, controllable generation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_8i3EOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Layout2Video generates indoor videos from semantic 3D bounding-box layouts and text using frozen pretrained models. It uses SpatialGen to generate layout-conditioned candidate views, selects four uniformly spaced anchors, and applies Wan first–last-frame generation between consecutive anchors. The paper also curates many room layouts from HSSD and evaluates 30 rooms using object presence, layout adherence, and appearance consistency metrics.

Critical strengths
  1. The framework addresses a useful control problem: maintaining specified indoor object arrangements as previously unseen regions become visible. Combining layout-conditioned views with endpoint-conditioned video generation provides a practical solution without task-specific training.

  2. The scene-aligned layout collection and detailed implementation documentation are useful contributions. The appendix specifies room selection, camera construction, generation settings, evaluation procedures, and bootstrap confidence intervals, making the experimental setup relatively transparent.

Critical weaknesses
  1. The evaluation structurally favors methods with more input anchors. Appendix D.2 permits tracking initialization only from input-anchor detections, and D.3 assigns zero even when an object appears in a generated frame but lacks an accepted anchor reference. Layout2Video has four anchors, whereas baselines have one or two, giving it more opportunities to initialize object tracks. AC additionally takes the maximum similarity over available anchor references, and camera alignment uses the method-specific anchor intervals. Thus, both the main comparison and anchor-count ablation change the evaluator alongside the generation method. The reported improvements cannot reliably separate better generation from these evaluation advantages.

  2. The methodological contribution is insufficiently isolated from the pretrained components. Layout-conditioned multi-view generation is supplied by SpatialGen, while video synthesis uses standard endpoint conditioning and concatenation. The main comparison changes backbone, layout access, and anchor count simultaneously. Moreover, the independent-versus-joint comparison uses different image generators, so it does not isolate joint generation. Stronger matched comparisons are needed to establish what the proposed integration contributes beyond the existing components.

Questions for authors
  1. Do the main ranking and anchor-count improvements persist with a common evaluation procedure that detects objects in generated frames, uses equal reference information, and avoids method-dependent anchor corrections? This directly affects my confidence in the quantitative conclusions.

  2. Can you provide matched comparisons using the same video backbone and anchor budget, including alternative layout-to-view pipelines? Please also evaluate temporal continuity at segment boundaries, since independently generated segments are concatenated without blending or explicit motion-continuity constraints.

Reviewer_Rkw7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Layout2Video generates indoor videos from semantic 3D layouts and text prompts. It uses frozen SpatialGen to generate candidate views. Four uniformly selected anchors then condition Wan2.1 to generate three video segments. The segments are concatenated into one video. The paper also introduces an HSSD-based benchmark and instance-level evaluation.

Critical strengths
  1. Layout-controlled indoor video generation is useful for virtual walkthroughs and interior design.
  2. The pipeline requires no additional task-specific training. Its modular design is straightforward.
Critical weaknesses
  1. Whats the main difference between layout2scene and (spatialgen/scenecrafter)? Spatialgen generates the multiview images with layout conditions and scenecrafter refines the coarse images with WAN.
  2. The comparisons do not isolate the proposed contribution. The main method uses Wan2.1 and four anchors. The Wan baselines use different backbones and fewer anchors. Table 2(b) also compares FLUX with SpatialGen, which does not isolate joint generation. Important comparisons are missing. These include SpatialGen followed by dense-view interpolation or 3DGS rendering, and a suitable VideoFrom3D configuration.
Questions for authors
  1. Do the rankings remain when all methods use the same detection-seeding schedule and the same number of appearance references? 2 How does the method compare with SpatialGen followed by dense-view interpolation or 3DGS rendering, under comparable inputs and computational budgets?
Reviewer_fUJWOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper generates camera flythrough videos of indoor scenes conditioned on a 3D semantic layout and a text prompt. It adopts a two-stage generation framework: (1) Layout-Guided Anchor Generation, which synthesizes four anchor frames conditioned on the input layout, and (2) Sparse-Anchor Video Generation, which generates intermediate video segments between adjacent anchor frames and stitches them together to produce the final video.

Critical strengths

The task of generating indoor videos from 3D semantic layouts, as addressed in this paper, is of practical relevance.

Critical weaknesses

I have serious concerns regarding the technical novelty and contributions of this work. The proposed method appears to be a straightforward integration of two existing pretrained models, without introducing a new algorithmic component or a technically meaningful improvement over closely related prior work.

  • The authors directly adopt the pretrained SpatialGen model to generate multi-view images conditioned on 3D semantic layouts and employ the pretrained Wan2.1-FLF2V model to generate intermediate frames between consecutive anchor images. Both stages rely on the existing capabilities of pretrained models, without introducing any new network architecture, training objective, or inference-time guidance mechanism. The final video is obtained by simply concatenating independently generated video segments.
  • This overall methodology has already been extensively explored in prior work: VideoFrom3D (SIGGRAPH Asia 2025), I2V3D (ICCV 2025), which both established the effectiveness of combining image diffusion for sparse keyframe generation with video diffusion for intermediate-frame synthesis. Furthermore, they introduce dedicated algorithmic designs to improve the quality and consistency of both stages. The authors should clearly explain what new algorithmic insights or technical mechanisms are introduced beyond those already established by the aforementioned works.
  • I have serious concerns regarding the inadequate baselines and unfair comparison settings. The baselines in Table 1, including ViewCrafter, SEVA, and Wan2.2-based models, do not receive the same 3D semantic layout information as the proposed method. More appropriate comparisons should include prior works that explicitly support 3D semantic layout guidance, such as: ControlRoom3D (CVPR 2024); SceneCraft (NeurIPS 2024); Layout2Scene (2025), VideoFrom3D and I2V3D.
  • The proposed anchor selection strategy is also extremely straightforward, which is simply uniform sampling and does not involve any adaptive selection or optimization based on scene geometry, semantic coverage, camera motion, visibility, or cross-view overlap. I do not see how this strategy constitutes a meaningful methodological contribution.
Questions for authors
  • What is the specific technical contribution of Layout2Video beyond directly combining the pretrained SpatialGen and Wan2.1-FLF2V models? How does the proposed method fundamentally differ from the established two-stage frameworks of VideoFrom3D and I2V3D?

  • Why are layout-conditioned scene generation methods, such as ControlRoom3D, SceneCraft, and Layout2Scene, not considered when constructing the baselines? Can the authors provide comparisons under more comparable structural conditioning?

  • Why is uniform sampling used for anchor selection? What technical insights does this strategy provide beyond the straightforward observation that additional keyframes improve interpolation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.