acceptodds
Under review as a conference paper at ICLR 2027

WorldExpand: Building Coherent 3D Worlds via Sparse Joint Panoramic Expansion

Abstract

Constructing complete 3D worlds from a single panorama is challenging because large portions of the scene remain unobserved. Existing panoramic-video methods expand spatial coverage through dense sequential generation, but neighboring frames contain substantial redundancy, long trajectories accumulate errors, and independently generated directions may be inconsistent. We present WorldExpand, which formulates single-panorama world expansion as the generation of sparse, complementary, high-resolution panoramic observations. This representation concentrates the generation budget on informative scene regions and preserves the fine-grained details required for 3D construction. WorldExpand addresses two key challenges: where to observe and how to synthesize. Our Coverage-Guided View Selection identifies informative target locations using scene-adaptive quadrants and warp-based coverage scores, while our Joint Panoramic Diffusion uses Gaussian-splatting-based geometric warps to jointly synthesize four pose-grounded, cross-view-consistent panoramas. A diagnostic-guided global–local attention schedule retains global attention only in sensitive layers, reducing the training and inference costs of high-resolution joint generation. The resulting posed panoramas can be directly fed into an existing reconstruction pipeline to build an explicit 3D representation. Experiments show that WorldExpand recovers content behind occluders and around distant corners, improves both novel-view synthesis and downstream 3D reconstruction metrics across various datasets, and generalizes robustly to diverse in-the-wild scenes. Compared with the state-of-the-art panoramic-video method, WorldExpand achieves up to an speed-up with higher-resolution observations and stronger cross-view consistency.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D World Generation, Panoramic Novel View Synthesis

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_uEv9OpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces WorldExpand, a multi-view panoramic image generation and 3D fusion framework, aiming to synthesize an explorable panoramic world. The authors propose Coverage-Guided View Selection and Joint Panoramic Diffusion to determine which panoramas are to be generated, and finally reconstruct them into a world through depth alignment.

Critical strengths

The writing is easy to follow and the supplementary video is high-resolution.

Critical weaknesses
  1. The motivation of this paper is problematic. The authors claim that dense panoramic video generation is redundant; however, this so-called "redundancy" is actually necessary to simultaneously capture spatio-temporal continuity. What the authors describe as sparse prediction, in fact, discards critical dynamic information and focuses only on spatial aspects, which is essentially a step backward rather than an improvement. For instance, how does this method model dynamic objects, such as moving vehicles? As shown in a scene in the supplementary material, the water flow appears completely static, which is a significant limitation.

  2. Following up on the previous point, if the authors' intention is to explore static 3D world generation rather than panoramic video generation, then WonderJourney, WonderWorld, and LayerPano3D appear to be more scalable and reasonable solutions. Unfortunately, the authors overlook these works and fail to cite them.

  3. The experiments are insufficient. Why are comparisons made only with GenEx and Matrix-3D? Why are WorldExplorer and OmniRoam not included?

  4. Joint Pano is not particularly novel; many prior works, such as VideoPanda, ViewPoint, and CubeComposer, share similar ideas. The authors merely extend this idea from a single panorama to multiple panoramas.

[1] Wonderjourney: Going from anywhere to everywhere. CVPR 2024.

[2] Wonderworld: Interactive 3d scene generation from a single image. CVPR 2025.

[3] LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generation. SIGGRAPH 2025.

[4] WorldExplorer: Towards generating fully navigable 3d scenes. SIGGRAPH Asia 2025.

[5] OmniRoam: World wandering via long-horizon panoramic video generation. SIGGRAPH 2026.

[6] VideoPanda: Panoramic Video Diffusion with Multi-view Attention. arxiv 2504.11389.

[7] ViewPoint: Panoramic video generation with pretrained diffusion models. NeurIPS 2025.

[8] CubeComposer: Spatio-Temporal Autoregressive 4K 360 Video Generation from Perspective Video. CVPR 2026.

Questions for authors
  1. What is the range of scenes to which N=4 applies (e.g., geometric complexity, number of rooms, topological connectivity)?
  2. Why are HY-World 2.0, WorldExplorer, and OmniRoam not included for comparison?
1 comment

Thank you for your response, I raise my rating to 2. And, can you provide some video results on the comparison?

Reviewer_FdAvOpenReview ↗
Rating
4(+1)
Confidence
4(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

WorldExpand builds an expanded 3D scene from a single panorama by jointly generating four novel-view panoramas. It selects target camera positions using source visibility and scene boundaries, and conditions generation on Gaussian-splat renderings of the source point cloud. A diagnostic-guided global–local attention schedule reduces the cost of joint high-resolution generation. The input and generated panoramas are then reconstructed into a Gaussian scene using an existing pipeline. Experiments evaluate novel-view quality, camera alignment, reconstruction quality, and the effects of view selection and attention design.

Critical strengths

1. Generating a small set of complementary panoramas is a useful way to organize observations for 3D reconstruction. This is the aspect I find most interesting. A panorama covers all viewing directions but still misses surfaces hidden from its observation center. Selecting several displaced viewpoints and generating them jointly addresses this limitation while avoiding the need to synthesize many neighboring video frames. The view-selection and joint-generation components fit this objective well, making the overall design useful despite relying on established building blocks.

2. The experiments support the quality of the generated observations and their usefulness for reconstruction. Table 1 shows improvements over the compared methods in novel-view PSNR and reconstruction F-score on both datasets. The joint-generation ablation in Table 9 also improves PSNR from 18.27 to 20.16 dB, supporting the decision to generate the target panoramas together.

3. The global–local attention schedule provides a practical efficiency trade-off. Replacing 30 of the 60 global layers with per-view attention gives a 1.79× inference-step speedup in Table 13, with a best-PSNR gap of about 0.33 dB relative to full-global attention. The layer-sensitivity analysis and checkpoint comparison provide useful evidence for retaining cross-view communication selectively.

Critical weaknesses

1. The technical novelty is modest, and the benefit of diagnostic layer selection could be clearer. Joint multi-view generation is established, including in panoramic settings such as DiffPano, and the reconstruction stage follows an existing pipeline. I see the contribution mainly in adapting these components for sparse panoramic expansion and improving their efficiency. For the attention design, Appendix K compares the least-sensitive and most-sensitive layer sets, but this leaves open whether the diagnostic is substantially better than a simple alternating or uniformly spaced allocation with the same number of global layers.

2. The efficiency results do not yet cover the complete reconstruction workflow. Table 3 reports approximately 45 minutes for four-panorama denoising on one H20 and explicitly states that end-to-end time was not profiled. Depth prediction, alignment, and Gaussian optimization add further cost. The reported 8.6× speedup is useful as a generation-stage comparison, but does not establish the total time saved in constructing a 3D scene. A timing breakdown would make the practical benefit easier to assess.

3. Robustness to estimated source depth is supported mainly by qualitative examples. Appendix A.1 states that both quantitative benchmarks use ground-truth source depth, whereas practical inputs use predicted depth. This affects viewpoint selection, geometric warps, and reconstruction alignment. The warp augmentation in Appendix G is a reasonable response, but Table 8 measures its cost under clean depth rather than its intended benefit under noisy geometry. It remains unclear how much quality is retained when the source depth is estimated.

4. A comparison with a more recent panoramic generation method would be useful. OmniRoam (SIGGRAPH 2026) also generates camera-controlled panoramic videos and demonstrates 3D reconstruction from the generated observations. Although it focuses on long-horizon wandering, a comparison under compatible settings would help assess the practical advantage of sparse joint panoramic expansion over a recent video-based approach.

Questions for authors
  1. How does diagnostic layer selection compare with an alternating or uniformly spaced global–local schedule at the same layer budget?
  2. What is the total time to produce the final Gaussian scene, including depth prediction, alignment, and Gaussian optimization, under the setting in Table 3?
  3. On the existing test scenes, how do generation and reconstruction quality change when ground-truth source depth is replaced with predicted depth? How much does warp augmentation help in this setting?
  4. I am curious how the image-based generator distinguishes tokens from different views during global attention. Is this handled through positional or view-specific encoding, or is an explicit distinction unnecessary in your setup?
1 comment

Thank you for the detailed response and additional experiments. These address my concerns and make the benefits of the design clearer. Thus, I have raised my rating from 3 to 4. Please include the new results and implementation details in the revised paper, while keeping the distinction between measured runtime and estimated end-to-end speedups clear.

Reviewer_k92qOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

WorldExpand addresses 3D world expansion from a single panorama. Instead of dense, independently generated panoramic videos (GenEx, Matrix-3D), it generates a sparse set of four high-resolution (1024×2048) panoramas at displaced locations and feeds them, together with the source, into the existing HY-World 2.0 reconstruction pipeline (WorldMirror 2.0 + 3DGS).

The method has two components. (1) Coverage-Guided View Selection samples candidate cameras on rings around the source, scores each, rotates a four-quadrant partition to align with nearby walls, and keeps the best candidate per quadrant. (2) Joint Panoramic Diffusion fine-tunes Qwen-Image-Edit-2509 (20B DiT, LoRA) to denoise all four targets in one token sequence, conditioned on text, the source panorama, and per-view warps with disocclusion masks.

On 64 anchors each from held-out UE scenes and the RealSee3D synthetic subset, WorldExpand beats both baselines, and its denoising stage is much faster than Matrix-3D at 720p, validating the effectiveness of the proposed framework.

Critical strengths
  1. Replacing dense per-direction video with four jointly generated panoramas is a simple idea that matches what downstream reconstruction actually needs, and the evidence is broad. WorldExpand is best on every NVS metric for every target view on both benchmarks (Table 14). The gains carry over to geometry, most clearly in pose AUC@3° and F@5cm. Denoising four directions at 2K is also much faster than Matrix-3D running at lower resolution.

  2. The same-backbone single-view variant isolates the benefit of joint generation, serving as strong evidence for the paper's consistency claim. The attention schedule is supported by three pieces of evidence: a per-layer diagnostic, a cross-checkpoint stability check, and an inverse-selection control, which shows that the choice of layers to localize matters.

Critical weaknesses

W1

The main comparison does not isolate the representation. The central claim is that sparse joint panoramas are a better intermediate than dense video, but WorldExpand also differs from the baselines in:

(a) Backbone: 20B vs. 14B (Matrix-3D). (b) Output resolution: 2K vs. 480p/720p. How lower-resolution baseline frames are scored against ground truth is not stated. (c) Training data: the UE test set comes from the authors' own pipeline. Although all methods have access to UE-rendered training data, only WorldExpand is trained on the authors' specific scene collection and data-generation pipeline, so the UE benchmark remains in-domain only for WorldExpand.

On zero-shot RealSee3D, the margin narrows. The only same-backbone baseline is the single-view ablation, and no video- or perspective-view-based expansion pipeline from Sec. 2 (e.g., HY-World 2.0, WorldExplorer) is compared.

W2

Each run reports its own "best checkpoint" (Fig. 15, Table 8); Table 1's 20.16 dB equals the best global–local checkpoint in Fig. 15, while App. C mentions only a 30-scene "validation" split.

If checkpoints were selected on the test anchors, results are inflated. At most checkpoints in Fig. 15, the global–local curve is ~0.5–1 dB below full-global. And all quantitative results use ground-truth source depth, so the deployment setting (DepthMaster depth) and the benefit of warp augmentation are shown only qualitatively, although the synthetic benchmarks would allow measuring both.

Questions for authors
  1. On which split were the "best checkpoints" in Table 1, Fig. 15, Table 8 and Fig. 17 selected, and are the 64 UE test anchors disjoint from it? Please also report Table 1 at a fixed final checkpoint.
  2. Is Table 8's full-global model with augmentation (20.39 dB) the same as Fig. 15's full-global model (20.49 dB)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.