acceptodds
Withdrawn from ICLR 2027

World-δ: Continually Integrating Observations for Incremental 3D Scene Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Existing 3D scene generation methods can synthesize visually compelling scenes from fixed inputs, yet struggle to continually incorporate new observations to coherently expand and revise an existing 3D world. Existing iterative approaches often fuse real observations and generated content into a single deterministic representation without explicitly modeling their provenance or uncertainty. Consequently, early generation errors can accumulate and become difficult to correct with subsequent observations. To address these limitations, we propose World-, an incremental 3D scene generation framework with uncertainty-aware persistent world modeling and delta-guided diffusion for continual scene expansion and correction. Our framework consists of three components: 1) Visual Evidence Encoding integrates geometric and semantic cues from incoming images into unified world tokens. 2) Observation-Calibrated World Modeling maintains a persistent latent world state that tracks geometry, appearance, provenance, observation confidence, and generative uncertainty, distinguishing observation-supported content from provisional completion hypotheses. It compares new evidence with the existing state to infer selective world deltas, enabling the revision of conflicting hypotheses while preserving well-supported content. 3) Incremental World Diffusion Generation derives update masks from these deltas and restricts diffusion to regions requiring generation or revision, while freezing unchanged regions. Only affected 3D chunks are decoded and assembled with the existing scene representation. Together, these components enable the world to expand with accumulating observations and allow uncertain completions to remain revisable, while reducing redundant generation and preserving scene consistency. Extensive experiments demonstrate the effectiveness of World- in incremental 3D scene generation, observation-driven correction, scene consistency, and update efficiency.

Primary area:
generative models
Keywords:
3D Scene Generation, World Models, Incremental Generation, Diffusion Models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_4Jy7OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes World-δ, a framework for incrementally expanding and correcting a generated 3D scene as new RGB observations arrive. Its persistent world state stores latents together with provenance, observation confidence, and generative uncertainty. New evidence is compared against this state, and each region is assigned one of four actions: KEEP, FUSE, REVISE, or EXTEND. Masked flow matching then regenerates only the regions marked for change, and only the affected chunks are decoded.

Critical strengths
  1. The problem is well motivated. Distinguishing observed content from generated content, so that wrong completions can be revised later, is a real issue in incremental scene generation.

  2. The idea of updating and decoding only the affected regions is reasonable and saves computation.

Critical weaknesses
  1. The qualitative results are poor. The generated scenes in Fig. 5, Fig. 6, and the appendix are blurry and overly smooth, with obvious artifacts. They are clearly behind current 3D scene generation methods, and the visuals do not match the strong numbers reported in the tables.

  2. The reported numbers are hard to trust. The flow model is very small (six transformer blocks) and is trained only on indoor Gaussian datasets. Yet it is reported to outperform Lyra on single-image generation, including on outdoor Tanks and Temples, and to outperform Splat-SLAM on observed-view rendering while also running faster. The appendix tables also follow unusually regular patterns. In Table A4, the failure-rate gap between “w/o Q” and “Full” is exactly 4.0 points in every non-zero-perturbation setting, and the gap between predicted and GT poses is exactly 2.0 points in every row. In Table A1, several timing entries scale by exact factors. Please explain these patterns and provide per-scene raw results.

  3. The method is heavily hand-designed, and its novelty is limited. The discrete KEEP/FUSE/REVISE/EXTEND actions, combined with several metadata fields and loss terms, make up a complex hand-crafted pipeline. Masked latent updating is essentially standard diffusion inpainting. CUT3R is cited as related work but is not compared against.

  4. The evaluation is narrow. All continual-update experiments are on indoor scenes, and the baselines are mostly SLAM methods, which is a mismatch for a generation task. The single-image comparison also uses older baselines.

Questions for authors
  1. Why are the gaps in Table A4 constant across all settings? Could you release per-scene raw results?

  2. How can such a small model trained only on indoor data outperform Lyra on Tanks and Temples?

Reviewer_WcGVOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper addresses the limitation that existing 3D scene generation methods cannot readily incorporate new observations or revise earlier completions. The authors propose World-δ, an incremental generation framework that maintains a persistent world state tracking provenance, observation confidence, and generative uncertainty; infers selective world deltas through four actions (KEEP / FUSE / REVISE / EXTEND); and performs localized regeneration via masked flow matching. A chunk-level incremental decoder is trained to re-decode only affected chunks while freezing the rest, balancing scene consistency with update efficiency. Experiments on single-image generation and continual RGB updates, together with component-wise ablations, report improved rendering quality, correction, and update efficiency.

Critical strengths

The dependency-aware chunk-level decoding—with a two-cell halo, owner write permissions, and hard freezing—is a good engineering design. Decoding is restricted to a small fraction of chunks (8 vs. 32). Compared with full decoding, the two-cell halo achieves roughly 8× decoding speedup, 75% chunk reduction, and 31.7% memory reduction.

Critical weaknesses
  1. Several concurrent, highly relevant works are not cited:
  • OracleGS (Topaloğlu et al., 2025) already distinguishes multi-view-supported content from hallucinated regions using an MVS-derived 3D uncertainty signal and uses it to gate generated content -Terra (Huang et al., 2025) is a native-3D world model with progressive generation, evaluated on ScanNet v2—a direct generative competitor.
  • Gaussian Scenes (Paul et al., 2024) introduces a 3DGS confidence measure and a Gaussian-SLAM-style progressive integration with diffusion refinement. WorldExplorer (2025), BlockFusion (2024), CL-Splats (2025) are also related literatures.
  1. The continual-update comparison lacks generative baselines, while the single-image comparison does not. Table 1 appropriately compares against diffusion-based single-image generators (LucidDreamer, ZeroNVS, Lyra) and a feed-forward predictor (InfiniSplat). In contrast, Table 2 evaluates the continual-update task only against reconstruction-based SLAM systems (MonoGS, Splat-SLAM, S3PO-GS, OF3GS), which do not hallucinate unobserved regions. Since the paper's central claim is continual generation with revision, at least one generative or continual-learning baseline (e.g., Terra, CL-Splats) is needed, and the evaluation region—observed support vs. full scene—must be stated explicitly.
Questions for authors

The decoder in this paper seems to be trained in-house as part of the codec, but the "latent → Gaussians" family has established designs (e.g., L3DG's sparse-conv VQ-VAE; what is the difference in design between the repurposed Stable Diffusion decoder in Prometheus/Director3D; GaussianAnything) and the decoder in this paper?

Reviewer_qF2MOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper works on incremental 3D scene generation from sequential RGB observations. It first generates a Gaussian scene from the initial images, then expands and corrects the scene as new images become available. The main idea is to distinguish content supported by observations from generated guesses, so that later observations can correct earlier mistakes while preserving reliable parts of the scene. It combines geometric features from LingBot-Map and semantic features from DINOv2, and maintains a persistent world state that tracks provenance, observation confidence, and generative uncertainty. By comparing new observations with the existing state, it decides which regions to keep, fuse, revise, or extend. These decisions guide mask latent diffusion, and then decodes only the affected Gaussian chunks. They evaluate both single-image generation and continual scene updates against generation and reconstruction baselines. They also ablate the state information, update masks, and incremental decoding to study their effects on rendering quality, error correction, preservation of previous content, and update cost.

Critical strengths
  1. The incremental decoding gives a great speedup while preserving the quality of full decoding. It reuses unaffected Gaussian chunks and include the neighboring latent cells required by the decoder. On the sampled ScanNet scenes in Table 5, this reduces total update time from 13.2 to 4.8 seconds.
Critical weaknesses
  1. the problem formulation is confusing. Despite the title on 3d scene generation, little evidence of the deliverable showcases it is able to generate 3d scene beyond what is already observed, e.g., figures, supplementary video. it is hard to evaluate to the contribution from 3d scene generation standpoint. If i were to review from the 3d reconstruciton contribution, then it is unclear what additional inputs this paper provides beyond LingBot-Map. it is hard to justify their delta flow-matching update helping on increase the fidelity.

  2. The key comparison figures (Figure 5) are not from the same perspective, it is really hard to evaluate their pros and cons.

  3. It is hard to justify the additional contribution of this paper given SoTA performance is stronger. It is confusing to me the advantange of incremental generation over one-shot in this paper still.

Questions for authors

See critical weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.