acceptodds
Withdrawn from ICLR 2027

Holi-Spatial 360: Generative Reconstruction from Sparse Panoramas for Spatial Rendering and Intelligence

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Spatial intelligence requires scalable 3D environments that are both photorealistic and semantically grounded, yet existing indoor datasets are dominated by room-scale captures and costly LiDAR acquisition. Online real-estate platforms offer abundant panoramas of diverse multi-room homes, but sparse viewpoints and texture-poor regions make reconstruction severely under-constrained. We introduce Holi-Spatial 360, a generative reconstruction and data curation framework for converting these panoramas into annotated multi-room 3D environments. Because reconstruction quality bounds subsequent generation, our Correspondence-Guided Geometric Optimization converts feed-forward reconstruction priors into reliable cross-view constraints and builds a coherent 3D Gaussian Splatting representation as 3D spatial memory. Adapted single-image and video priors then restore appearance and complete continuous trajectories while preserving observed content. The reconstructed scenes support arbitrary RGB-D rendering and an automated curation pipeline that combines end-to-end 3D grounding with VLM-based verification. Holi360 contains over 1K multi-room 3DGS scenes, more than 100K instance annotations, and 2M spatial question-answer pairs. We also introduce the Holi360 benchmark to evaluate the spatial understanding of VLMs in real-world, multi-room indoor environments. Experiments demonstrate superior reconstruction and curation quality under sparse panoramic inputs, while fine-tuning VLMs on Holi360 substantially improves spatial reasoning capabilities.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
reconstruction spatial intelligence scaling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_QTncOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper describes a framework for spatial rendering and intelligence taking as input sparse panoramic images of multiroom panoramic environment. Authors propose a dataset for training, and a technology for arbitrary RGB-D rendering and 3D object detection in terms of bounding boxes; moreover authors demonstrate the propose method in a 3DGS trajectory through an interpolated path. The assessment is performed by comparing modern baselines on various tasks as static benchmarks.

Critical strengths
  1. Complete framework for spatial reasoning and intelligence: the problem setup is challenging and worth of exploration.
  2. Manuscript clear, and well organized; the methodology is described properly, and the system is carefully designed, even if there is no siginificant theoretical contribution.
  3. Estensive assessment and evaluation: various baselines are compared, and the static results appear convincing for a variety of tasks.
Critical weaknesses
  1. My main concern is the lack of dynamic assessment: while the static results appear convincing for the spatial intelligence, the claim related to rendering is not supported by adequate results.

  2. For rendering, I would have expected comparisons against baselines targetting rendering like PanoSplatt3r$^1$ or PanoFloor$^2$

  3. I would have expected as supplementary dynamic videos also showcasing the transition between different rooms, in a multiroom environment,

  4. I would have expected results considering datasets like Matterport or Replica360, that use exactly the same setting (sparse panoramas), and provide ground truth for unseen poses.

  5. The results provided consider only very short paths with consistent overlaps between the views.

  6. Computational performance in inference are missing, specifically for rendering purposes.

[1] Ren, Jiahui, et al. "Panosplatt3r: Leveraging perspective pretraining for generalized unposed wide-baseline panorama reconstruction." 2025 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2025. [2] Pintore, G., Jashari, S., Agus, M., & Gobbetti, E. (2025, October). Panofloor: Reconstruction and immersive exploration of large multi-room scenes from a minimal set of registered panoramic images using denoised density maps. In 2025 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) (pp. 414-424). IEEE.

Questions for authors

Provide more evidence in a dynamic setting, specifically in the context of real-time virtual tour, otherwise the claim related to spatial rendering is completely not justified.

Reviewer_L1XZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Holi-Spatial 360, a pipeline that converts sparse real-estate panoramas into multi-room 3DGS scenes and spatial training data. The main ideas include correspondence anchors for sparse-view reconstruction, generative view densification, and automatic 3D grounding and spatial QA generation. The Holi360 dataset contains over 1K scenes, 100K instances, and 2M QA pairs. Experiments cover novel-view synthesis, geometry reconstruction, 3D grounding, and VLM fine-tuning.

Critical strengths
  1. The problem is important and scalable. Real-estate panoramas can provide many diverse multi-room scenes without requiring RGB-D or LiDAR capture. The geometry-first pipeline is also reasonable.

  2. The reconstruction results are consistently strong. Improvements are shown across all three test batches, and the geometry results and ablations support the main design choices.

Critical weaknesses
  1. The spatial reasoning evaluation is too closely related to the proposed data pipeline. Holi360 training data and benchmark seem to use similar reconstruction, annotation, and QA generation procedures. Therefore, the large improvement of Qwen3.5-4B may partly come from adapting to similar templates, vocabulary, and annotations, instead of better general spatial reasoning. Evaluation on an independent spatial benchmark would make the result more convincing.

  2. Dataset quality and reproducibility need more evidence. More than 100K instances are described as "verified," but the verification is automatic, and no human audit results are reported. The RealSee-based ground truth is also noisy. In addition, it is unclear which panoramas, reconstructed scenes, and annotations can actually be released due to licensing and privacy issues. Since the dataset is a major contribution, these issues are important.

Questions for authors
  1. Did the authors perform human evaluation on randomly sampled instances and QA pairs? If yes, what are the error rates?

  2. How does Holi360 fine-tuning perform on an independent spatial reasoning benchmark with different annotation and QA generation?

  3. Are all benchmark scenes fully disjoint from the training scenes? How many questions and categories are included?

  4. Which parts of the panoramas, 3DGS scenes, annotations, QA data, and trained models will be released, and under what license?

Reviewer_PcHBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Holi-Spatial 360, a data curation framework that converts sparse panoramic images into semantically annotated multi-room 3D environments. It leverages correspondence constraints from large reconstruction models (DepthAnything V3) to optimize an initial 3D Gaussian representation. Image generative model (DiFix) refines intermediate views, while video generative model (Wan-VACE) refines appearance and temporal consistency. A subsequent pipeline combines VLM, 3D object grounding, and geometric and semantic verification to produce instance annotations and spatial question-answer pairs. The resulting Holi-360 dataset contains more than 1K multi-room 3DGS scenes, over 100K annotated 3D instances, and 2M spatial question-answer pairs. Evaluations on held-out Holi-360 scenes and small subsets of RealSee 3D report improvements in novel-view synthesis, geometric reconstruction, and 3D grounding, while fine-tuning on Holi-360 improves spatial-reasoning performance within its evaluation protocol.

Critical strengths
  • The paper’s main strength is its systematic pipeline of geometric reconstruction, generative restoration, and automated annotation without human-in-the-loop labeling. The paper provides useful engineering evidence for sparse-panorama reconstruction.
Critical weaknesses
  • My main concern on this paper is the limited and insufficiently characterized evaluation, making it difficult to assess the robustness and representativeness of the reported improvements.

    • A pipeline designed to process more than 1K multi-room environments should demonstrate robustness across diverse panoramic inputs. However, Table 1 evaluates 30 held-out Holi-360 scenes, while Tables 2, 5, and 6 evaluate on 10, 5, and 5 RealSee3D scenes out of the official 10,000 unique indoor scenes (1,000 real scenes and 9,000 procedurally generated scenes, as seen in the public repository), respectively.

      The paper does not sufficiently explain how the evaluation scenes were selected or justify the small evaluation subsets. Consequently, it is difficult to determine whether the improvements are consistent across diverse environments or concentrated in a few examples. Given the limited quantitative evaluation, I would have expected to see qualitative videos comparing methods across all evaluated scenes in the supplementary, which would help contextualize the results.

    • The downstream evaluation demonstrates improvement within the Holi-360 evaluation protocol (Table 4) but does not establish transfer to other spatial-reasoning benchmarks or question templates. Because training and evaluation follow the same annotation protocol and question-template family, the experiment does not determine how much of the improvement would persist beyond this framework.

  • Some important details are not included in the main paper. For instance:

    • The paper states that geometric and semantic verification remove unreliable object instances and 3D bounding box predictions (Line 302). A deeper analysis of the sources of these errors (such as: 3DGS artifacts, out-of-distribution viewpoints, or errors and hallucinations from Qwen or Boxer) would help identify the limitations of the pipeline and its individual components and guide future improvements.

    • Holi-360 is a central contribution, but the manuscript does not provide sufficient distribution statistics beyond number of scenes, number of instances, and number of QA. Information, such as environment scales, room-count distributions, canonical-label diversity, and the number and density of annotated objects per scene. These statistics are important for assessing the dataset’s scope, diversity, and usefulness for future research.

Questions for authors
  • Please address the issues raised in Weaknesses.

  • Please clarify the experimental setup of Holi-360-recon, listed under “Reconstruction Baselines” in Table 1.

  • What computational resources and training time are required to fine-tune DiFix and Wan-VACE? Are all model parameters updated, only selected parameters updated, or LoRA adapters trained?

  • What are the optimization budget and end-to-end computational cost per reconstructed and annotated environment? This information would help assess the framework’s practicality for deployment.

  • Could LRMs that directly predict 3D Gaussians, such as PanoSplatt3r and NoPoSplat, be incorporated into this framework?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.