acceptodds
Under review as a conference paper at ICLR 2027

Pano123: Single-Panorama to Explorable 3D Scene Generation

Abstract

Generating an explorable 3D scene from a single equirectangular panorama remains underconstrained because camera translation reveals surfaces occluded in the input. These disoccluded regions contain no observed geometry or appearance, making their completion the key bottleneck. We present Pano123, a feed-forward framework that predicts a complete 3D Gaussian Splatting scene from a single panorama in one forward pass, without per-scene optimization. Our key insight is that disocclusion should be treated as a sparse, geometrically structured completion problem rather than as dense scene hallucination. Disoccluded regions typically occupy only a small portion of a target view and arise predictably near occlusion boundaries. They can therefore be localized first and completed only where necessary. Accordingly, Pano123 comprises three stages: a sphere-aware Gaussian initializer that accounts for the non-uniform sampling of equirectangular projection; an annotation-free query generator trained with cross-view reprojection pseudo-labels to localize disocclusions and sample candidate points behind visible surfaces; and a context-grounded decoder that synthesizes new Gaussians from neighboring attribute priors and bounded residuals, while preserving visible Gaussians. Experiments on indoor benchmarks show that we outperform prior feed-forward methods, with the largest gains under large camera translations.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3DGS, Single-View Reconstruction, Panorama

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Ab4kOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The work proposes Pano123, which is a 3D reconstruction framework from a single panorama input. The approach is proposed in a feed-forward setting. The main observation assumes that only the disoccluded regions have to be filled in such a setup, and those are typically sparse. To this end, the authors propose a 3-stage approach. Firstly, an initial Gaussian Splats estimation is performed with an off-the-shelf depth prediction as the initialisation, with an additional predicted refinement step. The next step predicts a set of disocclusion candidates. Finally, a set of candidates behind the occlusion surface is updated with a learnable decoder.

Critical strengths

I believe the paper's motivation and main design choice are compelling. Predicting only the candidate disoccluded Gaussians, while preserving the visible ones intact, is a good, interesting idea. It is shown in the paper that the approach works as described, and quantitative results show substantial improvements over the compared baselines. Similarly, qualitative samples show convincing results with the scenes fully completed, as opposed to producing holes (as is shown in the baselines). The parametrisation in spherical coordinates is also a good design choice, which is well motivated in the paper (namely, preventing the stretched splats). Finally, it is sensible to restrict stage 3 attention to nearest neighbours, which should help with the prediction speed.

Critical weaknesses

While the results are convincing, most of the very scarce visual samples present rather flat regions (e.g. walls). Without a generative prior, the method can struggle with severe object occlusions; the results could get way worse if the disoccluded region cannot be smoothly interpolated (this also ties to initialising queries with the average of neighbouring Gaussians).

The evaluation setup is not very extensive. The only ablation is Stage I vs Full Model. While this is a good ablation, one could ablate many more parameters. Starting with the dependence on the initial depth prediction quality, stage I refiner (with and w/o), stage II trained vs a simple heuristic, extent of query candidates, etc. Similarly, the authors do not provide many baseline comparisons; one such could be Stage I + inpainting. Further, the authors cite Scene4U, which even though have not released the code yet, could be compared by reusing the same dataset and protocol (and notably, the method solves a very similar problem). Similarly, the cited LayerPano3D could possibly be adapted to the task. There should be nothing preventing testing PanSplat on a single input view. Finally, CylinderSplat explicitly offers a single panorama setup. Conversely, I am not convinced about PanoSplatt3R as a baseline with adaptation to multi-view by input view duplication.

One important work, not included, is PERF [1], which is a single-panorama to a radiance field approach. Further, I would also refer to PanoGRF [2] as an important prior work.

[1] Guangcong Wang, Peng Wang, Zhaoxi Chen, Wenping Wang, Chen Change Loy, Ziwei Liu, "PERF: Panoramic Neural Radiance Field from a Single Panorama", TPAMI, 2024. [2] Zheng Chen, Yan-Pei Cao, Yuan-Chen Guo, Chen Wang, Ying Shan, Song-Hai Zhang, "PanoGRF: Generalizable Spherical Radiance Fields for Wide-baseline Panoramas", NeurIPS, 2023.

Questions for authors
  • Have the authors considered using a baseline of Stage I plus an off-the-shelf inpainting method (in low-opacity or missing regions)? It could help to assess the performance of Stage III.
  • It would be interesting to see results of splatting only the candidate stage 3 Gaussians, without refinement.
  • Why do "near" and "far" distances overlap? There could be samples in "near" that are farther than some in "far".
  • It would be good to prove that the cosine scale modulation and spherical residual parametrisation are important by ablating them.
  • Have the authors tried using a depth gradient in place of the learned disocclusion detector?
  • What is the method's sensitivity to the number of candidate depths?
Reviewer_tqYEOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes Pano123, a three stage 3D Gaussian scene generator from a single equirectangular panorama. In the first stage, Pano123 back-projects panoramic image features to a sphere with depth, considering area distortions caused by an equirectangular projection. Then it refines the Guassian using a lightweight head. In stage 2, Pano123 selects occluded positions to be filled with learnable detection on a sphere and ray marching. Lastly, in stage 3, the selected positions are filled by concatenating new Gaussians that are inferred by self-attention in the occlued regions and cross-attention with the nearest visible tokens. The results from Pano123 outperformed previous approaches in novel view synthesis. The paper also validates stage 2 and 3 through an ablation study.

Critical strengths
  1. Derivation of initial Gaussians in stage 1 including spherical back-projection and latitude-aware scale correction is mathematically well-formulated.

  2. Pano123 produced the best novel view synthesis performance compared to the previous methods. This advantage is from a novel and reasonable approach to select candidate positions and generate occluded Gaussians.

  3. The insight of the paper, utilizing sparsity and locality of occluded regions, is informative, sound, and well-described.

Critical weaknesses
  1. Stage 1 contributes to the input-view generation. To evaluate the performance of stage 1, stage1-only results should be compared with baseline approaches in the input-view instead of a novel-view. Otherwise, initializing the scene with previous approaches instead of stage 1 can be a better choice.

  2. In the occluded regions, artifacts (e.g. in the photo frame of figure 3 far distance, the wall in figure 4) can be found. As the paper is providing rendered images in novel-view, it is unclear how those artifacts will affect actual 3D. Furthermore, as the output format of the method is 3D Gaussian, how the results look in 3D space is important. However, the paper doesn't provide any example in actual 3D space.

Questions for authors
  1. The term "disocclusion" is a bit confusing. In the perspective of single camera generation, which the paper is addressing, isn't it precise to say the unseen region as "occluded" region instead of disoccluded region since it will never appear during generation? It would be better if this question is also clarified in the paper.

  2. Related to the weakness 1, could authors provide qualitative comparison in the input-view as well in the paper, compared with baseline approaches?

Reviewer_mLG6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Pano123 generates an explorable 3D Gaussian scene from a single panorama in one forward pass. Panoramic coverage leaves occluded surfaces unobserved, while projection distortion and independent perspective crops introduce artifacts. Its localize-then-complete formulation targets sparse hidden regions rather than generating content throughout the scene. A sphere-aware initializer corrects latitude-dependent sampling, and a query generator learns completion locations from cross-view pseudo-labels. A context-grounded decoder predicts bounded residuals around neighboring Gaussian attributes, adding hidden content without modifying visible Gaussians. Under large viewpoint changes, PSNR exceeds PanoSplatt3R by 4.99 dB on HM3D and 2.51 dB on Replica, although nearby-view advantages are less consistent.

Critical strengths
  1. Strong problem formulation. Selective disocclusion completion directly targets hidden surfaces revealed by camera translation, providing a clear rationale for concentrating generation capacity in sparse regions.
  2. Coherent architectural decomposition. Sphere-aware initialization addresses panoramic distortion, learned queries identify completion locations, and context-grounded residual decoding constrains hidden-surface synthesis.
Critical weaknesses
  1. Insufficient geometric justification. Rotation-only pseudo-label transfer ignores translational parallax, while fixed rectangular ERP cells do not ensure equal solid angles, weakening the correspondence between geometric claims and implementation.
  2. Incomplete component validation. Comparing initialization alone with the full pipeline does not independently establish the necessity of latitude correction, learned query selection, neighboring attribute priors, or bounded residual prediction.
  3. Limited comparative coverage. Evaluation includes only two baselines and duplicated inputs for the multi-view PanoSplatt3R, while missing runtime measurements, repeated-run uncertainty, and geometry-specific metrics restrict conclusions about efficiency and reconstruction quality.
Questions for authors

Please seek the weakness.

Comments on the paper

1 comment
Area_Chair_MUyeOpenReview ↗

Please engage with the authors.

Dear reviewers,

The authors have submitted detailed responses to your reviews.

Could you please review their feedback, engage with the authors if any points remain unclear, and update your evaluation and score if their response has addressed your concerns?

Best regards, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.