acceptodds
Under review as a conference paper at ICLR 2027

Beyond the Frustum: Single-View Amodal 3D Scene Reconstruction

Abstract

Single-image 3D methods either reconstruct the visible surface or generate a plausible room, leaving a gap for metric completion tied to the input. We study evidence-grounded amodal occupancy completion: recover the scan-observed geometry of every object supported by a single RGB(-D) view, including occluded and out-of-frustum parts, while predicting no unsupported objects. We present \method, a feed-forward framework that back-projects depth into a sparse, DINO-decorated 3D seed and completes it with a 3D U-Net inside a predicted scene-fitted box. A scalar coverage channel, aggressive seed dropout, and a completion-masked BCE–Dice loss are designed to discourage trivial seed copying and to focus learning on unseen voxels. On 3RScan, \method substantially outperforms reconstructive, generative, and indoor-occupancy baselines on unseen geometry; explicit in-frustum/out-of-frustum evaluation and a volume-matched dilation control confirm that the gain reflects completion rather than surface inflation. The model also transfers zero-shot to ScanNet. Code and trained models will be made public.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D scene reconstruction, amodal completion, scene completion, single-view 3D

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_9PpMOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper aims at the problem of reconstruction of partial visible objects. In contrast to existing work that mainly deals with partial visibility through occlusion, in this work the goal is to handle objects that are partially visible due to the frustum. This of course shares lot's of properties as it can be seen as a partial occlusion where the occlusion boundary is known. The core trick in this paper is to only complete the objects partially visible in the scene, and not hallucinate beyond. The pipeline is straight forward, an image with depth is fed, Dino features are extracted, a bounding box location is predicted and then a unet performs completion. Training is performed fully supervised and a low res voxelbased representation is chosen

Critical strengths

I like the posed task and the approach to only complete the parts that are partially visible in the scene. There is extensive experimental evaluation.

Critical weaknesses
  • VisHall3D pointed out as closest in related work and seems very similar. the contribution is phrased as collapsing this into a single stage - it however is not compared or ablated against this model
  • The paper is hard to read, and this already starts with the abstract which introduces quite some jargon which is not common when the paper was not read (e.g. DINO-decorated 3D seed or scene-fitted box). When hybrid is used the first time in line 30 it is unclear what are the components? the related work section is more a list than supported with a story
  • the method seems strongly limited by the training data which is 3D. several works have shown that solving parts of the problem in 2D might ease it through the massive better availability of training data. I'm surprised that the image is not outpainted before reconstruction which would serve as a good baseline.
  • the dataset chosen for zeroshot generalization is collected by the same lab as the training data and has therefore strong hardware and data modality overlap
  • the core contribution is stated as defining a task and one particular solution. When a new task is proposed I would hope to see multiple approaches or variants

minor comments:

  • Introduction is missing citations (e.g. "by methods that synthesize an entire room from one image")
  • Figure 3: it would help a lot if the visible parts could be highlighted somehow and if the poses of the images would be closer (especially first and last one are 90 degrees of, which took me a while to parse).
  • the name of the method seed2sene is misleading, as the core trick from my perspective is not to actually complete the scene, but only the partiallyvisible parts
  • figures are pixelgraphics instead of vector graphics
Questions for authors

It is unclear to me, how the crops of the views during training and during testing where performed (sample at random is the only description I found). I would like to hear in the rebuttal what the distribution of the crops in both phases were (perhaps I just missed that in the paper).

Reviewer_i19nOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents Seed2Scene, a feed-forward framework for single-view amodal 3D scene reconstruction. The method lifts RGB-D observations into a feature-decorated 3D seed, predicts a scene-aligned bounding box, and employs a 3D U-Net to complete unobserved geometry. Experiments on 3RScan and ScanNet demonstrate improvements over existing reconstruction and generation baselines.

Critical strengths

The method achieves substantial improvements on unseen-region occupancy completion and demonstrates reasonable cross-dataset generalization.

Critical weaknesses
  • The proposed framework largely combines well-established components rather than introducing fundamentally new 3D learning mechanisms. Its core pipeline, consisting of depth lifting, a 3D CNN encoder-decoder, and voxel occupancy prediction, closely follows approaches explored in SSCNet (CVPR 2017), ScanComplete (CVPR 2018), and SG-NN (CVPR 2020). Given recent advances in 3D generative priors and representations, the authors should better justify their reliance on such a conventional architecture and clarify its technical advantages over modern alternatives, such as Points-to-3D (CVPR 2026).
  • The method overview figures are entirely AI-generated, without further refinement or redrawing into a vector-based format. This raises concerns about the professionalism and overall presentation quality of the paper.
  • The coverage channel and completion mask yield only marginal improvements, while the independent contribution of seed dropout is not evaluated. A critical missing baseline is a standard 3D U-Net or SSCNet-style architecture trained with identical DINO features, bounding boxes, datasets, and supervision. Without such controlled comparisons, it remains unclear whether the reported gains stem from genuine methodological innovations or simply from the training and evaluation setup.
Questions for authors
  • What are the fundamental technical differences between Seed2Scene and earlier volumetric completion methods such as SSCNet, ScanComplete, and SG-NN, beyond the task formulation and combination of existing modules?
  • Given recent advances in 3D generative priors and representations (e.g., Points-to-3D), what motivates the use of a conventional dense 3D U-Net? What advantages does this design offer over modern alternatives?
  • How would a standard 3D U-Net or SSCNet-style model perform when trained with the same DINO features, bounding boxes, datasets, and supervision? Such a comparison seems essential to establish whether the improvements arise from the proposed designs rather than the training setup.
  • The coverage channel and completion mask provide relatively marginal improvements. Could the authors further justify their necessity and provide an independent ablation of seed dropout?
Reviewer_NQNyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Seed2Scene for completing the 3D occupancy of objects from a single view. It uses an RGB image and a metric depth map to create a sparse 3D seed, predicts a bounding box, and uses a 3D U-Net to fill in the unseen parts. The authors report improved unseen-voxel F-scores on 3RScan and zero-shot transfer to ScanNet.

Critical strengths

The paper addresses an interesting and potentially important problem: recovering the full geometry of objects that are only partially visible in a single view, including their occluded and out-of-frustum parts, while avoiding the generation of unsupported objects. This provides a useful middle ground between reconstructing only the visible surface and generating an unconstrained plausible scene.

The task is clearly defined: the target includes the scan-observed geometry of objects with at least one supporting pixel in the input. The experiments also include useful ablations and controls, such as the volume-matched seed dilation control, to examine whether the gains reflect completion rather than simple expansion.

Critical weaknesses

The technical contribution seems limited. The method mainly combines depth back-projection, a bounding-box regression network, and a 3D U-Net for occupancy completion. The coverage channel, seed dropout, and completion-focused loss are reasonable design choices, but I do not see a major technical contribution beyond this combination. The main contribution seems to be the task formulation. However, I am not sure how clearly this task differs from existing amodal reconstruction or scene-completion tasks.

I am also not sure how much of the target can be inferred from a single view. The target includes the complete scanned geometry of every object that has at least one visible pixel. This can include large parts of an object outside the camera frustum, as well as floors and walls. It is unclear how reliably a model can recover this geometry from one view, especially when only a small part of an object is visible. Since the geometry behind occluders is not observed in the input, the model can only infer it from learned priors. The paper does not show whether the predictions recover the actual non-convex geometry of heavily occluded objects or mainly produce coarse, inflated completions around the visible seed. The volume-matched dilation control rules out simple uniform expansion, but it does not establish accurate instance-level shape recovery for these cases.

The paper mentions robot manipulation and navigation as possible applications, but does not evaluate these applications. The output is a geometry-only occupancy grid at 128^3 resolution, and the method has low precision at the reported default threshold. It is therefore unclear whether the output is sufficiently accurate for the proposed applications.

Questions for authors

Please address the concerns mentioned above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.