acceptodds
Withdrawn from ICLR 2027

Unravel: Recovering an Interactive 3D Scene from a Single Image

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

We present Unravel, a training-free framework that turns a single image into an interactive 3D scene, where every object is an independent and movable asset. Existing generators either produce one entangled geometry in which no object can be moved, or predict all objects in one feed-forward pass, where a wrong guess cannot be detected or repaired. Unravel instead takes a different approach: it first takes the scene apart, and then puts it back together. Starting from the input image, an agentic harness that iteratively peels the image layer by layer, and adopts point cloud as major modality for spatial relationship recovery. Specifically, a vision-language model plans which objects are fully visible, an image editor removes them and restores the occluded regions. Every edit is verified before being incorporated into the reconstruction. This process gradually turns occluded parts of the scene into direct observation, allowing objects to be reconstructed when they are best exposed rather than hallucinated from the original image alone. The scene is then rebuilt in reverse removal order, so that the supporting objects are naturally placed before the objects they support. Since a single-view image only reveals part of an object, we treat the observed points as a lower bound on its size: an asset may grow to cover its points, but can not shrink away from them. Finally, we physically settle the reconstructed scene to remove floating objects and interpenetrations, followed by an agentic audit that corrects remaining placement errors. Extensive experiments on diverse indoor scenes show that Unravel recovers more objects, places them more faithfully, and produces scenes that are more physically-plausible than existing methods.

Primary area:
generative models
Keywords:
3D Scene Generation, Interaction, Agentic Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_pAGDOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Unravel is a training-free framework for recovering an interactive 3D scene from a single image. It iteratively removes visible objects and completes the newly exposed regions using an image editor, verifying each edit before proceeding. Independent assets are generated from each object’s best-observed snapshot. Point maps from successive rounds are aligned and fused, while removal rounds and pixel locations associate points with individual objects. The scene is then assembled in reverse removal order using constrained registration, physical settling, and a final agentic audit.

Critical strengths
  1. The deconstruction-before-recomposition strategy provides a practical way to handle occlusion and generate independently editable assets. Reusing the deconstruction record for point attribution and placement connects the image-editing stage with scene reconstruction.

  2. The framework combines geometric registration with support and collision constraints rather than relying solely on semantic placement. The qualitative examples demonstrate promising asset completeness and scene reconstruction across different indoor environments.

Critical weaknesses
  1. Limited methodological novelty. The methodological novelty is not sufficiently established. Several central ingredients—including iterative object removal and image completion, independent asset generation, geometric registration, and physical correction—have already been explored in prior work. Unravel integrates these ingredients using strong pretrained models, but the paper does not clearly explain what methodological advance this integration introduces. The more specific designs, such as point attribution using removal rounds and pixel locations and one-sided extent constraints, are potentially useful, yet their distinction from existing geometric bookkeeping and partial-observation registration strategies needs clearer justification. Overall, the contribution currently appears to be primarily a system integration rather than a substantial methodological advance.

  2. Insufficient experimental validation. The experimental evidence is insufficient to establish the advantages of the proposed framework. The main quantitative study covers only five scenes containing 160 objects, while comparisons with external baselines are predominantly qualitative. These comparisons do not clearly establish the magnitude of improvement or separate the benefits of the proposed mechanisms from those of the strong foundation models employed. More detailed ablations are also needed to isolate the contributions of edit verification, cross-round alignment, one-sided extent constraints, physical settling, and the final agentic audit. Furthermore, several evaluation references come from Unravel’s own edited images, masks, or fused point clouds. Quantitative baseline comparisons on a larger scene set, together with validation against independent real photographs, would provide stronger evidence of reconstruction accuracy and physical validity.

Questions for authors
  1. A more detailed ablation would help establish the importance of the proposed components. With the foundation models and generated assets fixed, how much do point attribution using removal rounds and pixel locations, one-sided extent constraints, and physical settling each contribute?

  2. The baseline comparisons are mainly qualitative. What are the quantitative differences on the existing test scenes in object placement accuracy, interpenetration, and support validity?

  3. For the real-photograph examples, is there quantitative evidence of reconstruction accuracy beyond the rendered results? If additional views of the same scenes are available, do the completed objects and their placement agree with these observations?

  4. How consistent are the improvements beyond the five evaluated scenes? Please report average performance and variation across a broader range of scene types and clutter levels.

Reviewer_DzdfOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

An agentic approach that iteratively removes objects from the original image using GPT and GPT-image2, reconstructs objects from occlusion-free observations using TRELLIS, and reassembles objects into the point clouds estimated by MoGe3 using ICP.

There is no quantitative comparison of 3D reconstruction quality with 3D metrics.

Critical strengths

None

Critical weaknesses
  • The most critical weakness is that there is no quantitative comparison of 3D reconstruction quality using 3D metrics. The current synthetic datasets can support this comparison. Otherwise, it is extremely difficult to verify a 3D reconstruction approach based on human and VLM visual feedback.

  • The lack of 3D quantitative metrics makes the comparison with Astra hard.

  • In L268, stable pixels defined by the unedited regions are not fully reliable; an image diffusion model can somehow alter the layout of the scene when some objects are moved. The stability of the background regions should be investigated as well.

Questions for authors

Quantitative results under 3D metrics are needed when a paper focuses on 3D reconstruction. I am fine with seeing fewer scenes evaluated under the standard benchmark because of the token issues. But removing the metrics is unacceptable to me, especially when the eval images are generated, not real, and only five are presented in the paper.

Reviewer_DwnWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes a training-free approach of taking a single image and deconstructing it into its constituent objects and recombining them to create an interactive 3D scene. A VLM plans and verifies object removals, while an image editor removes objects and completes occluded regions. Individual assets are reconstructed from their best-observed snapshots. The method aligns and fuses per-round point maps, retains round-and-pixel provenance for object attribution, and recomposes the scene using constrained registration, reverse removal order, physical settling, and a final agentic audit. Experiments include a quantitative comparison with direct reconstruction on five scenes containing 160 objects, qualitative comparisons against MIDI, SAM 3D, and GPT-6 Astra variants, and additional generated and real-image examples.

The framework is coherent and produces promising qualitative results. However, the evaluation process is insufficient and the paper is missing strong comparative comparison with the closest method “Seeing Through Clutter”.

Critical strengths
  1. The paper addresses a useful problem. Recovering editable objects and their configuration from a single image has several applications. The paper shows promising scene level coherence across different indoor scenes.
  2. Reusing the deconstruction history for recomposition is a useful constraint. The extent constraint also accounts for mismatch between complete generated assets and partial point clusters. Altogether these form a coherent pipeline design.
Critical weaknesses
  1. The quantitative evaluation covers five scenes and uses the same mask and fused point clusters as coverage reference. This results in a perfect score which is not meaningful. The human asset evaluation uses crops from the edited, best-observed images as reference which is measuring agreement with generated completions rather than evaluating hidden geometry. Placement comparisons and real-image results are qualitative and there’s no evaluation on placement accuracy, penetration, support, or physical stability.
  2. Seeing Through Clutter already uses VLM based orchestration to iteratively remove objects and reconstruct without task-specific training. Seeing Through Clutter reports benchmark evaluations on 3D-FRONT and ADE20K, and qualitative outdoor examples, whereas this paper does not include any comparable benchmark evaluation or any outdoor examples.
Questions for authors
  1. How does the method perform with independent reference geometry or annotations, rather than the generated masks, point clusters, and edited images? How well does it place objects and perform in terms of physical plausibility?
  2. How much does the proposed recomposition step contribute?
  3. How many scenes were evaluated from Seeing Through Clutter, which foundation-model components were matched, and is the appendix example representative? How does the method perform on the benchmark datasets?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.