acceptodds
Under review as a conference paper at ICLR 2027

TopCode: Disentangling Visual Evidence and Physical Dependencies for Executable 3D Scene Synthesis

Abstract

Generating executable 3D indoor scenes from top-down references supports interior design and embodied intelligence. However, existing methods struggle to reconcile visual fidelity with physical completeness while preserving unrelated content during revision. We propose TopCode, an agent harness that translates Top-down images into executable scene Code by disentangling physical dependencies from visual evidence. Scene Memory preserves accepted attributes and their evidence in DeltaTree, while a separate Coupling Graph captures physical dependencies. Incremental scene compilation in Blender completes unobserved geometry and articulation, accepting updates only after visual and execution checks pass. When checks fail, counterexample-guided revision uses dependencies and hierarchical pose composition to localize changes while preserving accepted values with valid visual support. Across 100 top-down references and 120 local editing tasks, TopCode improves reference consistency, scene validity, and preservation of unrelated content over evaluated baselines. Generated scenes also improve navigation through post-training and support simulated manipulation, demonstrating their utility for embodied tasks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Scene Synthesis, 3D Layout Generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_GZU9OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents TopCode, an agentic framework for generating executable 3D indoor scenes from top-down reference images. The key idea is to disentangle visual evidence from physical dependencies to improve visual fidelity, physical completeness, and consistency during scene revisions. Specifically, the authors introduce two complementary scene representations: DeltaTree, which maintains hierarchical scene attributes along with their provenance and supporting visual evidence, and Coupling Graph, which explicitly models physical dependencies such as support, contact, kinematics, and motion clearance. Exhaustive experiments demonstrate improvements in visual consistency, scene validity, and editing performance over the evaluated baselines. The authors further explore the applicability of generated scenes to embodied intelligence through navigation post-training and simulated manipulation demonstrations.

Critical strengths
  1. The proposed scene memory and dependency-guided revision mechanisms offer a controllable approach to scene generation and editing. The Coupling Graph further introduces explicit physical constraints to guide scene completion and revision.

  2. TopCode demonstrates improvements over evaluated baselines in scene generation and local editing, with preliminary evidence of utility for embodied applications.

Critical weaknesses
  1. Limited Novelty Compared with Existing Agentic Frameworks. The technical novelty of TopCode relative to existing agentic scene generation frameworks is not sufficiently established, particularly in light of the closely related Code-as-Room work. Code-as-Room already addresses essentially the same task of generating executable 3D indoor scenes from top-down reference images through agentic code synthesis. It also incorporates multi-stage scene generation, hierarchical scene representations, persistent cross-stage memory, and iterative rendering-based refinement. These components substantially overlap with the overall design of TopCode. Although TopCode introduces attribute-level visual evidence tracking, a separate Coupling Graph for physical dependencies, and dependency-guided local revision, it remains unclear whether these represent fundamental methodological advances or incremental extensions of existing agentic frameworks. More importantly, the authors neither provide a direct comparison with Code-as-Room nor sufficiently demonstrate the additional benefits introduced by their proposed representations and revision mechanisms.

  2. Limited Backbone Evaluation. TopCode is essentially an agent-harness framework. However, the authors evaluate it using only Qwen3.8-Max, making it difficult to establish whether the proposed harness generalizes across different foundation models. Furthermore, the paper does not provide a direct comparison with more advanced models or agent systems, such as GPT-6 Astra. In my view, Astra is likely to achieve substantially better performance than the results reported by TopCode.

  3. Limitations of the Scene Generation Benchmark. The scene generation benchmark has two major limitations. First, the benchmark consists exclusively of 2D top-down scene images generated by GPT-Image2, without corresponding ground-truth 3D scenes. Consequently, the authors cannot evaluate the generated scenes using 3D geometric metrics, such as 3D IoU, making it difficult to assess the actual quality and accuracy of the reconstructed 3D geometry. Second, the authors do not evaluate their method on real-world top-down images with complex scene layouts. Surprisingly, the paper does not even provide qualitative results demonstrating the method's performance in such scenarios.

  4. Insufficient Ablation Studies. First, it remains unclear why DeltaTree and the Coupling Graph must be represented as two separate structures rather than being integrated into a unified Scene Graph. For example, a reasonable baseline would be to store visual evidence, physical dependencies, and attribute-update permissions within a single graph while retaining the same validation and local revision procedures. If such a unified representation achieves comparable performance, the necessity and benefits of the proposed disentanglement would be questionable. Second, the authors claim that Dependency Closure helps localize scene revisions. However, they do not provide sufficient quantitative analysis of the actual revision scope, such as the number of objects, attributes, or graph nodes modified during each revision, or how this scope scales with scene complexity. In densely populated indoor environments, physical dependencies may propagate across numerous interconnected objects, potentially turning a supposedly local revision into a large-scale modification.

  5. Insufficient Validation of Physical Completeness and Embodied Applications. For navigation, the authors demonstrate performance improvements through post-training on generated scenes. However, these improvements may primarily result from better visual appearance, furniture layouts, or training-data distributions, rather than the proposed Coupling Graph or other physical modeling components. The experiments do not isolate the contribution of physical dependencies to navigation performance. For manipulation, the experiments are essentially proof-of-concept demonstrations. The authors only show a few predefined interaction tasks, without training manipulation policies or providing systematic quantitative evaluations.

Questions for authors

How do the authors ensure a fair comparison across baseline agentic frameworks when the number of model calls and computational budgets are not strictly controlled? Could the authors also report the average scene generation time, token consumption, and computational cost for TopCode and the competing methods?

Reviewer_NzLwOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

TopCode generates executable 3D indoor scenes from top-down reference images through incremental construction and local revision. DeltaTree stores hierarchical scene attributes with provenance and image-evidence references, while a separate Coupling Graph records physical dependencies. A controller combines dependency closure, hierarchical localization, and evidence-based write permissions to constrain revisions, accepting updates after applicable visual and execution checks. Experiments evaluate generation on 100 shared references, 120 independent editing tasks, sequential editing, and navigation post-training, with additional simulated interaction examples.

Critical strengths
  1. The paper identifies a meaningful tension between visual fidelity and physical completion. Generating interactive scenes from top-down references requires preserving visible layout and appearance while inferring hidden geometry and articulation. The proposed distinction between visual evidence and physical dependencies gives a clear organizing principle: image evidence constrains observed attributes, while physical relations guide the completion of unobserved structure.
  2. Explicit scene memory and constrained local revision form a coherent system design. DeltaTree retains hierarchical scene attributes, provenance, and evidence references, while the Coupling Graph records support, contact, and motion constraints. Dependency closure, hierarchical localization, and write permissions connect failed checks to a restricted set of candidate changes. This design provides concrete controls for coordinating related repairs and protecting accepted content during incremental construction and editing.
Critical weaknesses
  1. The independent benefit and novelty of the central disentanglement mechanism are not sufficiently established. REST3D[1] already represents physical support structure and preserves layout consistency during physics-based refinement. TopCode's methodological distinction from this work is insufficiently explained. Its ablations retain provenance, image evidence, and visual-evidence protection, leaving the independent benefit of the central disentanglement mechanism unisolated.
  2. Important details of dependency maintenance and concrete repair decisions remain unclear. The paper does not specify when newly discovered conflicts become persistent Coupling Graph relations, how obsolete relations and attribute bindings are removed after object replacement or deletion, or whether updates are local or global. It also does not explain how the agent chooses concrete attributes and operations among alternative repairs within the permitted write scope.

[1]https://arxiv.org/abs/2605.30338

Questions for authors
  1. How does TopCode differ from prior physical-relation modeling, and what evidence isolates the benefit of visual-evidence protection?
  2. How are Coupling Graph relations updated after scene edits, and how does the agent choose which attributes to modify?
  3. What repair and token budgets are used, and does the advantage over VIGA persist under comparable computational costs?
Reviewer_NXPHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper adresses the task of 3D scene synthesis from a top down image. The proposed method is agentic and the paper proposes a harness that explicitly separates attributes based on their source. Are they observed from the image (layout for example) or invisible and thus have to be infered (inner structure of furniture). The DeltaTree holds all these attributes and organizes objects similar to Blenders parent-child hierachical graph. The coupling graph contains information about what is physically dependend on each other, i.e. what is coupled. When a scene element has to be corrected due to intersections for example, a controller identifies the attributes in the DeltaTree that are directly related to that and gives write access to them to the agents. Crucially, observed attributes are kept read only. This gating prevents accidental scene changes that are unrelated to the diagonsed failure or would overwrite image evidence. The method outperforms common baselines in scene reconstruction from images or agentic text to scene methods.

Critical strengths
  • The idea to clearly separate observed vs inferred scene attributes and only allow local edits is interesting and seems effective.
Critical weaknesses
  • This paper proposes an agentic harness, but does not evaluate it across models with different capabilities. An evaluation on weaker models/smaller models and also on stronger models (Fable, Astra) would show how much it compensates reasoning failures and how future proof this contribution is. In addition, i would say that the main baselines for are SAGE and SceneSmith and i would recommend moving them into the main paper.
  • The writing is very dense and makes it difficult to identify the contribution and check the method for correctness. The method introduces a lot of components, whose relevance is not immedeatly clear. This in turn makes it hard to follow the experiments and evaluate the impact of the contribution.
Questions for authors
  • What distinguishes the DeltaTree structure from Blenders parent-child hierarchy?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.