- Rating
- 2
- Confidence
- 4(+1)
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a three-stage framework, including scene initialization, environment construction, and scene refinement, for single image to 3D scene generation. At the same time, the paper proposes a geometry-aware layout predictor based on point map supervision to estimate the scale, rotation, and translation of objects.
Critical strengths
- The system design is quite comprehensive, integrating segmentation, 3D object generation, layout prediction, room construction, and scene refinement into an end-to-end scene generation framework.
- Converting the continuous geometric parameter prediction into the "geometry proposes, agent verifies" approach is relatively intuitive.
Critical weaknesses
- "The geometric module generates candidates + VLM makes acceptance/rejection decisions" is more akin to a reasonable system design principle rather than a new agent reasoning method, lacking innovation.
- The performance mainly stems from the combined effect of multiple modules such as SAM3, SAM3D, layout predictor, a large number of manually designed candidate rules, and deterministic physical finalization, etc. There is no fundamental method innovation.
Questions for authors
- The experiment extensively uses VLM. How can we ensure that the evaluation does not have model bias?
- Problems such as floating objects, collisions, and wall alignment ultimately still rely on fixed deterministic physical finalization. It is unclear whether the performance improvement comes from the agent's reasoning or from stronger geometric rules and post-processing.
- The method was only tested on 160 real indoor images. The main perceptual quality indicators were derived from the VLM scoring of Qwen3.5-9B and the CLIP similarity. There were no human ratings, and the reliability of the VLM scoring was unclear.
- The method is only applicable to indoor rooms and cannot directly handle outdoor, large city, multi-room or multi-level scenarios.
2 comments
Rebuttal by Authors (2/2)
(Q2) Even without finalization, ours already outperforms the other methods, and finalization changes our scenes only a little. Without finalization, ours is already judged above every baseline (Qwen 7.10 / 7.40 on MIT-67 / COCO in Table 5, 7.24 over all 160 scenes, against 7.14 for the best baseline in Table 3) and already ranks first or second on every physical metric against the baselines of Table 1 (Table S1). Finalization leaves about half of our objects within 0.2 cm and about two thirds within 5 cm, almost never rotates them, and raises our judge score (7.24 → 7.41). Applied to our Stage 2 instead, the same finalization lowers the score, so our advantage comes from the agent's decisions in Stage 3.
| Change from finalization (ours) | Value |
|---|---|
| Objects moved < 0.2 cm | 47% |
| Objects moved < 5 cm | 67% |
| Objects rotated more than 1° | 0.4% |
| Object pixels changed, median scene | 13% |
| Qwen of ours, without → with finalization | 7.24 → 7.41 |
| Qwen of our Stage 2, without → with finalization | 7.00 → 6.79 |
Figure 7 and Figure S7 show our scenes without and with finalization from the camera of the photograph, where the two look similar. Furthermore, the result does not hinge on finely tuned constants, as it comes from the agent's facilitated 3D reasoning (W2). When each shared finalization constant, such as the 3 cm distance at which an object is attached to a wall, is halved or doubled, every physical metric stays within about one point of ours, and constants chosen per scene by our agent do worse (Table S7). We discuss this in Supp. Sec. G.6.
(Q4) We agree that our framework covers a single indoor room, and we now state this scope as a limitation. Outdoor, city-scale, multi-room, and multi-level scenes are outside its scope, so we scope the title and abstract to single-image indoor scenes and state these limits in Sec. 4.2 (Limitations and future work). As a direction, more views could enter our framework as further measurements, since its geometry already measures what the input shows. Figure S6 shows a first step on one SAGE scene, where two more views refine the objects without any agent call.
We thank the reviewer again for the constructive comments, all of which are now in the revision.
Rebuttal by Authors (1/2)
We sincerely thank Reviewer ESKE for the review and for finding the "geometry proposes, agent verifies" conversion intuitive.
(W1) Our novelty is to identify the agent's weakness at 3D spatial reasoning as the core problem, and our framework facilitates this reasoning, whereas prior methods leave the agent to do it alone. Current VLMs often misjudge sizes, distances, and angles in 3D (SpatialVLM, VSI-Bench). Prior propose-then-verify methods offer no way around this weakness, as the agent still reasons in 3D when it writes the values, sets a solver's arguments, or picks among sampled layouts (Supp. Sec. G.4). Our framework instead measures everything it can from the photograph and its point map, and asks the agent only yes/no questions that measurement cannot decide, such as which masks belong to one object. Our three stages build this observed scaffold (Sec. 2), and the next response shows that the gain comes from facilitating the agent's 3D reasoning rather than from the modules.
(W2) The modules alone do not make a good scene, and neither does a VLM that writes the scene. Thank you for raising this. SAM3D alone is judged lowest in Table 1, and REST3D, which uses the same SAM3, SAM3D, and backbone with its own agent, is judged lower than ours and puts over ten times more objects through walls (Table 1). With our own modules, what matters is what the agent is asked, as writing the values or the whole plan fixes far fewer of the problems our checks flag and is judged lower.
Note that Fixed is the share of problems flagged by our geometric checks, such as floating or wall-penetrating objects, that no longer fail right after the agent's decisions, before any finalization. Applying every candidate without the agent's yes/no decisions keeps only about half of the objects (Table 5), and with our questions even the open 27B model scores above baselines run on Claude Opus 5.5 (Table S8). Details are in Table 7 and Sec. 4.2.
(Q1, Q3) Other judges and human raters also rank ours first.
All three judges and the raters rank ours first (Table 3, Sec. 4.1). On 137 further test photographs (85 MIT-67, 52 COCO), ours is again judged highest, with the highest CLIP and the fewest floating and wall-penetrating objects (Table S15).