- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents an agentic system for reconstructing real-captured indoor environments as sim-ready 3D scene. The input to the system is the 3D scan of a room and the corresponding RGB frames. The overall pipeline is divided into two stages, where the first stage reconstructs objects and an initial layout, whereas the second stage refines the scene through iterative rendering and comparison. The authors test their system on 10 rooms and compare it with frontier models, such as Astra and Fable, showing better results.
Critical strengths
- The quality of the reconstrution looks pretty impressive, and are better than vanilla coding agents. The comparisons with GPT Astra and Claude Fable are informative and make the position of this paper clearer, although the implementation details of those baselines are not very clear.
- The agentic design for this task makes sense, as the scene reconstruction is a long-horizon task requiring many steps of reasoning and tool use (e.g., object reconstruction / collision check), and a dedicated harness can be useful.
Critical weaknesses
- My main concern with the paper is the evaluation: The authors only mention that they use 10 scenes, but it is unclear which dataset the scenes are from, or whether they were collected by the authors. The authors should either properly cite the data source or provide details on how the data are captured. Moreover, 10 scens are very limited in scale. Finally, the authors should test on the same scenes are the baseline LiteReality for an apple-to-apple comparsion, espeically since LiteReality seems to totally fail in the scenes selected by the authors as shown Fig 3.
- The authors should provide the exact prompts used for the agents in the proposed system, as well as the agent-based baselines, so that the results are reproducible and it is possible to judge if the baseline implementations are best-effort.
- The writing of the paper needs improvement, especially the methods section. There are some undeclared terms such as "Geometric checks", "Deterministic repairs", etc (L203-209), making it very hard to understand how the layout agent work. The 3 levels in section 3.3.1 are also confusing, and I think it would be helpful to explain with some diagrams or example figures.
- This is not a particular weakness of this paper, because papers have been published in top venues on the same task. But I kind of don't understand why the scene level real to sim is a useful task, especially compared to the text-to-3D-scene task. Note the approcah is not very scalable, as it requires carefully scan and reconstriction of the scene. The reconstruction is also far from faithful, compared to 3D reconstrtion method such as VGGT or gaussian splatting. From my opioion, both of these places this real-to-sim task in an awkward position, where the reconstrructed scenes are neither diverse nor faithful.
Questions for authors
N/A
2 comments
Response needed
Dear Reviewer, As the discussion phase deadline is approaching, could you please review the authors' response and let us know whether your concerns have been adequately addressed, and if you have any further questions? Thank you very much for your time.
We thank the reviewer for their detailed feedback and thoughtful questions. We would like to first address Weakness 4 regarding the usefulness and significance of the task.
First, this task produces 3D environments that are directly compatible with traditional graphics and simulation pipelines, making the results immediately useful for robotic simulation, training, and industrial digital twins. Crucially, robotics requires interactive, articulated, and physically grounded environments for training and evaluation, whereas other visually faithful 3D representations, such as Gaussian splatting, image-to-3D reconstructions, and point clouds, cannot be directly used for these purposes without further processing. Our pipeline therefore produces practical, reusable assets that can be directly integrated into a wide range of commercial and robotic applications.
Second, we believe this approach offers a promising path toward scalable reconstruction. As agent capabilities continue to advance, increasingly complex and diverse environments can be reconstructed with less manual intervention. The improvements observed from LiteReality to LiteReality-Agent demonstrate this potential. Prior to LiteReality-Agent, there are no publicly available tools capable of performing this task at all. A central motivation of our work is to make this challenging reconstruction process significantly more accessible, streamlined, and practical for the broader community.
We hope this clarifies both the usefulness of the task and the practical value of our contribution, and we respectfully encourage the reviewer to reconsider this aspect of their assessment. We address the remaining concerns below.
CW1: Dataset and comparison with the original LiteReality paper
The ten scans were captured in real environments, including offices, kitchens, bedrooms, meeting rooms, and hallways, covering a diverse set of scenes. We also provide a comparison on the original LiteReality paper’s scenes below, including its other baselines.
LiteReality-Agent upgrades the original deterministic pipeline to an agentic approach, providing substantial improvements in reconstruction quality.
CW2: Prompt fairness and additional comparisons
We agree and will provide the exact text prompts. We use the same task prompt and input information for the corresponding comparisons. We also provide more detailed comparisons covering:
1. Multiple agents with and without the proposed harness
2. The same LLM at four reasoning-effort levels (Astra as base model)
LiteReality-Agent achieves better scores than the baseline at each effort level in these results.
CW3: Writing and methodological details
We included additional methodological details in the supplementary material and open-sourced the entire codebase to support reproducibility. We will improve the main paper’s explanations and add diagrams to make the workflow easier to understand.