- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper introduces a pipeline that leverages Large Language Models (LLMs) to perform 3D spatial reasoning and placement by writing executable programs. An evidence programs represent the 3D scene as voxels and surface normals arranged on a regular grid to reason about the scene. A pose program uses this representation to place an object into the scene according to a user instruction. Numerical and scene renderings are used to iteratively refine the program until the target pose is achieved. To demonstrate the effectiveness of the proposed pipeline, the paper introduces a benchmark, VoxProg-Bench, with 71 placement tasks across diverse spatial requirements. Experiments show that the proposed approach outperform other approaches in comparison in terms of Intent Alignment and Physical Validity.
Critical strengths
The paper demonstrates that recent advancement of frontier LLMs can contribute to solving specialized problems in 3D in a zero-shot manner with competitive performance.
Critical weaknesses
-
Overall, the technical contribution of the paper is minor. The proposed pipeline only formulates the problem for a frontier LLM (e.g. ChatGPT 5.6) to write programs to solve it. It is not clear to me how the proposed pipeline differs from delegating the task to an agentic coding assistant and asking it to solve the task. Moreover, the paper does not provide any useful insights or analysis that can be interesting for this type of problems.
-
The paper cites the highly relevant work PlaceIt3D (Abdelreheem et al., 2025), but does not compare against it. Given that PaceIt3D is one of the first models to perform 3D object placement using LLMs, it is crucial to compare against it.
-
The paper evaluates only ChatGPT 5.6 Sol as a backbone LLM. To the experiments more exhaustive and informative, more proprietary and open-source models should be evaluated.
-
The benchmark is limited in size and scope. To be able to tell if the benchmark is exhaustive enough, the taxonomy and definition of the benchmark need to be explained in more details (e.g. why is the benchmark limited to tabletop scenes?)
-
The evaluation protocol is limited in scope; it only evaluates Intent Alignment and physical validity. For an instruction-based placement task in 3D, many evaluation axes need to be covered such as language adherence, position, and rotation accuracy. Given that Intent Alignment is computed on rendered images rather than 3D, it poorly captures these axes.
Questions for authors
-
Why is the benchmark limited to tabletop scenes? How is the taxonomy of the benchmark defined? Does it cover a large and diverse number of placement tasks?
-
Why is Intent Alignment done in the image space rather than 3D? Given the nature of the task as 3D placement, the evaluation should be done in 3D. This is evident in the failure case in Figure 7 where a 3D-based metric would have captured this failure.
-
It is not clear how occlusions are handled at different parts of the pipeline. For instance, if the contact points of the object are self-occluded and can not be directly raycasted.