- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents RAPP3R for physically plausible multi-object 3D reconstruction, combining inter-object attention, spatial encoding, and reinforcement learning. While the method improves reconstruction quality, its contributions are largely incremental combinations of existing techniques, and the experimental evaluation remains insufficient to fully support its claims of physical plausibility.
Critical strengths
The paper addresses a meaningful problem and demonstrates improvements in reconstruction accuracy and physical plausibility through its proposed framework.
Critical weaknesses
-
Limited methodological novelty. The proposed framework largely combines existing techniques. Gated Inter-Object Attention integrates the inter-object attention mechanism from MIDI [1] with standard zero-initialized gating. Similarly, the RL formulation primarily adapts existing Flow-GRPO/GDPO techniques, with limited fundamental innovation in the learning algorithm.
-
Insufficient justification of Volume-RoPE. The improvements attributed to Volume-RoPE may partly arise from extent-dependent attention scaling rather than more effective spatial relationship modeling. The current experiments do not sufficiently isolate these effects.
-
Lack of competitive architectural baselines. The evaluation lacks simple yet competitive alternatives to Volume-RoPE, such as Center-RoPE with bounding-box size embeddings, Center-RoPE with learned extent-dependent attention bias, and 3D relative position bias. These comparisons are necessary to determine whether the proposed sinc-based formulation offers meaningful advantages over simpler spatial encoding strategies.
-
Limited evaluation of physical plausibility. The reported Collision, Floating, and Ground metrics primarily measure geometric consistency rather than actual physical stability. Following simulation-based evaluations in REST3D [2], the authors should consider reporting stability under gravity, post-simulation object displacement, and the proportion of physically stable reconstructions.
References
[1] Huang et al., MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generation. CVPR, 2025.
[2] Ma et al., REST3D: Reconstructing Physically Stable 3D Scenes from a Single Image. 2026.
Questions for authors
-
Have the authors tested simpler alternatives to Volume-RoPE, such as Center-RoPE with bounding-box size embeddings or learned extent bias?
-
Can the physical stability of the reconstructed scenes be evaluated through simulation, using metrics similar to those in REST3D?
-
How does the method compare with physics-based optimization approaches in terms of reconstruction quality and inference time?
5 comments
5. Quality and Time Compared to Optimization
| ADD-S↓ | [email protected]↑ | Collision↓ | Floating↓ | Ground↓ | Time↓ | |
|---|---|---|---|---|---|---|
| SAM3D | 0.0818 | 0.7455 | 18.33 | 1.74 | 17.44 | 60.22 |
| REST3D | 0.1617 | 0.5091 | 0.00 | 3.34 | 27.24 | 1046.83 |
| Ours | 0.0661 | 0.8909 | 7.78 | 1.97 | 11.31 | 63.22 |
(inference time was measured on RTX Pro 5000)
Here, we report quality and inference time comparison to physics-based optimization method, REST3D. We note that other methods (e.g. SPARCS, -scene, etc.) mentioned in the paper did not release their source code so that we could not measure their inference time as the implementation details are unavailable. We used one sample (first frame) for each 12 scenes in YCB-V due to the limited rebuttal period.
As can be seen in the inference time, our approach requires similar time to the pre-trained model. On the other hand, REST3D required about 17 times longer time, with 977.45 seconds (among 1046.83 seconds) consumed at post-optimization. Moreover, despite longer inference time, optimization often produces bad results especially when there are multiple local minima (e.g. separated two collided objects in a wrong direction). Therefore, our RAPP3R outperformed REST3D in all the metrics except for collision which can be perfectly solved through optimization.
We additionally inform that PICASSO spent 2551.35 seconds per scene in the same 12 samples with (full 900 samples) quality comparison in the paper.
Thank you for the detailed responses. While I appreciate the practical improvements and the efficiency advantage over optimization-based methods, my main concerns remain insufficiently addressed. The mathematical derivation of volume-RoPE explains the origin of the sinc factor but does not establish whether the improvements arise from meaningful spatial relationship modeling or simply from extent-dependent attention modulation. The additional results show that Center-RoPE with bounding-box embeddings outperforms Volume-RoPE on four of five metrics, while standard Center-RoPE achieves fewer collisions. These results demonstrate the stronger competitiveness of simpler approaches. Compatibility with FlashAttention is a practical benefit but does not establish its methodological superiority. The comparison with REST3D demonstrates a compelling efficiency advantage. However, geometric consistency metrics alone cannot establish physical stability, which is central to the paper's claims. Overall, while I acknowledge the practical value of the proposed framework, the additional evidence remains insufficient to establish its core methodological contributions. Therefore, I am inclined to maintain my original score.
We appreciate the reviewer’s valuable feedback. Here are the responses to your specific concerns that are not addressed in Common Response.
1. Novelty
Please refer to Common Response 1.
2. Isolation of Volume-RoPE and Extent-Dependent Attention Scaling
If what the reviewer calls “extent-dependent attention scaling” refers to the sinc factor in Volume-RoPE, we would mention that the scaling itself is a core part of Volume-RoPE and cannot be separated from “spatial relationship modeling.” As shown in Figure 2 and Equation 1, the scaling term is mathematically equivalent to integrating (continuously averaging) the RoPE between a pair of objects. Using sinc is not an arbitrary or heuristic scaling - it is derived mathematically by considering the spatial and volumetric relationship between two objects. Therefore, the scaling is our “spatial relationship modeling,” and the improvements come from the scaling because it is the core of the proposed idea.
3. Alternate Approaches for Volume-RoPE
Thank you for suggesting promising alternative approaches. We have tested alternative options and reported the results in the table. We are afraid that we excluded Center-RoPE with learned attention bias because it is not suitable for FlashAttention (FlashAttention computes QK multiplication and softmax at the same time for efficiency, but attention bias should be added between operations), takes a long time in RL sampling, and due to limited computational resources and the rebuttal period.
As shown in the table, the bbox-size embedding exhibits a deterioration in collision score compared to the SAM3D baseline. 3D relative position bias has more collisions than ours and is has even worse floating score compared to SAM3D.
Please note that attention biases are not suitable for FlashAttention. Therefore, RoPE-based approaches can benefit from FlashAttention more than the suggested attention-bias approaches. In addition, for bbox size embedding, extra MLPs and trainable embeddings are needed, increasing training and inference cost.
4. Physics Stability
Please refer to Common Response 4.