- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a pose-reasoning pipeline for multi-view compositional 3D generation. Instead of treating view-specific object pose hypotheses independently, the proposed method coordinates the hypotheses using multi-view geometry through a Guide-Route-Reconcile paradigm. It first uses VGGT-omega to estimate the camera poses to guide the object pose generation, then it improves object layout by selecting the most plausible layout based on the geometric observability. Lastly, it uses differentiable Gaussian Splatting to jointly optimize the camera poses and object poses. Results show that the proposed method can produce a more accurate object placements and scene layouts compared to the baselines methods.
Critical strengths
- The idea overall is clearly presented, and the paper is also well-motivated.
- The obtained scene layouts and object placements are better than the baseline models on ARSG-110K and MIDI-3D-Front.
Critical weaknesses
The whole system seems to be a combination of multiple existing methods / techniques, which raises the concerns of 1) novelty, 2) robustness if individual components fail, and 3) computational overhead. I would like to hear how the authors defend their ideas, and would like to see a robustness analysis.
Questions for authors
There are quite some post-processing refinement steps like in sec 4.2 and 4.3. Can you apply the proposed Route and Reconcile steps to the output of MV-SAM3D or other methods? This is to see if the proposed components themselves are universally useful, and which parts of the system bring most of the gains.
5 comments
Response to Reviewer RyeY
Q1. Applicability of Route and Reconcile to Existing Methods.
Yes. Route and Reconcile can be applied to the generated object assets and view-conditioned pose hypotheses of MV-SAM3D. This is supported by our controlled ablations. On MIDI-3D-Front (4 views), applying Route alone improves layout IoU from 0.597 to 0.617 (Table 2), while further incorporating Reconcile increases it to 0.677 (Table 4, w/o Guide), compared with 0.684 for the full SPOON pipeline. These results show that the downstream pose coordination stages contribute effective gains independently of Guide.
Our additional few-view experiments further support this finding. On ARSG-110K, Route improves IoU from 0.5020 to 0.5216 (2 views) and from 0.5760 to 0.5883 (3 views). Adding Reconcile further increases IoU to 0.5476 and 0.6364, respectively. These results demonstrate the effectiveness of Route and Reconcile across different datasets and input-view settings.
Few-view comparison on ARSG-110K.
| Views | Method | CD-S ↓ | F-S ↑ | CD-O ↓ | F-O ↑ | IoU-B ↑ |
|---|---|---|---|---|---|---|
| 2 | MV-SAM3D | 0.0954 | 73.36 | 0.1984 | 59.29 | 0.5020 |
| 2 | + Route | 0.0885 (+7.25%) | 74.85 (+2.02%) | 0.1817 (+8.39%) | 60.92 (+2.74%) | 0.5216 (+3.90%) |
| 2 | + Route + Reconcile | 0.0823 (+13.80%) | 76.87 (+4.78%) | 0.1709 (+13.86%) | 63.95 (+7.86%) | 0.5476 (+9.09%) |
| 3 | MV-SAM3D | 0.0863 | 80.38 | 0.1424 | 68.60 | 0.5760 |
| 3 | + Route | 0.0841 (+2.58%) | 81.24 (+1.07%) | 0.1372 (+3.67%) | 69.71 (+1.61%) | 0.5883 (+2.13%) |
| 3 | + Route + Reconcile | 0.0807 (+6.50%) | 84.21 (+4.76%) | 0.1327 (+6.82%) | 75.25 (+9.69%) | 0.6364 (+10.47%) |
Relative improvements over MV-SAM3D are shown in parentheses. Guide is disabled in both settings.
These results demonstrate the complementary contributions of the three stages: Route improves pose hypothesis selection, Reconcile further enhances scene-level layout coherence, and Guide provides additional benefits through geometry-grounded pose generation. Beyond MV-SAM3D, Route and Reconcile are not specific to a particular generative backbone, suggesting broader applicability to methods with compatible representations.
Dear Reviewer RyeY,
As the discussion period is closing soon, we would like to kindly follow up. Our detailed responses are above; in case it helps, we highlight three results particularly relevant to your concerns:
- Independent effectiveness of Route and Reconcile (W1, Q1): On MIDI-3D-Front, applying Route and Reconcile to MV-SAM3D without Guide improves layout IoU from 0.597 to 0.617 and 0.677, respectively. Additional 2/3-view experiments further demonstrate their effectiveness when Guide is disabled.
- Robustness under inaccurate reconstruction (W2): Guide fallback increases from 37.70% to 59.78% for the highest-reconstruction-error group, with conditional RMSE fallback rising from 12.98% to 43.08%. Component ablations also show that SPOON retains improvements over MV-SAM3D when individual stages are removed.
- Computational overhead (W3): Our additional experiments cover 4–12 views and 2–6 objects. Guide and Route introduce only 3.8%–6.1% additional runtime, with no increase in peak GPU memory, while Reconcile accounts for the primary additional runtime.
We hope these results and clarifications address your concerns and would appreciate your feedback if any questions remain.
Response to Reviewer RyeY
1. Novelty and Technical Contributions.
The central contribution of SPOON lies in geometry-grounded pose reasoning for coherent compositional scene generation. While existing models provide reconstructed scene geometry and generated object assets, accurately placing these assets into a coherent scene-level configuration remains challenging.
SPOON addresses this challenge by making reconstruction-derived geometry directly inform three stages of object placement: Guide establishes a reconstruction-to-canonical surface bridge and incorporates geometry-derived pose guidance into the generative flow. Route evaluates the geometric reliability of view-conditioned pose hypotheses, and Reconcile jointly refines object placements and camera poses while preserving the generated asset geometry.
Importantly, these contributions are evaluated under the same foundational models as MV-SAM3D, including its existing layout injection and pose refinement. The controlled ablations in Table 4 further demonstrate the benefits of the proposed mechanisms beyond the inherited components.
2. Robustness to Component Failures.
SPOON incorporates an explicit fallback mechanism to reduce the risk of unreliable geometric guidance during pose generation. Specifically, Guide is disabled when fewer than four valid views are available or the surface-alignment RMSE exceeds 0.15, reverting to the original pose generation process while retaining Route and Reconcile.
We further analyze Guide's reliability on ARSG-110K (4 views) by ranking objects according to their geometric reconstruction error, categorizing the worst 20% as Poor and the remaining 80% as Normal. As shown below, the overall fallback rate increases from 37.70% to 59.78% for the Poor group. Notably, the insufficient-view rates remain comparable (28.42% vs. 29.35%), whereas the conditional RMSE fallback rate rises substantially from 12.98% to 43.08%. This indicates that the surface-alignment reliability check more frequently rejects guidance for objects with larger reconstruction errors.
Guide fallback under varying reconstruction quality on ARSG-110K (4 views).
RMSE fallback is measured among objects with at least four valid views; the other rates are computed over all objects in each group.
Furthermore, Table 4 evaluates the effect of removing individual components on MIDI-3D-Front (4 views). Removing Guide, Route, or Reconcile yields IoUs of 0.677, 0.669, and 0.617, respectively, compared with 0.684 for the complete pipeline and 0.597 for MV-SAM3D. These results show that SPOON retains improvements over the baseline even when an individual stage is omitted.
3. Computational Overhead.
We further profile the computational overhead of SPOON across 4/8/12 input views and 2/4/6 objects, reporting the runtime of each stage and peak GPU memory below.
Guide and Route introduce modest additional costs, accounting for only 3.8%–6.1% of the MV-SAM3D baseline runtime across the evaluated settings. Reconcile constitutes the primary additional runtime due to joint object–camera optimization. Importantly, all stages execute sequentially, and none exceeds the peak GPU memory of the baseline generation process. Thus, SPOON maintains the same peak GPU memory footprint as MV-SAM3D across the evaluated configurations.
These results characterize the computational trade-off: geometry-guided generation and pose routing introduce limited overhead, while scene-level reconciliation incurs additional optimization time to improve object placement and spatial coherence.
Stage-wise runtime and peak GPU memory under varying numbers of views and objects.
G+R Overhead is measured relative to the MV-SAM3D baseline runtime. Peak memory is unchanged from the baseline, as all stages are executed sequentially.