Align3D addresses single-image indoor scene layout generation. It constructs a scene graph using object detection, depth estimation, and camera estimation, and combines image and graph features to condition a layout diffusion model. Image alignment is supervised through a differentiable Gaussian proxy renderer, while graph consistency is enforced through soft object matching and relational losses. The proposed observability-aware fidelity-preserving guidance (OFG) projects conflicting collision gradients and weights them by object observability. Experiments on 3D-FRONT report competitive geometry and improved image-alignment scores, supported by component ablations, a human preference study, and qualitative examples on real images.
Presentation: 2 — Generally understandable, but requires substantial effort in places.
The overall pipeline is understandable, but the training objective and gradient implementation need a more precise description.
Informativeness: 2 — I learned something interesting or useful.
Soundness: 1 — I have significant concerns about the paper’s soundness.
Critical Strengths
-
The paper addresses a relevant tension in image-conditioned scene generation: improving an inferred layout’s physical plausibility can move visible objects away from the input observation. Using observational reliability to determine where layout adjustments are permitted is a useful direction, particularly when the scene graph itself is estimated from imperfect single-view evidence.
-
The experiments include comparisons with several recent methods and separate ablations for image alignment, graph alignment, confidence weighting, and gradient projection. The reported results suggest that image and graph supervision provide complementary benefits. Align3D improves CLIP and DINO similarity over SceneGen while remaining competitive on geometric metrics, although SceneGen retains better Chamfer distance and FID.
Critical Weaknesses
-
The fidelity-preservation argument does not extend to the final OFG update. Equation 14 removes the conflicting component of the quality gradient, making the projected gradient approximately orthogonal to the alignment gradient. Equation 17 then multiplies that projected gradient by the nonuniform object mask C. This operation generally destroys orthogonality: gco-align⊤g~quality=0 does not imply gco-align⊤Cg~quality≥0. The masked correction can therefore conflict with alignment again. The first-order argument following Eq. 14 is insufficient to justify the final update, which is central to the claimed fidelity-preserving behavior.
-
The transported observability score is not bounded as intended. The matching matrix in Eq. 7 is normalized over generated objects for each observed node. However, Eq. 16 sums over observed nodes without normalizing the incoming weights. Several observed nodes can assign substantial probability to the same generated object, allowing its observability score to exceed 1. For example, two observed nodes with observability 0.78 and matching weight 0.9 to the same generated object give a score of 1.404. The corresponding entry of C becomes negative, reversing that object’s projected quality update rather than suppressing it. No normalization or clipping is specified in the paper or Appendix A.
-
The diffusion training objective and gradient domain are underspecified. Equation 1 presents the objective as alignment plus a modified quality term, and Table 2 similarly lists these terms as the training objectives. A standard diffusion denoising loss is not explicitly included, although Appendix A specifies a diffusion schedule and DDIM sampling. It is also unclear whether the gradients in Eqs. 14–17 are computed with respect to predicted clean layouts, noisy layouts, or network parameters. An object-block mask has a natural interpretation in layout space, but its application to network optimization requires an explicit description. These omissions make it difficult to determine the actual training procedure and interpret the ablations.
-
The confidence-calibration claim lacks supporting evidence. Node confidence combines detection confidence with a hand-designed distance decay, while edge confidence additionally decreases with inter-object distance. These are reasonable heuristics, but the paper does not show that they are calibrated against localization error or relation correctness. The construction of the global camera confidence is also unspecified. Furthermore, Eq. 8 weights observed relations uniformly; edge confidence enters the alignment objective through the global scalar csg, rather than directly reducing the contribution of an unreliable relation. This provides weaker local uncertainty handling than the description suggests.
-
The evaluation does not directly establish physical plausibility beyond the observed view. Chamfer distance, F-Score, and matched ground-truth box IoU measure reconstruction agreement rather than collision avoidance or support validity. The human study uses matched-view renderings, which also limit the assessment of hidden interpenetrations and clearances. Direct collision and support measurements, together with an evaluation of occluded regions, would better support the central claim. The benefit of the observability mask over projection alone is relatively small in Table 3, and no uncertainty estimates are provided to establish its consistency.
Questions For Authors
-
Clarification of the implemented OFG update would help resolve the two mathematical concerns above, particularly any normalization or clipping after Eq. 16 and any conflict check after applying C. Measurements of alignment-gradient conflict before and after masking would help establish the behavior of the final update.
-
The complete training objective, including the diffusion denoising term and its weighting relative to the auxiliary losses, is needed to interpret the method. Pseudocode specifying the gradient variables, timestep handling, and how the modified layout gradients reach the network parameters would make the procedure reproducible.
-
Details of the depth checkpoint, conversion to metric depth, world-coordinate alignment, and construction of camera confidence would clarify the single-image geometry pipeline. An analysis relating the proposed confidence scores to actual node and relation errors would help substantiate the calibration claim.
-
A precise account of the evaluation settings would help assess the comparisons. Appendix A describes known-camera substitution and global-yaw correction as optional; identifying which results use these options, and whether equivalent processing is applied to baselines, would remove this ambiguity. The treatment of retrieved assets versus directly generated meshes also needs clarification.
-
Collision rates, penetration severity, and support violations would provide a more direct assessment of the intended physical improvements. Reporting these alongside image alignment for the sum, projection-only, and full OFG variants would help establish the contribution of observability weighting.
Flag For Ethics Review: No.
Details Of Ethics Concerns
No specific concern requiring a separate ethics review was identified from the submission.
Rating: 2 — Weak rejection.
The motivation is useful and the reported results are promising. However, the final OFG update is not covered by the stated fidelity-preservation argument, and the observability transport can produce negative correction weights as written. Together with the incomplete training description, these issues prevent a reliable assessment of the main contribution. Clarification that the implementation addresses these problems, supported by corresponding experimental evidence, could change my assessment.