- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper addresses scene-graph-to-3D layout generation, motivated by the observation that existing graph-conditioned generators reach high edge-level relation accuracy while many scenes still violate at least one requested relation. TOSCA represents each relation as an explicit set of allowed feature values (a "transport set"), e.g., a half-space for directional relations, a box for close-by, and two reflection branches for symmetrical-to, with inverse relations sharing parameters. For each edge, the projection residual onto its set conditions a graph-attention network inside a flow-matching generator, and is also used for a geometric correction term. After sampling, 500 residual-gradient refinement steps further reduce violations. For fixed affine convex subproblems, the paper derives infeasibility certificates (negative cycles for difference constraints), primal-dual bounds on the minimum violation, and a decay bound on how relation edits affect anchored optima.
On SG-FRONT, TOSCA reports higher relation satisfaction (e.g., bedroom Micro .997 vs. about .975 for baselines, with large gains on close-by and symmetrical-to) and lower FID/KID than retrained EchoLayout, MMGDreamer, and InstructScene. Further results cover Inf3DLayout, the InstructScene benchmark, component ablations, conflict detection on synthetic contradictions, relation editing, footprint overlap, and a user study with 30 participants.
Critical strengths
-
The evaluation is very careful. The authors retrain baselines on the same splits with a shared renderer, average over seeds and sampling runs, and audit things that are usually ignored: training overlap in the released EchoScene checkpoint, agreement between annotations and the benchmark predicates, the frame dependence of the symmetrical-to rule, and layout twins between train and test. They also report Joint (scene-level) satisfaction in addition to edge-level scores. The user study uses matched rendering and reports confidence intervals. Even where I disagree with the conclusions, I could trust that the numbers were measured carefully, and several of these diagnostics are useful to the community on their own.
-
Making relation constraints explicit, instead of relying only on learned relation embeddings, is a reasonable and practical direction. It enables per-edge checking and correction, and it makes it possible to distinguish an infeasible request from incomplete optimization. The fact that the feasibility check found three contradictory scenes in the released SG-FRONT annotations is a nice concrete demonstration of this.
Critical weaknesses
-
The relation-satisfaction gains seem to come mostly from post-hoc refinement rather than from the proposed learned components. In Table 18, removing final refinement drops Micro to 0.829 and close-by to 0.576, which is clearly below all baselines in Table 2 (around 0.95–0.975 Micro). With singleton sets, Micro is 0.661. So the residual-conditioned flow by itself does not satisfy relations better than existing methods. Most of the improvement comes from 500 steps of projected gradient descent toward sets that were fitted to approximate the benchmark predicates. This makes it essential to know what the baselines achieve on the relation metrics when given the same refinement. The paper only reports FID for "EchoScene + our final refinement" (Table 16), not relation satisfaction. Without this comparison, I cannot tell whether the contribution is the generator or a constraint-projection post-processing step that could be applied to any method.
A related concern is symmetrical-to. Table 13 shows that this predicate is tied to the scene origin, to the point that translating ground-truth layouts by 0.5 m drops satisfaction from 1.0 to 0. TOSCA explicitly aligns layouts to that origin, and it is not clear whether the baselines received the same alignment. Part of the large Sym. gain, and therefore of the Macro gain, may reflect this frame alignment rather than better generation.
-
The FID improvement is not clearly attributable to the main contribution, and the "certified" framing claims more than is delivered. According to Table 20, the transport-set-only configuration has FID 36.7, close to EchoLayout's 38.4. The drop to 25.1 comes from orientation projection (a per-category frame choice with a quadrant classifier) and category-specific size sets, which are not part of the transport-set or residual-conditioning idea. The baselines do not have these components. It is also unclear why final refinement improves TOSCA's FID (38.2 to 25.1) but makes EchoScene's FID worse.
On the theory side, the guarantees cover only fixed affine convex subproblems. They rely on standard tools (Farkas' lemma, Bellman-Ford negative cycles, primal-dual bounds for squared-distance objectives, decay of sparse SPD inverses), and Theorem 1 is close to immediate. The certificates do not cover size-dependent bounds, branch choices, or collisions. In practice the pipeline's conflict detection misses some cases (bedroom TPR 0.914), even though exact negative-cycle checking is complete for these synthetic conflicts. Finally, the paper is very dense and heavily hedged, which made it hard to identify the actual contribution of each component.
Questions for authors
- Could you report the full relation-satisfaction table (Micro/Macro/Joint and per-relation) for the baselines after applying the same final refinement, and with the same reference-frame alignment? This is the main factor in my assessment.
- Table 20 suggests that most of the FID gain comes from orientation projection and size sets. Could these components be added to a baseline for a fairer comparison? Also, why does refinement improve TOSCA's FID but worsen EchoScene's?
- Why does the complete pipeline miss some injected conflicts (bedroom TPR 0.914) when exact negative-cycle checking is complete for these inputs?
I am very willing to raise my score if all the questions are answered adequately.