- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DiScGraph, a discrete diffusion framework for scene graph generation. The method factorizes a scene graph into objects, edge existence, and relation semantics, with the constraint (edge existence to relation). Based on this representation, the authors introduce an edge-aware forward corruption process and a dependency-aware reverse sampler following Node(V)-Edge(E)-Relation(R). Text-conditioned scene graphs are obtained at inference time via CLIP-based reward tilting and Sequential Monte Carlo sampling. The method is evaluated on Visual Genome, COCO, and LAION-SG, with experiments covering unconditional graph generation, graph completion, layout prediction, and downstream text-to-image generation.
Critical strengths
- Scene-graph-specific discrete diffusion formulation. The paper goes beyond directly applying generic graph diffusion by explicitly separating edge existence from relation semantics and incorporating this hierarchy into the diffusion formulation. This is well motivated for sparse scene graphs with long-tailed relation vocabularies, and the ablations indicate that both the factorized state and dependency-aware sampler improve semantic consistency.
- Strong empirical validation. The evaluation is broad and includes both graph-level and downstream image-generation tasks. In particular, under the same SDXL-SG backbone, DiScGraph-generated scene graphs achieve performance close to ground-truth scene graphs and substantially outperform DiGress and DiffuseSG, providing evidence that the learned structural prior is useful beyond graph-distribution metrics.
Critical weaknesses
-
The forward diffusion process appears incomplete for edge reactivation. The edge process can produce $ e_{ij,t-1}=0\rightarrow e_{ij,t}=1$. Since the constraint implies , it is unclear how the active-edge transition $ q_R(r_{ij,t}\mid r_{ij,t-1}=0,e_{ij,t}=1)$ is defined when . This also makes the claimed closed-form relation marginal unclear when an edge becomes inactive and later reactivates. Because the main novelty lies in redefining the discrete diffusion process itself, this issue directly affects the technical soundness of the paper.
-
The claimed factorization is not fully reconciled with the implementation. The method defines $ p_\theta(E_0\mid V_0,x_t)$, but the implementation appears to predict object and edge logits from the same Graph Transformer representation, while explicit conditioning is only described for the relation head through edge logits. It is therefore unclear whether the implementation truly realizes the proposed sequential probabilistic dependency or a weaker shared-representation factorization. In addition, the large effect of relation-class reweighting makes it difficult to isolate how much of the long-tail improvement comes specifically from the proposed diffusion formulation.
Mathematical soundness hole should be solved.
Questions for authors
Please refer to the critical weaknesses above
2 comments
Response to reviewer QcuW (2/2)
[W2]
(a.) factorization:
Thanks for catching this. We apologize for this error in the architectural description. We have corrected the equations and clarified the conditioning in Appendix A.4.2 (revisions colored in blue).
Particularly, the prediction heads are explicitly coupled in the order (). Object states are passed to the edge head, and the resulting edge states are passed to the relation head.
Therefore, DiScGraph implements hierarchical conditioning at both the prediction-head and reverse-sampling levels, rather than relying solely on shared Graph Transformer representations.
(b.) Class Weighting:
We agree that relation-class reweighting improves long-tail relation modeling. However, the improvements cannot be attributed solely to reweighting because DiScGraph's factorized diffusion formulation addresses a key imbalance inherent to scene graphs, sparsity dominates relation semantics. To isolate its contribution from the proposed diffusion formulation, we extend Table 7 with a new Joint + Effective-num ablation. The original flat edge–relation diffusion formulation is retained in joint and effective-num relation weighting is applied in its training. Specifically, positive relation classes use the same weights as DiScGraph, , where denotes the frequency of relation class , and . Since the joint categorical representation also includes the dominant no-edge class (), we assign it a separate weight to limit its influence on the training loss.
| Method | N-MMD () | R-MMD () | ID-MMD () | OD-MMD () | TRIP-TV () | R-K-TV () |
|---|---|---|---|---|---|---|
| Joint (row1) | 0.0077 | 0.0078 | 0.0068 | 0.0062 | 0.6910 | 0.8118 |
| Joint + Effective_num (new) | 0.0076 | 0.0074 | 0.0077 | 0.0079 | 0.6653 | 0.7892 |
| Factorized state (row2) | 0.0066 | 0.0066 | 0.0056 | 0.0056 | 0.5822 | 0.6099 |
| Factorized state + dependency-aware sampler - DiScGraph (row3) | 0.0046 | 0.0058 | 0.0043 | 0.0046 | 0.4878 | 0.5199 |
The results show that reweighting alone does not obtain the benefits of factorized diffusion in joint. Joint improves Rare-K-TV and R-MMD with reweighting, but factorized formulation is better. These improvements arise from two complementary mechanisms. Conventional graph diffusion models edge existence and relation semantics using a single categorical variable , where the overwhelmingly frequent no-edge category dominates the corruption and denoising distributions. DiScGraph instead separates structural sparsity from relation semantics, restricting relation diffusion to on active edges. Class reweighting only modifies the training loss to address frequency imbalance among relation classes , without changing the diffusion dynamics.
Thus, the results support that factorized diffusion addresses structural sparsity to enable relation learning, while relation-class reweighting provides complementary improvements in long-tail learning.
Response to reviewer QcuW (1/2)
[W1]
Thank you for pointing out the missing discussion on forward edge reactivation, we should have expanded on all scenarios. We have revised the manuscript to include the edge reactivation case and the corresponding marginal in Appendix A.1.2 (colored blue).
When an inactive edge becomes active during forward diffusion (eij,t−1=0→eij,t=1), its previous relation information is lost since (rij,t−1=0). However, the newly activated relation can simply be initialized from the prior (πR), i.e., a positive relation is sampled:
qR(rij,t∣rij,t−1=0,eij,t−1=0,eij,t=1)=πR(rij,t). (Eqn 8)
Likewise, the closed-form relation marginal can be derived from the edge state via the edge–relation evolution (Eqn 59), QtER=QtE(et−1,et)qR(rt∣rt−1,et−1,et) in place of relation-only QtR (Eqn 55) defined when eij,t=1.
For example, consider an edge initially labeled person riding horse. During forward diffusion, its state may evolve as (e,r):(1,riding)⟶(0,0)⟶(1,beside). When the edge disappears, its relation is set to zero. Upon reactivation, a new positive relation is sampled from πR, allowing the edge to reappear with a potentially different relation. This entire trajectory is captured by the cumulative transition QtER=∏s=1tQsER which accounts for edge disappearance and reactivation while preserving the constraint (eij,t=0⇒rij,t=0). In particular, the relation marginal for an active edge is obtained by conditioning the coupled E,R marginal on edge existence (Eqn 60):
$q(r_{ij,t}=a|z_{ij,0},e_{ij,t}=1) = \frac{[\overline Q_t^{ER}]{z_{ij,0},(1,a)}} {[\overline Q_t^E]{e_{ij,0},1}}, \qquad a\in\mathcal R^+.\qquad z = (e,r)$.
The object and edge corruption kernels remain unchanged. Thus, the edge reappearance is handled in DiScGraph’s factorized diffusion formulation and maintains its key advantages: decoupling structural sparsity from relation semantics, explicitly modeling object, edge, and relation dependencies in the forward and reverse processes, and preserving valid scene-graph states throughout diffusion.