Hierarchical Prototype Learning for 3D Scene Graph Generation from RGB-D Sequences

3D Scene Graph Generation (3DSGG) aims to construct structured scene representations by recognizing object nodes and predicting relational edges from 3D environments. Existing RGB-D sequence-based methods have made promising progress by reconstructing 3D geometry and enhancing node and edge features with multi-modal inputs. However, they still struggle to address the severe inter-class similarity and intra-class diversity of both objects and relations in complex 3D scenes. In this work, we propose HPSSG, a hierarchical prototype learning framework for RGB-D sequence-based 3DSGG, inspired by the hierarchical perception process of human scene understanding. Specifically, semantic-level prototypes serve as stable semantic anchors to improve the separability between similar classes, while mode-level prototypes capture multiple latent patterns within each category to model diverse object instances and relation configurations. To maintain hierarchical consistency, we further introduce hierachy-aware prototype entailment learning, which constrains mode-level prototypes with their corresponding category semantics. Moreover, prior-guided compositional prototype learning uses subject-object compositional priors to select and refine mode-level edge prototypes, improving the structural consistency of relation prediction. Experiments on the 3DSSG dataset demonstrate that our method outperforms various leading methods without any additional inference cost. Our code will be released.