Relational Structure as a Prior for Weakly-Supervised Part Semantic Segmentation

We study weakly-supervised part semantic segmentation using only image-level object categories and the semantic parts present for each object, without part-level spatial supervision. In this setting, the main difficulty is to establish correspondence between fine-grained part concepts and visual regions when several visually similar parts may coexist within the same object. We observe that although their locations are unknown, part concepts are not independent: pretrained part-text representations already encode relations among them. We therefore use this relational structure as a prior for part-level visual-semantic alignment. Our method identifies part-specific visual support through relative competition among the parts present in the same object, forms image-specific visual prototypes, and adapts part representations with a shared orthogonal transformation that preserves their relational geometry. Experiments on PP-116 and ADE-234 show clear improvements under this weak supervision setting, supporting the use of pretrained relational structure for learning fine-grained part correspondence without part-level spatial annotations.