Hierarchical Symbol Coupling for Generative Modeling and Symbolic Inference

Hierarchical latent variable models provide a promising framework for representing data at multiple levels of abstraction. Their priors, however, typically use Gaussian conditionals, which model dependence across layers while leaving latent units within each layer conditionally independent. Moreover, lower layers can absorb most of the data variation, leaving upper layers weakly informative for semantic abstraction. We propose HiSC-EBM, a hierarchical symbol-coupling energy-based prior. Layer-wise energy terms provide informative corrections to the Gaussian hierarchy, while discrete symbols, such as class labels and attributes, are coupled to selected upper layers and the remaining layers remain available to model residual variation. The resulting joint prior therefore provides a unified formulation in which marginalizing the symbols supports generation, inferring them enables prediction, and conditioning on them allows controllable generation. Because unobserved symbols can be marginalized in closed form, the same formulation accommodates fully labeled, partially labeled, and unlabeled data. The generator, prior, and inference model are learned jointly from scratch within a variational framework. Various experiments and analyses demonstrate the effectiveness of the proposed method.