When Do Relations Reveal Factors: Learning Identifiable Representations from Typed Relations

Disentangled representation learning seeks to recover the key factors of variation in data, providing representations that can be more interpretable, robust, and generalizable. Observational data alone generally do not uniquely determine the underlying factors, since different nonlinear mixtures can explain the same observations equally well, leading to an identifiability problem. To guarantee identifiability, additional information beyond observations themselves is therefore needed. In this work, we derive this information from *typed relations* between observations, with examples in healthcare including kinship ties, patient similarity, and, in social networks, various forms of social connections. We develop an identifiability theory that characterizes when and to what extent such relational information can help identify the underlying factors. Specifically, we define for each factor its *relation signature*, i.e., the set of typed relations affecting this factor, and our main results show that, under a common sparsity condition, the partial-order structure between relation signatures can determine to what extent each factor can be identified. For different partial-order structures, the theory produces a series of identifiability results, ranging from no guarantee, to directional ambiguity, further to identifiability of blocks of factors, and ultimately to factor-wise identifiability. Guided by this theory, we develop a variational method that jointly learns representations and relation-specific activity masks under a sparsity constraint. Experiments on controlled settings show that the learned factors can identify the true factors exactly in a way that our theory predicts. We also test our method on real image benchmarks, demonstrating that theoretical results hold even when conditions are not strictly satisfied and that our method consistently learns better-disentangled representations than baselines.