Hessian Rank Constraint for Learning Structure of Nonlinear Latent Variable Models

Uncovering latent variables and their causal relations from observed data is a fundamental yet challenging problem. Existing methods often rely on restrictive assumptions such as linear relations or invertible mixing functions. To better address this problem under general nonlinear mixing procedures, we propose a condition called Hessian Rank Constraint (HRC) as a primitive rank-based tool for nonlinear latent causal discovery. In particular, we show that, interestingly, a particular rank-based property arises from the cross-Hessian of the observed-data log-density in the nonlinear case that reveals information about the latent variables and is reduced to the Tetrad constraints in the linear Gaussian case. When two groups of observed variables are d-separated by a set of lower-dimensional latent variables, the rank of this cross-Hessian is equal to the latent dimension, under a mild local linearity assumption on the conditional log-density derivatives. This assumption can be naturally satisfied when the noise level is low or the relevant nonlinearity is not very strong. Based on HRC, we identify the locations of latent variables and recover causal relations among latent variables up to the Markov equivalence class by using observed variables as surrogates. Experimental results on synthetic and real-world datasets support the theoretical claims.