Disentangling Latent Representations by Groups, not Dimensions, via Group-wise Total Correlation

Many unsupervised representation learning methods enforce statistical independence at the level of individual latent dimensions, typically through total-correlation (TC) penalties or ICA-style priors. Natural factors, however, are often block structured: several latent dimensions may need to co-vary on a constrained manifold while remaining independent of other groups. We introduce Group-wise Independent Latent Disentanglement (GILD), which learns such group-wise independent representations with a group-wise total correlation (GTC) objective in latent space. Experiments on synthetic grouped latents show that component-wise objectives and prior-only grouped models miss the intended group structure, demonstrating the correctness and practicality of GILD. When applied to real-world datasets, GILD further reveals interpretable multi-component groups that fully disentangled training tends to suppress, implying its versatility and effectiveness.