Interpretable Interaction Dictionaries for Mixture Recovery in Discrete Data

Unsupervised representation learning typically maps observations into spaces where latent structure becomes easier to recover, but the resulting coordinates are often difficult to relate to observable properties of the data distribution. We study an alternative for discrete data in which representation coordinates are explicit binary interaction moments. Rather than seeking only a set of statistics sufficient for a downstream recovery task, we aim to construct a structured and human- interpretable representation whose higher-order coordinates remain traceable to simpler dependencies in the data. We instantiate this principle with ODIN, which uses an information-geometric criterion to select a nested hierarchy of interaction atoms, and apply the resulting representation to mixtures of product-Bernoulli distributions. Mixture parameters are estimated by constrained moment matching in the learned moment space. For any fixed dictionary, we show that full column rank of the lifted-moment Jacobian provides a local identifiability certificate, and that enriching the dictionary cannot decrease its rank. This separates the statistical role of dictionary construction from the geometric conditions governing recovery. We evaluate how the learned interaction representation affects parameter recovery and Jacobian geometry on synthetic mixtures, and assess its usefulness on real- world binary clustering benchmarks