Learning the Essentials: Clusterable Representations via Locally Gaussian Deep Embedding

Clustering — the grouping of similar objects in an unsupervised fashion — is a foundational task in machine learning. Deep learning methods have shown promise in this regard, as they can learn information-rich representations from complex data. However, all current deep learning-based clustering methods suffer from weaknesses in at least one key aspect: they require input parameters inaccessible to the user, their internal representations lack interpretability, or the resulting clustering cannot be guaranteed to truly correlate with the input data. To overcome these limitations, we introduce a novel method called Locally Gaussian Deep Embedding (LGDE). LGDE is an autoencoder featuring an embedding space that can be meaningfully clustered without requiring the number of clusters as an input. We derive a new loss function, mSIGReg — a variant of the SIGReg (Sketched Isotropic Gaussian Regularization) method — and demonstrate that its optimization yields well separated and easily identifiable clusters. Additionally, we show how compactness constraints can be incorporated, resulting in clustering that is highly descriptive of the input data. We demonstrate that LGDE retains its effectiveness in generating informative and clusterable representations across multiple data types ranging from established image datasets such as MNIST and EMNIST to structural biology data on signaling proteins relevant to drug design.