Hierarchical Dictionary Learning Tree for Interpretable Concept Discovery

Standard dictionary learning finds a single set of atoms to sparsely represent all data, conflating coarse and fine structure. We present Hierarchical Dictionary Learning Tree (HDLT), a framework that organizes learned sparse dictionaries into a tree whose structure is grounded in Geometric Multi-Resolution Analysis. The key architectural choice, codes as signals, passes the sparse code at each node as the input to its children, so each level learns to represent the activation patterns of the level below. Dictionaries are trained end-to-end by backpropagating through the sparse coding optimality conditions, with an optional classification loss that aligns atoms with class boundaries. Our central result is unsupervised concept discovery: trained on CLIP image features with no labels, text, or ontology, HDLT recovers a WordNet-consistent concept hierarchy from a $30$-class ImageNet subset ($0.98$ leaf purity, cophenetic $\rho =0.92$) so that a single image's route traces its WordNet path (animal$\to$carnivore$\to$big cat$\to$tiger). Under an optional task loss, leaf codes approach a raw-CLIP linear-probe baseline on CIFAR-100 and ImageNet while exposing a tree-shaped representation that flat dictionary methods cannot provide.