acceptodds
Withdrawn from ICLR 2027

Hierarchical Dictionary Learning Tree for Interpretable Concept Discovery

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Standard dictionary learning finds a single set of atoms to sparsely represent all data, conflating coarse and fine structure. We present Hierarchical Dictionary Learning Tree (HDLT), a framework that organizes learned sparse dictionaries into a tree whose structure is grounded in Geometric Multi-Resolution Analysis. The key architectural choice, codes as signals, passes the sparse code at each node as the input to its children, so each level learns to represent the activation patterns of the level below. Dictionaries are trained end-to-end by backpropagating through the sparse coding optimality conditions, with an optional classification loss that aligns atoms with class boundaries. Our central result is unsupervised concept discovery: trained on CLIP image features with no labels, text, or ontology, HDLT recovers a WordNet-consistent concept hierarchy from a -class ImageNet subset ( leaf purity, cophenetic ) so that a single image's route traces its WordNet path (animalcarnivorebig cattiger). Under an optional task loss, leaf codes approach a raw-CLIP linear-probe baseline on CIFAR-100 and ImageNet while exposing a tree-shaped representation that flat dictionary methods cannot provide.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
dictionary learning, concept discovery

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_SHn7OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Hierarchical Dictionary Learning Tree (HDLT), which organizes sparse dictionaries into a tree based on Geometric Multi-Resolution Analysis (GMRA). The method passes sparse codes between tree levels and trains the dictionaries through differentiable sparse coding, with an optional classification loss. Experiments on CLIP image features show a WordNet-consistent concept hierarchy on a 30-class ImageNet subset. The authors also evaluate classification performance on CIFAR-100 and ImageNet.

Critical strengths
  1. The paper explores an interesting approach to hierarchical dictionary learning. Passing sparse codes between tree levels allows the model to represent patterns of atom activation rather than repeatedly processing the original features.
  2. The authors attempt to evaluate concept discovery using quantitative metrics, including leaf purity, cophenetic correlation, and atom–concept alignment.
  3. The experiments include comparisons between end-to-end and greedy training, as well as different ways of passing information between tree levels. These ablations are relevant for understanding the proposed design.
Critical weaknesses
  1. The main contribution is not clearly separated from the existing GMRA framework. The method starts with a tree constructed from the geometry of CLIP features, which may already capture much of the reported concept hierarchy. It is unclear how much the learned dictionaries actually contribute.
  2. The atom–concept alignment results are not convincing. Table 2 shows that the GMRA centroid baseline achieves a cosine similarity of 0.81, compared with only 0.32 for HDLT. The root dictionary alone achieves 0.30, suggesting that the additional learned layers contribute very little to this metric.
  3. The comparison with hierarchical clustering methods is not fully fair. The paper compares HDLT's cophenetic correlation with K-means and Ward using flat cluster assignments, even though Ward naturally produces a hierarchical tree. Using the full clustering hierarchy could substantially change the comparison.
  4. The paper does not include the appendices.
Questions for authors

Please see weakness.

Reviewer_1iRfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes HDLT, a tree of sparse dictionaries built on a GMRA k-means hierarchy, where each node’s sparse code serves as input to its children. Trained on CLIP image features without labels, HDLT recovers a hierarchy aligned with WordNet on a 30-class ImageNet subset. With an optional classification loss, it achieves accuracy close to a raw-CLIP linear probe on CIFAR-100 and outperforms the evaluated K-SVD and HCEP baselines.

Critical strengths

S1. The codes-as-signals design is a simple and interesting idea. Passing sparse codes rather than residuals between levels gives a natural way to build increasingly abstract representations, and the paper’s ablation shows it outperforms residual-passing.

S2. The paper includes informative ablations. It compares end-to-end with greedy training, and the tree with a flat dictionary of matched size, which helps clarify the role of cross-level gradients and the hierarchical structure.

Critical weaknesses

W1. Unclear contribution beyond the GMRA scaffold. The paper does not sufficiently isolate what hierarchical dictionary learning adds beyond CLIP features and GMRA clustering. The root dictionary already achieves nearly the same concept alignment as the full model.

W2. Unequal comparison of hierarchy recovery. Table 1 compares HDLT’s multilevel tree distances with binary same-cluster indicators for K-means and Ward. Comparing against the full Ward hierarchy and the original GMRA tree would better establish HDLT’s advantage.

W3. Limited experimental scale. The main concept-discovery results are evaluated on a small, curated subset of ImageNet. It remains unclear whether these findings generalize to larger and more diverse datasets.

m1. Table 3 reports five-seed results only for HDLT (μ=0.5). Stochastic baselines, particularly K-SVD, and HDLT (μ=0) should also report variability across seeds.

m2. Figure 2 contains two consecutive “animal” labels in the penguin’s HDLT path. The authors should explain this repetition and qualify the claim of rung-by-rung correspondence with WordNet.

Questions for authors

Q1. What additional gains come from learning local dictionaries and training them end-to-end on the same GMRA scaffold? A controlled ablation using hierarchy-recovery metrics and, where applicable, atom–concept alignment would clarify each component’s contribution.

Q2. How do the full Ward hierarchy and the original GMRA tree perform under the same tree-distance evaluation as HDLT? Does training change the tree topology, the assignment of samples to nodes, or both?

Q3. Is there evidence that the concept-discovery findings generalize to larger or more diverse hierarchies?

Q4. How sensitive are the Table 3 results for K-SVD and HDLT (μ=0) to random seeds?

Reviewer_bGLQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes the Hierarchical Dictionary Learning Tree (HDLT) for unsupervised concept hierarchy discovery, identifying coarse-to-fine relationships among concepts from image features without additional class labels, concept text, or a predefined ontology, while providing a tree-structured concept path for each image. The paper argues that conventional flat dictionaries mix features of different granularities within a single set of atoms, making concept hierarchies difficult to express explicitly. Meanwhile, the local linear representations in Geometric Multi-Resolution Analysis (GMRA) primarily serve reconstruction and lack sparsity and task adaptability. HDLT therefore uses a geometric tree constructed through recursive clustering as its scaffold and learns a local sparse dictionary at each node. It passes parent sparse codes as inputs to child nodes, performs soft routing based on the activation strengths of atom groups, and enables end-to-end training through implicit differentiation of the sparse-coding optimality conditions. Experimental results demonstrate the effectiveness of HDLT in unsupervised concept hierarchy discovery and classification.

Critical strengths

1.The paper addresses the discovery of hierarchical relationships among concepts from image features. Compared with flat sparse representations, the tree structure further exposes how concepts are organized and provides a representation path for each image.

2.HDLT integrates geometric trees, local sparse dictionaries, activation-based routing, and end-to-end training. In particular, passing parent sparse codes as inputs to child nodes provides an explicit mechanism for information transfer across levels and allows downstream task losses to update dictionaries at earlier levels.

3.The experiments evaluate the method across multiple dimensions, including hierarchy recovery, atom-concept alignment, and classification performance. The supervised variant approaches the accuracy of a linear probe on the original CLIP features on CIFAR-100, suggesting that the tree-structured sparse representation preserves substantial predictive information in this setting.

Critical weaknesses
  • HDLT relies on a geometric tree constructed through recursive k-means, but the paper lacks a direct comparison between the initial clustering tree and the trained HDLT using the same hierarchy recovery metrics. In Table 2, cluster-centroid differences achieve higher concept alignment scores than HDLT, while the root dictionary alone performs similarly to the full composed dictionary. The current results therefore do not sufficiently establish that semantic hierarchy recovery primarily benefits from cross-level dictionary learning rather than CLIP features and the initial clustering structure.

  • The baseline comparison for hierarchy recovery is not sufficiently fair. Table 1 uses multilevel tree distances for HDLT but computes correlations for K-means and Ward using only a binary same-leaf/different-leaf relationship. Since Ward clustering itself produces a complete hierarchical tree, this setup does not adequately assess HDLT’s advantage over standard hierarchical clustering.

  • The central results are obtained on a selected ImageNet subset containing 30 classes from six super-categories, without sufficient validation across different class selections and multiple random seeds. Moreover, the reported leaf purity of 0.98 is computed at the super-category level and does not directly establish fine-grained class recovery. The description of the tiger leaf in the figures should clearly distinguish super-category purity from class-level purity.

  • A decrease in code support size with depth does not directly demonstrate increasing semantic abstraction. It may also reflect information loss. In addition, the implicit differentiation section introduces a quadratic regularization term that is not defined in the preceding objective, and the optimal-code formula does not explain how coefficient signs are handled.

Questions for authors
  • Please report the correlations with WordNet distances for both the initial recursive k-means tree and the trained HDLT under exactly the same evaluation protocol. Please also describe which sample routes or hierarchical relationships change during training, to clarify the gains attributable to dictionary learning over the initial clustering scaffold.

  • It is recommended to compare against Ward’s complete hierarchical tree using the same distance definition and evaluation procedure as HDLT, rather than evaluating the baseline solely through binary same-leaf/different-leaf relationships. This would more adequately assess HDLT’s advantage over standard hierarchical clustering.

  • Could the authors clarify the relationship between the super-category purity of 0.98 in Table 1 and the 98%-pure tiger leaf in Figure 3? Please report super-category purity and fine-grained class purity separately, and provide hierarchy recovery results across different class subsets and multiple random seeds to assess the stability of the findings.

  • Please provide the complete sparse-coding optimization objective corresponding to Equation (6), explicitly stating whether it includes L2 regularization or coding constraints, and explain how the signs of active coefficients are handled. We also recommend reporting convergence diagnostics for the finite number of FISTA iterations and numerical checks of the implicit gradients.

  • The data split for the concept hierarchy discovery experiments is unclear, particularly whether the clustering tree and dictionaries are constructed using only the training set and whether hierarchy recovery is evaluated on an independent test set. The mapping of the 30 classes to the 16 leaf nodes and the computation of pairwise tree distances between classes also require clarification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.