- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces -faithful prototype rules for hierarchical classification. A linear classifier is interpreted as assigning features to class prototypes under a Mahalanobis metric, potentially with class-dependent offsets; -faithfulness requires these offsets to vanish and the squared distances between prototypes to equal the corresponding taxonomic distances. Theorem 3 characterizes such rules through the pseudo-inverse of the Kron-reduced tree Laplacian. Based on this characterization, the proposed method constructs and freezes a faithful classifier head, then trains the encoder with a KL objective toward taxonomy-derived targets so that features approach their designated prototypes. Theorem 4 separately analyzes an MSE objective in the unconstrained features model and identifies class-dependent offsets and dependence on effective feature regularization as obstacles to learning the same geometry without freezing the head. Experiments on CIFAR-100, FGVC-Aircraft, and Stanford Cars compare against related baselines under uniform and CLIP-derived edge weights. The method obtains the lowest TreeDist@5 in all six evaluated dataset–taxonomy settings, with competitive accuracy on uniform taxonomies and some accuracy loss on CLIP taxonomies.
Critical strengths
-
A precise decision-rule formulation of hierarchy-aware classification: The paper separates two requirements that are often conflated: the class prototypes must reproduce taxonomic distances, and the class-dependent offsets in the equivalent distance rule must vanish. Theorem 3 then gives an exact, constructive characterization of such -faithful prototype rules, while the frozen-head objective realizes the desired decision geometry by construction rather than merely encouraging it through a loss. Although the characterization draws on classical multidimensional scaling and effective resistance, this formulation is clean and useful. It provides a verifiable guarantee about the classifier's actual decision rule, not only about its targets or learned features.
-
A substantial analysis of why the same guarantee is difficult with a learned head: Theorem 4 derives the geometry induced by an MSE objective with generic admissible targets and identifies two concrete obstacles: unavoidable class-dependent offsets on asymmetric taxonomies and distortion from an effective feature-regularization strength that is not known before training. The predicted geometries are checked across multiple datasets, architectures, targets, and regularizers in §K. The analysis motivates the frozen-head design and provides evidence that the theoretical phenomena occur in real networks. The main comparative experiments also obtain the lowest TreeDist@5 across all six evaluated taxonomy settings.
Critical weaknesses
The empirical evidence is limited to relatively shallow taxonomies: The evaluation covers three datasets with at most 196 leaf classes and taxonomies of only two or three levels. Bertinetto et al. (2020) and the closely related HAFrame work additionally evaluate on tieredImageNet-H (608 leaves, hierarchy height 12) and iNaturalist2019 (1,010 leaves, hierarchy height 7). Since the central contribution is a decision geometry that represents fine-grained taxonomic relationships exactly, comparisons on these deeper label trees would provide useful evidence that the observed benefits extend beyond shallow groupings. The current results are encouraging, but the empirical support for the method in the deeper-taxonomy regime remains limited.
Questions for authors
- For HXE (CLIP), how are the CLIP-derived non-uniform edge weights incorporated into HXE's edge weights ? Bertinetto et al. (2020) define these weights from node depth, so the adaptation to arbitrary edge weights is unclear.
- The paper's empirical claim concerns reduced mistake severity, but the reported TreeDist@ metrics are averaged over all test examples. Why not also use an error-conditional metric, analogous to the “hierarchical distance of a mistake” in Bertinetto et al. (2020), with LCA height replaced by ? Please clarify how the reported TreeDist@ improvements distinguish less severe mistakes from fewer mistakes or better-ranked alternatives.
3 comments
Thank you for the response and for the additional experiments. Your clarification on how non-uniform weights enter HXE resolves my first question, and adding an error-conditional severity metric resolves my second: it separates mistake severity from accuracy, and I'm satisfied that the reported gains reflect less severe rather than merely fewer mistakes. I'd be glad to see the complete runs in a revision. I will maintain my recommendation.
Thank you for your reply. Unfortunately, we will be unable to provide full experimental results on the deeper taxonomies before the end of the discussion period; training models on these datasets takes much longer than on the shallower trees used in our initial submission. Our preliminary results on both iNaturalist-2019 and tieredImageNet-H show that our method can make better mistakes (as measured by TreeDist|Err) than HAFrame, a strong baseline. We will include full results in the final version of the paper.
Response to Reviewer YFMd
Thank you for your thoughtful review. We respond to each of your concerns below.
Need for deeper trees:
As mentioned in our overall response, we are working on adding two deeper trees (iNaturalist-2019, tieredImageNet-H) to our evaluation. Early results are promising.
How are non-uniform weights integrated in HXE (CLIP)?
This detail should’ve been clearly documented in our original submission, and we regret leaving it out. The HXE loss (equation 4 in [1]) includes a weight λ(Cℓ) for each edge Cℓ→Cℓ+1, which is a function of a node’s height (equation 5 in [1]). We rescaled λ(Cℓ) to wℓ⋅λ(Cℓ), where wℓ is the weight our tree places on the edge. In the uniform trees, wℓ=1 everywhere and we recover the original objective.
Mistake severity metric:
As mentioned in our overall response, we have added TreeDist|Error to Table 1 to disentangle mistake severity from accuracy. Our method maintains strong performance on the new metric.
Please let us know if you have any further concerns, we would be happy to discuss them with you.
[1] Luca Bertinetto, Romain Mueller, Konstantinos Tertikas, Sina Samangooei, and Nicholas A. Lord. Making better mistakes: Leveraging class hierarchies with deep networks. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12503–12512, 2020. Doi: 10.1109/CVPR42600.2020.01252.