- Rating
- 3
- Confidence
- 2
- Informativeness
- 3(+1)
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes HIVE, a hierarchy-inducing visual backbone that recursively organizes image patches into fine-, mid-, and coarse-level feature embeddings. The model combines feature binding, graph-based relational reasoning, learned hierarchical pooling, and multi-level projection, while preserving image-conditioned assignments between adjacent hierarchy levels. These assignments make the learned cross-level composition directly accessible for analysis. The paper evaluates HIVE on ImageNet classification, unsupervised part discovery on PartImageNet, and semantic segmentation on ADE20K, and further analyzes class separability and hierarchical mediation of the learned feature states.
Critical strengths
-
The empirical gains are substantial and are supported by useful controlled comparisons. I found the PartImageNet results particularly convincing. Under the same MPAE pipeline, replacing the raw ViT-S features with HIVE representations leads to sizeable and consistent improvements in both NMI and ARI across datasets and part granularities, without using part annotations for training. The controlled ADE20K comparison also improves the ViT-S baseline under the same decoder and training protocol. These results suggest that HIVE improves the underlying representation rather than merely producing visually appealing hierarchy diagrams.
-
The paper goes beyond task accuracy and provides evidence that the learned hierarchy is actually used by the model. I especially liked the analyses of feature-state organization and hierarchical mediation. Mid- and coarse-level states become substantially more class-aligned than fine-level states, and perturbing class-relevant fine-level features has a much larger effect when the upper hierarchy is recomputed than when it is held fixed. To me, these experiments provide meaningful evidence that the hierarchy participates in computation rather than serving only as an interpretability overlay.
Critical weaknesses
My main reservation is the degree of architectural novelty. Many individual ingredients of HIVE — competitive feature binding, graph attention, differentiable pooling, and multi-level representation aggregation — are established ideas. I understand the paper’s novelty as the assignment-preserving recursive organization that keeps image-conditioned child–parent relations accessible across levels. This is useful and well implemented, but the distinction from earlier multi-scale, grouping, and structured visual representation methods is more evolutionary than fundamental. Consequently, I find the empirical evidence important for establishing the contribution.
Questions for authors
My main question is whether the benefit specifically requires the recursive assignment-preserving hierarchy, rather than a strong but non-hierarchical multi-level feature aggregation mechanism. The current module and hierarchy-level ablations are useful, but I would be interested in a capacity-matched baseline that retains comparable numbers of feature states and multi-level projections while removing or breaking the recursive child–parent assignments. Such a comparison would help isolate the contribution of the explicit hierarchical organization from generic multi-scale representation enrichment.
As a clarification rather than a criticism, I also appreciate that the paper explicitly states that its model-internal hierarchy need not coincide exactly with human-defined semantic parts. I think this is an appropriate scope for the current evidence.
3 comments
Thanks for the thoughtful rebuttal and the additional experiments!
I particularly appreciate the recursive-vs-parallel comparison, which directly addresses my main concern. The new COCO and SPIN results are also helpful and make the hierarchical representation analysis more informative.
I still see the architectural novelty as relatively incremental, so I'll keep my overall rating at 3. However, the additional evidence has improved my understanding of the method and its contributions, and I'm happy to raise my Informativeness score from 2 to 3.
Thanks again for the careful revisions!
We sincerely thank you for revisiting our rebuttal and revised manuscript. We particularly appreciate your recognition that our recursive-versus-parallel comparison directly addresses your main concern, as well as your positive assessment of the additional MS-COCO and SPIN experiments.
We are grateful for your thoughtful feedback, acknowledgment of HIVE's contributions, and decision to raise your Informativeness score.
As the discussion period draws to a close, we sincerely thank you again for your constructive and supportive assessment. We hope that our capacity-matched comparison between recursive HIVE and parallel multilevel grouping, together with the additional MS-COCO experiments, has further strengthened the empirical evidence for HIVE's distinctive contributions.
We would greatly appreciate your feedback on whether any concerns remain. If our responses and additional results have been satisfactory, we would be grateful for your consideration of updating your evaluation accordingly.