acceptodds
Under review as a conference paper at ICLR 2027

On the Geometry of Hierarchical Concept Spaces in Vision Models: Testing Linear and Hyperbolic Representations

Abstract

Semantic concepts provide a human-interpretable basis for understanding the representations learned by vision models. Under the Linear Representation Hypothesis (LRH), concepts are modeled as linear directions in a flat embedding space, where they can be composed and translated relative to one another. Meanwhile, prior work has shown that semantic concepts are often organized hierarchically, which raises a fundamental question: is such hierarchical organization geometrically compatible with LRH? We introduce *residual reconstruction*, a geometry-agnostic framework that formalizes geometric compatibility as a quantitative measure, allowing us to assess how well a given geometry captures the structure of hierarchical concepts. We apply it to the flat linear geometry underlying LRH, evaluating the compatibility across four distinct vision models on ImageNet and its associated WordNet hierarchy. Our results show that hierarchical concepts exhibit substantially larger residual reconstruction error, indicating a systematic mismatch between hierarchical organization and the flat additive geometry underlying the LRH formulation. We then investigate whether hyperbolic geometry better captures the structure of hierarchical concepts. Across the same models and hierarchies, hyperbolic representations yield substantially lower residual reconstruction error, providing empirical evidence that the hierarchical concept structures are better captured by hyperbolic geometry than by the corresponding flat geometry. Our work provides a general quantitative framework for studying the geometric compatibility of semantic structures and reveals a systematic tension between hierarchical concepts and linear representations in vision models.

Primary area:
interpretability and explainable AI
Keywords:
Interpretability, Linear Representation Hypothesis, Hierarchical Concepts

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_zRSqOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether hierarchical semantic concepts are geometrically compatible with the flat additive structure assumed by the Linear Representation Hypothesis. The authors introduce residual reconstruction, a geometry-agnostic framework in which semantic nodes are represented as points and parent–child refinements as local concept vectors. Hierarchical concepts are composed along ancestor–descendant paths, and the reconstructed endpoint is compared with the observed descendant representation. In the Euclidean instantiation, this reduces to vector addition; the paper then compares this flat realization with a Poincaré-ball realization. Experiments on ImageNet/WordNet across four frozen vision backbones show substantial residual error for flat additive composition and lower reconstruction error for the tested Poincaré realization.

Critical strengths
  1. The residual-reconstruction formulation is a clean and useful way to test hierarchical concept composition. I like that the paper moves beyond asking whether individual concept directions exist and instead asks whether local parent–child refinements compose consistently along an entire hierarchy. The framework also makes different candidate geometries comparable through a common reconstruction principle.

  2. The empirical validation of the hierarchical concept directions is careful. The authors verify that the extracted directions generalize to held-out data and induce the expected model-level changes under intervention, with random and semantically mismatched controls. Multi-edge paths are also excluded from fitting and model selection in the Poincaré experiment. These choices make the residual-reconstruction results substantially more convincing.

Critical weaknesses

The central flat-versus-Poincaré comparison is not fully capacity-matched. The flat baseline directly composes the estimated concept vectors through addition, whereas the Poincaré realization additionally learns an encoder, decoder, and curvature, together with chart-fitting constraints. Therefore, the current experiments establish that the proposed learned Poincaré realization reconstructs the hierarchy better than the tested flat additive realization, but they do not fully isolate whether the improvement comes from negative curvature itself or from the additional flexibility of the learned chart. This distinction is important because the geometric conclusion is one of the paper’s central claims.

Questions for authors

Could the authors include a capacity-matched learned Euclidean baseline? In particular, I would be interested in a realization that uses the same or closely matched encoder/decoder architecture and fitting objectives as the Poincaré model, but with zero curvature / Euclidean geometry. If the Poincaré realization still provides a clear advantage under such a comparison, this would substantially strengthen my confidence that the observed gain is genuinely geometric rather than primarily due to the additional flexibility of the learned chart.

Presentation suggestion (not central to my score): I would encourage the authors to slow down the introduction and provide a more intuitive running example of hierarchical path composition before introducing the full geometric machinery. I also think the main figures, especially Figure 1, could be redesigned to communicate the geometric intuition more directly.

1 comment

W1: We have conducted additional experiments as necessary. Please refer to our response to Q1 below.

Q1: The Poincaré realization does introduce additional learnable degrees of freedom. However, we deliberately constrain this flexibility through three loss terms: λtan\lambda_{\mathrm{tan}}, λpt\lambda_{\mathrm{pt}}, and λmet\lambda_{\mathrm{met}}. The first two encourage the Poincaré realization to faithfully preserve the LDA concepts and population-level representations. The third imposes an even stronger constraint, requiring the hyperbolic latent space to be an isometrically embedded submanifold of the ambient Euclidean representation space. These constraints restrict the encoder from exploring potentially better hyperbolic manifolds MM.

To further isolate the advantages arising from geometric differences, now we train two additional realizations:

  1. A learned Euclidean realization using the same training procedure as the Poincaré realization, with only two modifications: (i) the encoder's underlying geometry is (M,g)=(Rm,⟨⋅,⋅⟩)(M,g)=(\mathbb{R}^m,\langle\cdot,\cdot\rangle); (ii) the nonlinear tanh layer is removed from the decoder, ensuring that the decoder's image remains an affine subspace.

  2. The original Poincaré realization with λmet=0\lambda_{\mathrm{met}}=0 to relax the metric-preservation constraint, allowing the encoder to explore potentially better hyperbolic manifolds.

The results are reported below.

Table 2A (same setting as Table 2).

ModelFlatPoincarélearned EucPoincaré (λmet=0\lambda_{\mathrm{met}}=0)
ConvNeXt-B.75.60.71.48
ViT-B/16.77.52.61.44
DINOv2-B/14.74.56.64.46
CLIP-B/32.82.57.61.46

As shown in Table 2, learned Euclidean realization narrows the gap with Poincaré, but Poincaré retains an advantage, especially without the metric-preservation constraint.

Meanwhile, we acknowledge that the current results do not directly support the broader interpretation that hierarchical concepts exhibit a systematic tension with all forms of flat additive geometry. Our claim specifically concerns Euclidean realizations under the LRH hypothesis, particularly those defined in $\S$2.4, rather than arbitrary flat-additive-geometry realizations. We will therefore refine the wording of our statement to avoid posible misunderstandings and ambiguity.

Lastly, we clarify (and will make explicit in the final version) that the central contribution of this work is the residual reconstruction error introduced in Eq. (4), rather than a categorical rejection of all reconstructions based on Euclidean metrics. That metric provides a general framework for evaluating any geometric realization that satisfies the proposed definition. In the current manuscript, we empirically observe that the Euclidean realization under the conventional LRH framework exhibits larger reconstruction errors than a simple hyperbolic realization.

Q2: We sincerely appreciate your valuable suggestions. We will incorporate them in the final paper by providing a more intuitive example of hierarchical path composition and redesigning Fig. 1 to convey its key ideas more clearly.


We will incorporate all the above clarifications and experiments into the final paper.

Reviewer_3t1FOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates whether hierarchical semantic concepts in pretrained vision models are compatible with the Linear Representation Hypothesis (LRH). The authors introduce a residual reconstruction framework that measures how accurately ancestor-to-descendant representations can be reconstructed by composing hierarchical concept vectors. Using ImageNet-1K and four frozen vision encoders, they report substantial residual errors under Euclidean composition. They furthermore introduce a learned Poincaré-ball representation, achieving lower reconstruction errors across all four backbones, and argue that hyperbolic geometry better captures hierarchical concept structure.

Critical strengths
  1. The problem motivation is meaningful. The relationship between hierarchical semantic organization and the geometry of learned concept representations is an important interpretability problem.
  2. Residual reconstruction provides a common conceptual formulation through exponential maps, parallel transport, and geodesic distances.
Critical weaknesses
  1. There is a mismatch between the experimental settings of Euclidean and the hyperbolic geometry. In the Euclidean space, fixed concepts are used while in the hyperbolic geometry a learned encoder is used. I think the authors should add a learned model in the Euclidean space as well.
  2. The authors should add some experiments and comparisons with existing hyperbolic vision-language models such as MERU [1], ATMG [2], HyCoCLIP [3] and HierarCaps [4] and how they fit into the framework.
  3. The models compared in the paper are relatively outdated. The authors should test the latest vision and vision-language foundation models like DinoV3, Tipsv2, Siglip-2, etc.
  4. The authors should also somehow measure how the quality of embeddings are in isolation rather than only in the hierarchical setting. The authors should add some metrics that directly measure the information capacity of the embeddings and disentangle semantic information from geometric information in the embeddings. If semantic information is alone insufficient, then geometry cannot possibly help in hierarchical composition.
  5. The authors should also add more diverse benchmarks like iNaturalist or HierarCaps to ensure the hypothesis and the proposed framework generalises beyond wordnet.

References

[1] Desai, Karan, et al. "Hyperbolic image-text representations." International Conference on Machine Learning. PMLR, 2023.

[2] Ramasinghe, Sameera, et al. "Accept the modality gap: An exploration in the hyperbolic space." 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024.

[3] Pal, Avik, et al. "Compositional entailment learning for hyperbolic vision-language models." International Conference on Learning Representations. Vol. 2025. 2025.

[4] Alper, Morris, and Hadar Averbuch-Elor. "Emergent visual-semantic hierarchies in image-text representations." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.

Questions for authors
  1. How does the latent dimensionality of Euclidean or Hyperbolic space affect hierarchical compositionally? There should be controlled study of how latent dimensionality plays a role.
2 comments

W1: The Poincaré realization does introduce additional learnable degrees of freedom. However, we deliberately constrain this flexibility through three loss terms: λtan\lambda_{\mathrm{tan}}, λpt\lambda_{\mathrm{pt}}, and λmet\lambda_{\mathrm{met}}. The first two encourage the Poincaré realization to faithfully preserve the LDA concepts and population-level representations. The third imposes an even stronger constraint, requiring the hyperbolic latent space to be an isometrically embedded submanifold of the ambient Euclidean representation space. These constraints restrict the encoder from exploring potentially better hyperbolic manifolds MM.

To further isolate the advantages arising from geometric differences, now we train two additional realizations:

  1. A learned Euclidean realization using the same training procedure as the Poincaré realization, with only two modifications: (i) the encoder's underlying geometry is (M,g)=(Rm,⟨⋅,⋅⟩)(M,g)=(\mathbb{R}^m,\langle\cdot,\cdot\rangle); (ii) the nonlinear tanh layer is removed from the decoder, ensuring that the decoder's image remains an affine subspace.

  2. The original Poincaré realization with λmet=0\lambda_{\mathrm{met}}=0 to relax the metric-preservation constraint, allowing the encoder to explore potentially better hyperbolic manifolds.

The results are reported below.

Table 2A (same setting as Table 2).

ModelFlatPoincarélearned EucPoincaré (λmet=0\lambda_{\mathrm{met}}=0)
ConvNeXt-B.75.60.71.48
ViT-B/16.77.52.61.44
DINOv2-B/14.74.56.64.46
CLIP-B/32.82.57.61.46

As shown in Table 2, learned Euclidean realization narrows the gap with Poincaré, but Poincaré retains an advantage, especially without the metric-preservation constraint.

Meanwhile, we acknowledge that the current results do not directly support the broader interpretation that hierarchical concepts exhibit a systematic tension with all forms of flat additive geometry. Our claim specifically concerns Euclidean realizations under the LRH hypothesis, particularly those defined in $\S$2.4, rather than arbitrary flat-additive-geometry realizations. We will therefore refine the wording of our statement to avoid posible misunderstandings and ambiguity.

Lastly, we clarify (and will make explicit in the final version) that the central contribution of this work is the residual reconstruction error introduced in Eq. (4), rather than a categorical rejection of all reconstructions based on Euclidean metrics. That metric provides a general framework for evaluating any geometric realization that satisfies the proposed definition. In the current manuscript, we empirically observe that the Euclidean realization under the conventional LRH framework exhibits larger reconstruction errors than a simple hyperbolic realization.

W2/W3/W5: We have conducted additional experiments: (1) using the HyCoCLIP image encoder as the backbone, and (2) training and evaluating on the iNaturalist 2021 Mini dataset. Since the original experiments already have DINOv2 and due to time constraints, we did not include other models such as DINOv3.

The results are presented below:

Table 2C (same setting as Table 2).

ModelFlatPoincarélearned EucPoincaré (λmet=0\lambda_{\mathrm{met}}=0)
ConvNeXt.85.82.89.61
ViT.75.53.57.44
DINOv2.68.46.52.39
CLIP.86.58.59.49
HyCoCLIP.95.66.75.63

Table 6B. Semantic validation

BackboneRep. Top-1 ↑\uparrowCAV AUC ↑\uparrowPerm. AUCΔP\Delta P: CAV ↑\uparrowΔP\Delta P: mismatch
ConvNeXt-B.88.94.50+1.92-0.97
ViT-B/16.78.93.50+4.38-1.99
DINOv2-B/14.84.98.50+1.54-0.64
CLIP-B/32.86.93.50+2.23-1.01
HyCoCLIP-B/16.81.90.50+2.48-1.50

Semantic validation following Table 1(a) (node representations), Table 1(b) (conditional concept discrimination), and Fig. 2 (interventions; α=2\alpha=2, ΔP\Delta P in percentage points).

Table 6B provides semantic validation that confirms that both the population-level representations and estimated LDA concepts satisfy their respective definitions. Building on this, Table 2C shows that the main trend remains consistent across different datasets and backbones: Poincaré outperforms both Flat and Learned Euc, with a more pronounced advantage when λmet=0\lambda_{\mathrm{met}}=0. Notably, among the five models, HyCoCLIP exhibits the largest improvement of Poincaré over Flat and learned Euc, suggesting that Poincaré is not only compatible with, but also well suited to, hyperbolic models.

W4: We appreciate this valuable suggestion. We clarify that our framework evaluates the semantic validity of both population-level representations and hierarchical concept vectors before geometric reconstruction. Specifically, Table 1(a) confirms that the population representations retain meaningful semantic information, while Table 1(b) and Figure 2 establish the semantic discrimination and intervention validity of hierarchical concept directions.

With these semantic properties validated, our subsequent experiments ($\S$3.4 and $\S$4) focus on whether the identified concept contributions can be consistently composed under different geometries, rather than reconstructing the entire backbone representation. In particular, the flat evaluation is restricted to the concept-explained subspace ($\S$2.4 and Appendix F), while the Poincaré realization evaluates the intrinsic geometric composition of the same fixed concepts.

We also note that the semantic information retained within the corresponding subspace or manifold is inherently constrained by the number and diversity of identifiable concepts available in the WordNet–ImageNet-1K hierarchy. This constraint further motivated our focus on geometric composition of the identified concept contributions, rather than requiring recovery of the full ambient representation.

Q1: We have now conducted a latent-dimensionality ablation across three learnable realizations: Poincaré, learned Euclidean, and Poincaré without metric matching, with results reported in Figs. 5--6, currently located in Appendix H.

Specifically, Fig. 5 shows that Poincaré consistently achieves lower intrinsic reconstruction errors than learned Euclidean for latent dimensions from 128 to 320 across all four backbones, with the advantage generally increasing at higher dimensions. Fig. 6 further shows that reconstruction errors in the common ambient space are more closely matched, with both geometries generally performing best at intermediate dimensions (128–192).

The flat LRH baseline has no learnable latent dimension, as its concept subspace is determined by the fixed concept directions. Appendix F, Table 3 reports its numerical subspace dimensions.

Overall, these results indicate that latent dimensionality does affect reconstruction performance, but the Poincaré realization retains its advantage in intrinsic geometric composition across a broad range of dimensions.


We will incorporate all the above clarifications and experiments into the final paper.

Reviewer_tQyTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Summary

This paper investigates whether hierarchical concepts in vision models can be explained through linear addition of concept vectors and whether hyperbolic geometry provides a better alternative when such reconstruction is inadequate. Under the authors’ concept construction and evaluation framework, linear addition produces substantial reconstruction errors, while a learned hyperbolic representation performs better. These findings do not, however, establish a general rejection of the Linear Representation Hypothesis (LRH) or demonstrate that the models’ internal representations are intrinsically hyperbolic.

Critical strengths
  1. Reasonable motivation. LRH treats semantic concepts as directions in representation space whose contributions combine additively. The paper investigates how this formulation accommodates hierarchical concepts and introduces residual reconstruction error to measure whether a child representation can be reconstructed from its ancestor and the intervening concept contributions.

  2. An informative investigation of linear reconstruction and a promising alternative. The experiments identify limitations of the tested linear composition scheme and show that a Poincaré-based realization reduces reconstruction errors across four vision backbones. The improvement also persists when evaluated in the original representation space.

  3. Clear organization and a coherent progression from formulation to evaluation. The paper first defines residual reconstruction as a geometric compatibility criterion, then evaluates its Euclidean instantiation and investigates a hyperbolic alternative. The semantic validation of concept directions and supplementary diagnostics help readers assess the empirical results.

Critical weaknesses
  1. A gap remains between the empirical comparison and its broader interpretation. The experiments directly establish that a learned hyperbolic realization outperforms a fixed LDA-based linear composition scheme. The broader interpretation—that hierarchical concepts exhibit a systematic tension with flat additive geometry—requires separating the effects of concept estimation and additional training flexibility from those of geometry itself.

  2. The failure of linear reconstruction may partly reflect the concept estimator. LDA estimates discriminative directions. The authors show that these directions distinguish concepts and influence model predictions, but neither property guarantees that they reconstruct parent–child mean displacements. Appendix C explicitly acknowledges this distinction. The experiments do not yet separate concept-estimation mismatch from geometric mismatch. Consequently, the results establish substantial reconstruction error for the tested LDA-based realization, while the broader geometric interpretation requires evidence from alternative concept estimators.

  3. The hyperbolic realization introduces additional learning capacity and alignment supervision. It learns an encoder, a decoder with a nonlinear component, and curvature, while using direct parent–child relations as fitting constraints. Although multi-edge paths are excluded from training and model selection, the comparison does not fully distinguish benefits from negative curvature from those of the learned mapping and alignment objectives. A learnable Euclidean baseline with matched capacity and training objectives is needed to isolate these factors.

Questions for authors
  1. Would a learned Euclidean realization with matched encoder–decoder capacity, training objectives, and model-selection procedures achieve comparable performance? How much advantage would the Poincaré realization retain, particularly in the common ambient-space evaluation?

  2. How sensitive are the findings to the concept estimator? In particular, would alternative linear concept directions or strengths calibrated to parent–child displacements substantially reduce the flat reconstruction error?

  3. Given the substantial single-edge errors, can the authors analyze how local reconstruction residuals propagate along the evaluated paths? Such an analysis should account for differences in node pairs and normalization across path lengths, rather than treating differences between group-average errors as an additive decomposition.

  4. Can the authors clarify which additional assumptions connect LRH to the near-zero reconstruction criterion in Hypothesis 1, beyond semantic and intervention validity? In light of Appendix C, which conclusions concern the tested concept realization, and which are intended to generalize to flat additive geometry?

4 comments

W1: We share your opinion. To further investigate the potential influence of other factors beyond geometric properties, we conducted additional experiments involving alternative concept estimators and learned Euclidean realizations. Please refer to Q1 and Q2 for detailed results. These experiments show that hyperbolic realizations retain a significant advantage in reconstruction performance within MM.

Meanwhile, we acknowledge that the current results do not directly support the broader interpretation that hierarchical concepts exhibit a systematic tension with all forms of flat additive geometry. Our claim specifically concerns Euclidean realizations under the LRH hypothesis, particularly those defined in $\S$2.4, rather than arbitrary flat-additive-geometry realizations. We will therefore refine the wording of our statement to avoid posible misunderstandings and ambiguity.

Lastly, we clarify (and will make explicit in the final version) that the central contribution of this work is the residual reconstruction error introduced in Eq. (4), rather than a categorical rejection of all reconstructions based on Euclidean metrics. That metric provides a general framework for evaluating any geometric realization that satisfies the proposed definition. In the current manuscript, we empirically observe that the Euclidean realization under the conventional LRH framework exhibits larger reconstruction errors than a simple hyperbolic realization.

Q1/W3: The Poincaré realization does introduce additional learnable degrees of freedom. However, we deliberately constrain this flexibility through three loss terms: λtan\lambda_{\mathrm{tan}}, λpt\lambda_{\mathrm{pt}}, and λmet\lambda_{\mathrm{met}}. The first two encourage the Poincaré realization to faithfully preserve the LDA concepts and population-level representations. The third imposes an even stronger constraint, requiring the hyperbolic latent space to be an isometrically embedded submanifold of the ambient Euclidean representation space. These constraints restrict the encoder from exploring potentially better hyperbolic manifolds MM.

To further isolate the advantages arising from geometric differences, now we train two additional realizations:

  1. A learned Euclidean realization using the same training procedure as the Poincaré realization, with only two modifications: (i) the encoder's underlying geometry is (M,g)=(Rm,⟨⋅,⋅⟩)(M,g)=(\mathbb{R}^m,\langle\cdot,\cdot\rangle); (ii) the nonlinear tanh layer is removed from the decoder, ensuring that the decoder's image remains an affine subspace.

  2. The original Poincaré realization with λmet=0\lambda_{\mathrm{met}}=0 to relax the metric-preservation constraint, allowing the encoder to explore potentially better hyperbolic manifolds.

The results are reported below.

Table 2A (same setting as Table 2).

ModelFlatPoincarélearned EucPoincaré (λmet=0\lambda_{\mathrm{met}}=0)
ConvNeXt-B.75.60.71.48
ViT-B/16.77.52.61.44
DINOv2-B/14.74.56.64.46
CLIP-B/32.82.57.61.46

Table 5A (same setting as Table 5).

ModelFlatPoincarélearned EucPoincaré (λmet=0\lambda_{\mathrm{met}}=0)
ConvNeXt-B.95.91.91.92
ViT-B/16.94.89.89.90
DINOv2-B/14.91.87.87.88
CLIP-B/32.93.91.92.91

Table 2 shows that the learned Euclidean realization narrows the gap with Poincaré, but Poincaré retains an advantage, especially without the metric-preservation constraint.

Table 5 shows that all three alternatives outperform Flat, though the differences are modest. This is expected for two reasons. First, the residual reconstruction error as defined in Eq. (4) focuses on the reconstruction quality of interpretable representation components, while increasing the proportion of such components in the original representation is beyond the scope of this work. Second, both the ambient representation space and the LDA concepts are based on Euclidean metrics, without additional adaptation to hyperbolic geometry. Thus, the modest advantage observed in the original representation space is unsurprising.

Q2/W2: We adopt an alternative concept estimator based on the filter-based CAV formulation studied by [1], fitting a one-vs-rest logistic classifier to each frozen backbone's image features within each parent category and using its normalized weight vector as the corresponding conditional concept direction.

[1] Pahde et al., “Navigating Neural Space: Revisiting Concept Activation Vectors to Overcome Directional Divergence”, ICLR 2025.

The experimental results are presented below.

Table 2B (same setting as Table 2).

ModelFlatPoincarélearned EucPoincaré (λmet=0\lambda_{\mathrm{met}}=0)
ConvNeXt-B.41.31.31.27
ViT-B/16.73.54.61.40
DINOv2-B/14.84.67.74.50
CLIP-B/32.73.47.52.38

Table 5B (same setting as Table 5).

ModelFlatPoincarélearned EucPoincaré (λmet=0\lambda_{\mathrm{met}}=0)
ConvNeXt-B.55.52.52.52
ViT-B/16.87.80.80.81
DINOv2-B/14.94.90.90.90
CLIP-B/32.80.78.78.78

Table 6A. Semantic validation

BackboneRep. Top-1 ↑\uparrowCAV AUC ↑\uparrowPerm. AUCΔP\Delta P: CAV ↑\uparrowΔP\Delta P: mismatch
ConvNeXt-B.971.00.49+7.43-5.17
ViT-B/16.91.99.49+9.28-5.62
DINOv2-B/14.91.99.48+7.82-4.77
CLIP-B/32.91.99.49+22.93-13.17

Semantic validation following Table 1(a) (node representations), Table 1(b) (conditional concept discrimination), and Fig. 2 (interventions; α=2\alpha=2, ΔP\Delta P in percentage points).

The results show that the main trend remains consistent across different concept estimators, with no substantial changes in performance.

In principle, concept estimation should not be calibrated to parent–child displacements but rely solely on Definitions 1–3. However, we recognize that this bears certain theoretical curiosity, as the reviewer also expressed; therefore, we introduce parent–child displacement supervision during the above training, which can be viewed as limited calibration of the estimated concepts. However, even with this additional flexibility, Table 2A in Q1 and Table 2B in Q2 both show that the Poincaré realization still achieves lower reconstruction errors.

Q3: We analyzed residual propagation along individual paths using the fixed LDA contributions and projected population points from Fig. 3. For P:w0→⋯→wLP:w_0\to\cdots\to w_L, define

δi=ci∣i−1−(pi−pi−1),DP=∥pL−p0∥2.\delta_i=c_{i|i-1}-(p_i-p_{i-1}),\qquad D_P=\|p_L-p_0\|_2.

The path residual is exactly ∑iδi\sum_i\delta_i, giving

Erec2(P)=∑i∥δi∥22dP2+2∑i<j⟨δi,δj⟩dP2\mathcal{E}_{\mathrm{rec}}^2(P) =\frac{\sum_i\lVert\delta_i\rVert_2^2}{d_P^2}+\frac{2\sum_{i<j}\langle\delta_i,\delta_j\rangle}{d_P^2}.

We calculate both terms separately for each path before averaging over the 214 multi-edge paths. This accounts for each path's node pair and normalization directly.

Detailed results are presented in Fig. 4, currently located in Appendix H of the updated paper. Panel (b) shows that local residual energy dominates the aggregate squared error. Cross-edge interactions yield modest reinforcement for ConvNeXt-B and ViT-B/16, and partial cancellation for DINOv2-B/14 and CLIP-B/32, changing the local residual energy by approximately +1.3%+1.3\%, +0.4%+0.4\%, −1.2%-1.2\%, and −4.6%-4.6\%, respectively.

To examine accumulation within fixed paths, Panel (a) plots the mean of AP,j=∥∑i=1jδi∥2DPA_{P,j}=\frac{\left\|\sum_{i=1}^{j}\delta_i\right\|_2}{D_P} separately over 142 two-edge, 50 three-edge, and 22 four-edge paths. Within each group, the same paths and their complete-path denominators are retained at every composition step. The mean normalized accumulated residual increases across composition steps for all four backbones. Panel (c) additionally displays residual and displacement norms separately across path lengths.

Together, these results show that substantial local reconstruction discrepancies persist through composition, with limited net cancellation, indicating that reconstruction errors accumulate along the path rather than being corrected.

Q4: We clarify that Hypothesis 1 is a testable compatibility hypothesis under the flat additive LRH formulation, rather than a direct consequence of semantic and intervention validity alone. Beyond these validity conditions, it requires that hierarchical concept contributions consistently approximate the corresponding population-level refinements.

As derived in Appendix C (Eqs. 17–23), we explicitly characterize this requirement through the local discrepancy

δi=ci∣i−1−(pi−pi−1),\delta_i=c_{i|i-1}-(p_i-p_{i-1}),

which yields the normalized reconstruction error

Erec(r,t)=∥∑i=q+1kδi∥2∥pt−pr∥2\mathcal{E}_{\mathrm{rec}}(r,t)=\frac{\lVert\sum_{i=q+1}^{k}\delta_i\rVert_2}{\lVert p_t-p_r\rVert_2}.

Hypothesis 1 predicts that this accumulated residual should be minimal if the hierarchical concept contributions are compatible with flat additive composition. But importantly, semantic and intervention validity alone do not guarantee this condition; testing whether it holds is precisely the purpose of our work.

Regarding generalization, as clarified in W1, our findings provide empirical evidence of limited compatibility for the tested LRH-based Euclidean realizations, rather than establishing a universal incompatibility between hierarchical concepts and arbitrary flat additive geometries. We will state the scope of our conclusions more explicitly in the revised manuscript.


We will incorporate all the above clarifications and experiments into the final paper.

Thank you for the additional experiments. Q3 and Q4 are resolved. However, the new results weaken the geometric conclusion.

  • W1: The revised manuscript still claims a “systematic mismatch” in the abstract and an ambient-space advantage for Poincaré in Section 4 (L460–461) and the Conclusion (L478–479). None of the new controls appear in the PDF.

  • W2: By the rebuttal’s own decomposition, the flat residual is exactly Σδᵢ. Since mean displacements telescope exactly in Euclidean space, the flat error reflects concept-estimation mismatch rather than geometric incompatibility. This is consistent with the large sensitivity to the estimator (.75 with LDA vs. .41 with CAV on ConvNeXt).

  • W3: The manuscript itself notes that intrinsic errors from different geometries are not directly comparable. In the common ambient space, Poincaré and the learned Euclidean realization are tied on all four backbones (Table 5A). The Euclidean control is also not capacity-matched: it has no tanh in the decoder and no λ_met=0 counterpart.

I therefore maintain my original rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.