- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper studies how to explain the predictions of hyperbolic neural networks using layer-wise relevance propagation, which assigns relevance scores to input features. The main difficulty in explaining hyperbolic neural networks is that hyperbolic operations are not isolated coordinate-wise, and different models of hyperbolic spaces can result in different explanations despite making the same predictions. To address this, the paper proposes LRP-radial-all, which separates hyperbolic operations at the origin (e.g. exp and log maps) into a geometric scaling component and an input feature component, and assigns all relevance to the input features instead of the geometric scaling factors. They introduce two criteria, geometric representation invariance and zero-curvature consistency. They then theoretically show that their method satisfies both while conserving relevance, which would not be the case if relevance are also assigned to the geometric scaling factor. They evaluate the method on a hyperbolic MLP for MNIST, a hyperbolic EEGNet variant for sEEG, and a Lorentz CNN for CIFAR-10, using feature removal and retention tests to measure attribution fidelity. The evaluation results for LRP-radial-all is largely mixed. While it achieves better average in some datasets/metrics than the baseline in Table 1, LRP-radial-all underperforms in 2 out of the 6 cases, and the improvements are also usually within confidence intervals (F+ and F- for MNIST, F+ for sEEG and CIFAR-10). In terms of runtime, LRP-radial-all is comparable to Gradient×Input and significantly faster than Integrated Gradients.
Critical strengths
-
The paper identifies an important issue with explaining hyperbolic neural networks, which is that equivalent geometric representations can produce different relevance scores despite making the same predictions. The authors formalize this through geometric representation invariance and zero-curvature consistency, and provide theoretical results showing that their proposed method satisfies both for exponential and logarithmic maps at the origin. These results provides theoretical motivation to the formulation of the method that are specific to hyperbolic geometry.
-
The authors evaluate their method across three different datasets and hyperbolic architectures, in both image and time-series classification tasks. While the results are mixed, the method achieves competitive performance and runs substantially faster than Integrated Gradients.
Critical weaknesses
- The biggest weakness is that the proposed framework focuses mainly on exponential and logarithmic maps at the origin, but many recent works have proposed fully hyperbolic operations or more complicated operations that do not rely on exp/log maps at the origin. For example, for the Lorentz space, recent works have proposed fully hyperbolic linear layers [a], activation layers [b], residual connections [c], and MLR layers [b]; for the Poincare Ball space, recent works have proposed fully connected layers and MLR layers [d]. It is unclear whether the paper's proposed criteria, such as GRI and zero-curvature consistency, extend to these operations. In fact, it seems to me that these operations do not necessarily converge to their Euclidean counterparts as the curvature approaches zero under the paper's formulation. As recent hyperbolic works have attempted to move away from repeated mappings to and from the tangent space, it is important to consider whether the proposed explainability framework applies to these newer hyperbolic operations. Additionally, I think empirical experiments should also include model architectures that use these operations, such as fully hyperbolic MLR layers for image classification [b], on top of operations on the tangent space. Finally, since the paper's main theoretical guarantees focus on radial operations at the origin, but uses operations that do not rely on exp/log maps in the experiments (e.g. Lorentz linear layers in the CNN experiments), The paper should better justify how it treats different geometric quantities during relevance propagation. For example, the authors assign no relevance to the norm-dependent geometric scaling factors in exp/log maps, but in their Lorentz CNN, they allow relevance to pass through the norm-dependent time coordinate and redistribute it to the spatial features. It's unclear what motivates the discrepancy (why should the time-component receive relevance and are GRI and zero-curvature consistency still satisfied in this case?)
[a] Hypformer: Exploring Efficient Transformer Fully in Hyperbolic Space, Menglin Yang, Harshit Verma, Delvin Ce Zhang, Jiahong Liu, Irwin King, Rex Ying
[b] Fully Hyperbolic Convolutional Neural Networks for Computer Vision, Ahmad Bdeir, Kristian Schwethelm, Niels Landwehr
[c] Lorentzian Residual Neural Networks, Neil He, Menglin Yang, Rex Ying
[d] Hyperbolic Neural Networks++, Ryohei Shimizu, Yusuke Mukuta, Tatsuya Harada
- The experimental results show very marginal improvements over existing methods, where all but 1 case show either improvements within confidence intervals or worse performance for LRP-radial-all. These results do not demonstrate that LRP-radial-all gives better explanation than the simple Euclidean counterparts. On CIFAR-10, the proposed method even performs worse than the random baseline on F+. Additionally, the paper only validates GRI through a simple controlled example using equivalent logarithmic map implementations, instead of on full trained hyperbolic networks. Given that geometric consistency is one of the main motivations of the formulation behind LRP-radial-all, demonstrating whether LRP-radial-all actually satisfies GRI on the actual models tested in the experiments is important empirical validation. Currently, stronger empirical evidence is needed to show that these theoretical properties lead to better explanations in practice.
Questions for authors
See weaknesses