acceptodds
Withdrawn from ICLR 2027

Spectral-TCAV for Pairwise Concept-Based Interpretability in Raman Classifiers

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Testing with Concept Activation Vectors (TCAV) interprets neural network predictions in terms of human-defined concepts, but has not been applied to Raman spectroscopy, where interpretability is usually limited to individual spectral bands or perturbation responses. We introduce Spectral-TCAV, adapting TCAV to Raman spectra for bacterial species classification, and show that it recovers spectrally meaningful concepts underlying model predictions. We further extend the framework from single spectral regions to pairs of regions, decomposing their interaction into three contrasts: Sum (joint magnitude), Diff (relative dominance), and Tradeoff (antagonistic behavior, isolated by masking for opposite-sign changes). Each contrast is binarized and evaluated with independent linear CAVs and TCAV directional-derivative scores tested against null distributions from randomly sampled regions. Across classes, the same region pair can diverge in significance and direction across contrasts, showing that Sum, Diff, and Tradeoff capture complementary information beyond a single concordant-versus-discordant label.

Primary area:
interpretability and explainable AI
Keywords:
interpretation, explainable AI, concepts, chemistry, interpretability, TCAV, learned representations

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_h7aeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper adapts TCAV to Raman spectroscopy by defining concepts as named spectral bands (user-specified wavenumber ranges) instead of example sets, and extends the framework from single bands to band pairs through three contrasts (Sum, Diff, Tradeoff). It applies this to a 1D CNN classifying six bacterial species from a public Raman dataset, reporting concept-by-class and pair-by-class TCAV score matrices together with a small perturbation-based check. The adaptation is sensible and the authors are candid about several limitations.

Critical strengths
  1. Misspecified significance test in Eq. (7). The significance test in Eq. (7) appears to be misspecified, and every frequency-based conclusion in the paper rests on it. As written, t = (TCAV_Q^{C,y,l} − μ_null) / (σ_null/√N′), with N′ ≤ N = 20, which is the form of a one-sample t-test applied to a mean of N′ observations. Each concept, however, produces a single TCAV score: the CAV is the average of the split weight vectors (5 splits for the named band, 3 for the control regions), and the score is then computed once from it. The numerator is therefore a single draw, not the mean of N′ replications of the null procedure.
Critical weaknesses

As written, t = (TCAV_Q^{C,y,l} − μ_null) / (σ_null/√N′), with N′ ≤ N = 20, which is the form of a one-sample t-test applied to a mean of N′ observations.

Questions for authors
  1. The control regions are described as 5–29 wavenumber indices wide, whereas some concepts are narrower than that (e.g. phenylalanine, 1000–1005 cm⁻¹). Were control regions matched in width to the concept being tested, and could part of the observed separation simply reflect region width?
  2. What does the Tradeoff contrast tell us that Diff does not, when both are evaluated on the same opposite-sign subset? 3.Several smaller issues concern the internal consistency and completeness of the manuscript. Section A.3 is an empty heading; the concept table is cited as "Table 1a" although it is labelled as panel (a) of Figure 1, whose panel (b) has no caption.
Reviewer_1VayOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper introduces Spectral-TCAV, an adaptation of TCAV for interpreting Raman spectral classifiers using biochemically meaningful spectral regions as concepts. It further extends the framework to pairs of spectral regions through three contrasts: Sum, capturing joint magnitude; Diff, capturing relative dominance; and Tradeoff, capturing opposing changes between regions. The method is evaluated on a CNN classifying six bacterial species, with single- and pairwise TCAV analyses across eight predefined spectral concepts. The authors report that the three pairwise contrasts reveal distinct patterns of model sensitivity and validate selected findings through direct input perturbations, which generally reproduce the directions predicted by the CAV-based analysis.

Critical strengths

A particularly valuable strength is the perturbation-based behavioral validation. Rather than relying solely on CAV-derived scores, the authors directly perturb paired spectral regions and test whether the classifier responds in the directions predicted by the pairwise analysis.

Critical weaknesses

The paper’s central claim about pairwise concept interactions is not sufficiently supported by the proposed formulation. Sum and Diff are simply an invertible linear reparameterization of the two standardized concept intensities, and Tradeoff conditions the analysis on opposite-sign samples. Consequently, all three contrasts can exhibit strong directional effects even for a purely additive model with no interaction between the two regions. Thus, the experiments characterize directional sensitivity in a two-concept space, but do not establish non-additive interactions or demonstrate that the pairwise analysis captures effects beyond an additive combination of individual concept contributions.

A second critical concern is the statistical significance procedure. The observed TCAV score is compared with N=20N=20 random-region null scores using a one-sample t-statistic whose denominator is the standard error of the null mean. It is unclear why testing whether a single observed score is extreme relative to an empirical null distribution should use uncertainty in the estimated null mean rather than the variability of the null distribution itself. This choice may substantially inflate apparent significance and is particularly concerning given that 85.9% of the 504 pairwise tests are reported significant, without multiple-comparison correction.

Questions for authors
  1. Pairwise interaction versus additive effects : Can the authors clarify what constitutes a pairwise interaction in the proposed framework? Since Sum and Diff are linear combinations of individual concept intensities, and Tradeoff restricts the analysis to opposite-sign samples, similar directional sensitivities could arise from a purely additive model. What evidence establishes that the reported effects reflect pairwise interactions beyond individual concept contributions?

  2. Consistency between TCAV formulation and reported scores : Section 3.1 and Appendix A.2 describe a GAP representation followed by a dense classification head. If the class logits are affine functions of the GAP features, their gradients with respect to those features should be constant across inputs. With a fixed CAV, Eq. (6) would therefore produce TCAV scores of either 0 or 1. Could the authors clarify how the intermediate TCAV scores reported in the experiments are obtained, including whether the implementation differentiates logits or softmax probabilities?

Reviewer_vW2rOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper is concerned with explaining Raman classifiers (spectroscopy data classifiers) with concepts. To this end, it extends the established Testing with Concept Activation Vectors (TCAV) method to spectroscopy data. The main contribution lies in the processing of the spectroscopy data and how a concept is defined. Specifically, the intensity of specific bands (Fig. 1 b) is used to define a concept (Eq. 2-3). Further, concept pairs are defined, such as the sum of two concepts, to assess the importance of relationships between spectral regions (Sec. 3.5). In the experiments, the proposed method is applied to an Raman classifier with a detailed analysis on the importance of the different concepts (bands) and concept pairs.

Critical strengths
  • The paper is well-written and easy to follow.
  • The problem is clearly motivated.
  • The idea of extending XAI to other domains is interesting.
  • The proposed method seems reasonable for most parts.
Critical weaknesses
  • W1 Comparison to other spectral-based XAI methods is missing. As also acknowledged in the limitations (L. 459), the paper does not compare the proposed approach to existing spectral or concepts-based XAI methods. This makes it hard to assess if the conclusions have only been enabled through the proposed method or if similar conclusion could have been drawn with existing approaches.
  • W2 Scope of the paper. I am not convinced that the scope of the paper fits ICLR. Overall, the paper is well-written with a clear method that seems sensible. However, the method is a rather straightforward extension of TCAV, with the contribution being more in the data preparation (i.e., what constitutes a concept) than in the actual method. Further, most part of the results (Sec. 4) focuses on analysing the underlying task rather than the proposed XAI method. As such, it is analysed how important which concept is (e.g., Sec. 4.2) istead of comparing the proposed method to existing approaches. Consequently, the results are probably more interesting to researchers of the underlying domain than to the general XAI or Machine Learning researcher which I see more as the target audience of ICLR.
  • W3 Unexplained information. There are works showing that concept based explanations are rarely enough to explain the full model decision (e.g., "ELUDE: Generating interpretable explanations via a decomposition into labelled and unlabelled features"). It could be interesting to study how much information is not explained by the given concepts and also what this information encodes.
Questions for authors
  • How does the proposed method compare to established XAI methods and can similar conclusions be drawn with these methods (maybe under the same data processing; i.e., the same concept generation).
  • How much information is being missed by the CAVs and what does this information encode?
  • Why is the paper a good fit for ICLR?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.