acceptodds
Under review as a conference paper at ICLR 2027

Faithfulness Is Conditional: Auditing Perturbation-Based Explanation Evaluation in Time Series

Abstract

Deep learning plays an important role in time-series classification (TSC), but its black-box nature motivates explanations that help users understand and inspect predictions. *Faithfulness* is a central criterion for evaluating these explanations, yet the suitability of the evaluator itself also requires examination. In particular, dataset-level results do not establish whether an evaluator remains informative within each class. We propose a *simple class-conditional audit that compares informative reference attributions with random rankings*. Across controlled synthetic datasets and an ECG benchmark, we find that *an evaluator can distinguish these attributions overall while failing the same comparison within particular classes*. Motivated by these findings and studies of animal perceptual decision-making, we examine perturbation baseline predictions and representations, identifying baseline class preference as a possible contributor to these failures. We introduce a *NULL response* that changes how the model evaluates baselines by training an additional output while keeping the original representation and real-class logits frozen. We also investigate *input-dependent counterfactual (CF)* as baselines that change the replacement input itself, using a transductive, label-informed construction. Both strategies improve meaningful–random separation in several settings, but neither passes the audit across all tested configurations. We therefore treat NULL and CF as two options whose use is guided by the class-conditional audit. *This audit acts as an initial gate for selecting evaluation configurations* for each trained model, dataset, and intended class scope before their scores are used to compare explanation methods. Although the audit does not certify explanation correctness or replace more comprehensive evaluation, this audit provides evidence of simple but mandatory meaningful–random explanation separation.

Primary area:
interpretability and explainable AI
Keywords:
Explanation, XAI, Time Series, Faithfulness, Evaluation

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_YzHcOpenReview ↗
Rating
3(+2)
Confidence
4
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes an algorithm for auditing whether a given attribution is more informative than a random attribution. Attributions that fail this test are considered uninformative. The approach is class-conditional, allowing informativeness to be assessed for specific target classes. Passing the test does not establish that an attribution is meaningful or plausible in an absolute sense; it only indicates that it is preferred over a random baseline. The evaluation uses several datasets, predominantly synthetic ones, with Integrated Gradients and Feature Ablation as reference attribution methods. The paper addresses an important problem in time-series attribution and presents a promising approach. However, the limited evaluation, insufficient justification of the decision threshold, and lack of analysis of performance differences across datasets and classes restrict the conclusions that can be drawn about its generalizability and practical applicability.

Critical strengths

Mathematical details are presented detailed and complete.

Critical weaknesses
  • The evaluation does not include sota network architectures. This limits the evidence that the approach generalizes across models and attribution methods. A broader comparison is needed to understand how the choice of attribution method affects the assessment.
  • The threshold used to classify an attribution as informative lacks sufficient statistical and empirical justification. The paper does not adequately examine sensitivity to this choice, variation across datasets, or whether a common threshold can be applied reliably across various settings.
  • The figures suggest that the approach succeeds only for certain datasets and classes, but the evaluation does not sufficiently explain these differences. It remains unclear which properties determine success or failure and how a practitioner could identify suitable settings in advance.
  • The evaluation relies heavily on synthetic and controlled datasets which are useful for systematic analysis, but the small number of real-world datasets provides limited evidence of effectiveness in practical applications.
Questions for authors

None aside from above

9 comments
AuthorsOpenReview ↗

Paper Summarisation

We thank the reviewer for the constructive review and for recognizing that our paper addresses an important problem in time-series attribution with a promising approach. We apologize that our unclear writing may have caused confusion about what the audit test is. Our audit tests the faithfulness evaluator rather than the attribution itself. To clarify this difference, the table below compares our audit with testing an attribution against random, which is similar to ROAR [A] (Hooker et al., 2019).

Testing an attribution against randomOur audit of the evaluator
What is testedIt tests the attribution map.The algorithm tests the evaluator, meaning the faithfulness metric, perturbation baseline and direction (like a meta-evaluator or meta-auditor).
Meaningful referenceThe system does not have, because the attribution itself is unknown.The algorithm uses a well defined meaningful reference ranking with random attribution map. This is GT on the synthetic datasets and MIT-ECG (QRS regions), and IG or feature-ablation maps on UCR, where no GT exists.
QuestionThe method asks whether a attribution beats random.The method asks whether this evaluator separates the meaningful reference from random within each class.
What a failure meansThe attribution is uninformative.The evaluator is Blind or Inverted in that class while the meaningful attribution is still meaningful. We only judge the evaluator.
What the result decidesThe method decides whether to keep or drop the attribution.The method decides whether the evaluator may rank explainers higher random attribution maps in this setup, which is our audit gate that evaluators need to pass.
ScopeThis is a faithfulness evaluation that scores attributions.The algorithm is for a case-specific audit of the evaluator, repeated for each model, dataset, class and setup, rather than a universal metric.

Reference:

[A] Hooker, S., Erhan, D., Kindermans, P.-J., & Kim, B. (2019). A benchmark for interpretability methods in deep neural networks. In Advances in Neural Information Processing Systems (Vol. 32).

AuthorsOpenReview ↗

Q1/W1: The evaluation does not include sota network architectures

Q1/W1: The evaluation does not include sota network architectures.

We thank the reviewer for this suggestion. We add three public datasets, CricketX and UWaveGestureLibraryX from the UCR archive and EpilepticSeizure, an EEG dataset used by [B], and three strong time-series classifiers. Mantis is a recent foundation model for time-series classification that outperforms existing foundation models across four dataset collections [C], and we fine-tune it on each dataset. HYDRA is faster and more accurate than the most accurate dictionary methods on 112 UCR datasets [D], and an independent benchmark finds its combination with MultiROCKET among the two best classifiers [E]. ModernTCN is a recent pure convolution architecture for general time-series analysis [F]. Table 1 gives the test accuracy of each model. We explain Mantis and ModernTCN with IG and HYDRA with FA, because the HYDRA transform is not differentiable. We audit each model with five perturbation baselines, including zeros, NULL zeros, Gaussian noise, NULL noise and CF train, where CF train is fitted on the train split. We use each baseline with both directions, MoRF and LeRF. A class passes when the explanation is better than the random in this class by at least ε = 0.005. Tables 2 to 4 count the classes that pass. First, the main failure in our paper also appears in these models. With zeros or Gaussian noise, the audit over all test series usually passes, but at least one class often fails. Second, adding NULL to the zeros baseline fixes some of these class failures, while NULL with noise and CF train fix only a few, and many classes still fail with them. Each fix therefore helps only in some settings, so every configuration needs its own audit.

Table 1: test accuracy / macro-F1

datasetMantisHYDRAModernTCN
CricketX0.795 / 0.7970.795 / 0.7930.710 / 0.702
EpilepticSeizure0.814 / 0.8120.791 / 0.7920.754 / 0.751
UWaveGestureLibraryX0.857 / 0.8560.851 / 0.8460.738 / 0.718

Table 2: CricketX, classes that pass (of 12)

modelDir.zerosNULL zerosnoiseNULL noiseCF train
MantisMoRF111111113
MantisLeRF1111111210
HYDRAMoRF812704
HYDRALeRF117916
ModernTCNMoRF121212126
ModernTCNLeRF1212121210

Table 3: EpilepticSeizure, classes that pass (of 5)

modelDir.zerosNULL zerosnoiseNULL noiseCF train
MantisMoRF34323
MantisLeRF44113
HYDRAMoRF33002
HYDRALeRF55125
ModernTCNMoRF44113
ModernTCNLeRF33133

Table 4: UWaveGestureLibraryX, classes that pass (of 8)

modelDir.zerosNULL zerosnoiseNULL noiseCF train
MantisMoRF56423
MantisLeRF66554
HYDRAMoRF22201
HYDRALeRF66706
ModernTCNMoRF88887
ModernTCNLeRF58681

Reference:

[B] Fan H, Zhang F, Wang R, Huang X, Li Z. Semi-supervised time series classification by temporal relation prediction. InICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2021 Jun 6 (pp. 3545-3549). IEEE.

[C] Vasilii Feofanov, Songkang Wen, Shifeng Xie, Simon Roschmann, Marius Alonso, Hongbo Guo, Romain Ilbert, Malik Tiomoko, Quentin Bouniot, Zeynep Akata, Lujia Pan, Jianfeng Zhang, Ievgen Redko. Mantis: Lightweight Foundation Model for Time Series Classification. Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:30692-30719, 2026.

[D] Dempster, Angus, Daniel F. Schmidt and Geoffrey I. Webb. “Hydra: competing convolutional kernels for fast and accurate time series classification.” Data Mining and Knowledge Discovery 37 (2022): 1779-1805.

[E] Middlehurst, Matthew, Patrick Schäfer and A. Bagnall. “Bake off redux: a review and experimental evaluation of recent time series classification algorithms.” Data Mining and Knowledge Discovery 38 (2023): 1958 - 2031.

[F] Luo, Donghao and Xue Wang. “ModernTCN: A Modern Pure Convolution Structure for General Time Series Analysis.” International Conference on Learning Representations (2024).

AuthorsOpenReview ↗

Q2/W2: The threshold used to classify an attribution as informative lacks sufficient statistical (1/3)

Q2/W2: The threshold used to classify an attribution as informative lacks sufficient statistical and empirical justification.

We thank the reviewer for this point. We support ε = 0.005 with two tests. The first test shows that ε is larger than the change that the standard deviation of different random ranking, so a Healthy label does not come from a lucky random draw. The second test shows that our conclusions do not change when ε changes.

First, we choose ε so that a label cannot come from a lucky random draw. Our audit draws one random ranking for each test series, from a manual seed. For a fixed model, this random ranking is the only random part of the audit, so ε must be larger than the change in the gap that a different random draw can cause. To measure this change, we redraw the random ranking with 10 seeds, m = 0, …, 9, where draw 0 is the one used in the audit. The GT map does not change, so we score it only once. For each draw m, we compute the same gap as in Equation 3:

Δ_m = S(GT) − S(rand_m),

where S is the true-class confidence averaged over the 11 retention ratios. For MoRF we flip the sign, so a positive Δ_m always means that GT beats random. Each draw then gets its own label, which is Healthy if Δ_m ≥ ε, Inverted if Δ_m ≤ −ε, and Blind otherwise.

The test covers 108 settings, which come from 6 datasets × 3 architectures × 2 directions × 3 scopes. The datasets are FPH, RPH, FPH-FD, RPH-FD, MC-1v5 and PP, and the architectures are Linear, CNN and RNN. The two directions are LeRF, which replaces the least important points with the baseline first, and MoRF, which replaces the most important points first. The three scopes are all test examples, the examples of class 0 and the examples of class 1. All settings use the base models with the background and seed 0.

Tables 5 to 7 report, for each setting, the gap of the audit's own draw (Δ₀), the mean of the 10 gaps (Δ̄) and their standard deviation (σ). The standard deviation is at most 0.0047, so ε = 0.005 is larger than the spread between random draws in every setting. A gap of at least ε therefore does not come from a lucky draw. Indeed, the audit label does not change in any of the 108 settings, for every single draw.

Reviewer_iW7YOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether perturbation-based faithfulness evaluators (MoRF/LeRF) that separate informative from random attributions on a whole dataset still do so within each class. It proposes a class-conditional audit and shows, on synthetic, SHAPEX, MIT-ECG, and UCR datasets, that aggregate separation often hides classes where the evaluator is blind or inverted. The authors attribute this to baselines being confidently assigned to one real class. They test two remedies, a NULL output on frozen representations and counterfactual references, and, since neither works everywhere, recommend the audit as a gate for choosing evaluation configurations.

Critical strengths

Simple, useful diagnostic:
The audit is easy to apply and exposes a real blind spot. CNN LeRF separates GT from Random overall (0.95 vs. 0.64) but not within class 0 (0.99 vs. 0.99), and the pattern holds across architectures, seeds, and datasets.

Coherent mechanism:
The baseline-collapse analysis matches where failures occur: the failing class is the one baselines collapse to, including the reversed case on MTCE.

Controlled design and candid framing:
The synthetic datasets each isolate a single factor (distractor, motif count, position), making the contrasting behavior of NULL and CF interpretable.

Critical weaknesses

Unclear novelty:
Closely related work is not cited. Baer et al. (XAI 2025) study class-dependent perturbation effects in time-series attribution evaluation, the paper’s central observation, and Schlegel & Keim (XAI 2023) and Šimić et al. (2025) analyse perturbation strategies for time-series XAI evaluation; work on baselines and OOD perturbations (Sturmfels et al., 2020; Hooker et al., 2019; Rong et al., 2022, Theissler et al., 2022) is also missing and most likely even more in the counterfactual domain.

NULL may measure plausibility rather than faithfulness:
On data where the only non-background content is the GT region, a NULL head trained on pure baselines may simply detect remaining signal, rewarding GT-aligned rankings by construction rather than because f_θ uses those features. It also evaluates the probabilities of a different model. A shortcut-model sanity check would address this.

CF and the gate rely on unavailable information:
The CF generator is fitted on the test set with true labels. The gate needs reference attributions, which for UCR are IG and feature ablation, risking circularity, and no separate audit set is used. No experiment shows the gate changing how real explainers are ranked.

Hooker, S., Erhan, D., Kindermans, P.-J., & Kim, B. (2019). A benchmark for interpretability methods in deep neural networks. In Advances in Neural Information Processing Systems (Vol. 32).
Rong, Y., Leemann, T., Borisov, V., Kasneci, G., & Kasneci, E. (2022). A consistent and efficient evaluation strategy for attribution methods. In Proceedings of the 39th International Conference on Machine Learning (pp. 18770–18795). PMLR.
Sturmfels, P., Lundberg, S., & Lee, S.-I. (2020). Visualizing the impact of feature attribution baselines. Distill, 5(1), e22.
Theissler, A., Spinnato, F., Schlegel, U., & Guidotti, R. (2022). Explainable AI for time series classification: a review, taxonomy and research directions. Ieee Access, 10, 100700-100724.
Schlegel, U., & Keim, D. A. (2023). A deep dive into perturbations as evaluation technique for time series XAI. In World Conference on Explainable Artificial Intelligence (pp. 165–180). Springer Nature Switzerland.
Šimić, I., Veas, E., & Sabol, V. (2025). A comprehensive analysis of perturbation methods in explainable AI feature attribution validation for neural time series classifiers. Scientific Reports, 15(1), Article 26607.
Baer, G., Grau, I., Zhang, C., & Van Gorp, P. (2025). Class-dependent perturbation effects in evaluating time series attributions. In World Conference on Explainable Artificial Intelligence (pp. 292–314). Springer Nature Switzerland.

Questions for authors

Which of your results are new relative to Baer et al. (2025), Schlegel & Keim (2023), and Šimić et al. (2025)?

Do class-0 failures persist with a baseline-corrected response such as f_c(x̂) − f_c(x̃), or a baseline with near-uniform prediction?

How many random rankings are drawn, and would a permutation test change any audit labels?

Does CF still work with a training-split reference pool or predicted labels?

Does the gate change the ranking of established time-series explainers?

18 comments
AuthorsOpenReview ↗

Q1: Which of results are new relative to Baer et al. (2025), Schlegel & Keim (2023), and Šimić et al. (2025)? (1/2)

We thank the reviewer for their constructive review.

Q1: Which of your results are new relative to Baer et al. (2025), Schlegel & Keim (2023), and Šimić et al. (2025)?

First of all, we compare with Baer et al. (2025) in Table 1:

Table 1: Ours vs. Baer et al. (2025)

Baer et al. (2025)Ours
ProblemThey study whether the degradation score (DS) stays consistent across classes, and they add a class penalty with weight α\alpha to get DSc=DS−αΔ\mathrm{DS}_c = \mathrm{DS} - \alpha\Delta. DSc\mathrm{DS}_c is a class-aware evaluation metric, so it cannot tell whether DS or DSc\mathrm{DS}_c itself is good or bad. Their work is therefore an analysis of a class-aware metric, not a meta-evaluation or an audit.We audit the evaluator with a known meaningful–random pair within each class. Our problem is therefore the meta-evaluation auditor, which asks whether the evaluator separates meaningful from random explanations in each class.
Empirical scopeThey use four UCR datasets without ground truth, and they report DSc\mathrm{DS}_c for five attribution methods to show that scores differ between classes. DSc\mathrm{DS}_c cannot tell whether a class gap comes from the explainer or from the evaluator.We run controlled experiments with predefined ground truth on 10 synthetic datasets and on MIT-ECG (QRS regions). We test the MoRF and LeRF metrics common in TSC XAI and the recent F-Fidelity (ICLR 2025). In this rebuttal, we also test whether evaluators rank explainers in the class level scope.
RecommendationThey recommend class-stratified analysis and a penalty, without auditor or evaluation.We give an audit gate that the evaluator must pass in every class before the next step.
ImpactThey do not link class effects to harm.Our audit can prevent dangerous failures that a good overall score hides. The example below will show this danger [Note 1].

[Note 1] The class-level failure is a dangerous failure in XAI evaluation, and it can harm people. Many problems hide behind a method that looks good overall, and they appear only when we check each group. For example, [A] studied a widely used commercial algorithm that affects millions of patients. At a given risk score, Black patients were considerably sicker than White patients, because the algorithm predicted health care costs rather than illness. Unequal access to care causes that less money is spent on Black patients, so Black patients with the same need were less likely to receive extra care. This failure came from a convenient setup that health care cost is chosen for the true target. The study teaches us that a simple group-by-group check can reveal serious problems hidden inside a system, which can harm many patients in our community. [B] show that explanations can be manipulated to look plausible and still pass a fidelity check, even when they do not reflect what the model uses. In a class that fails our audit, meaningful and random explanations receive the same faithfulness score, so this evaluator cannot detect whether the explanations in that class are faithful. The worst case in ECG classification is an abnormal ECG classified as normal, with an explanation that makes clinicians trust the AI. Such misdiagnoses are costly. The paper [C] estimates that 795,000 Americans become permanently disabled or die each year because dangerous diseases are misdiagnosed. In ECG monitoring, misleading explanations that make clinicians trust an abnormal ECG as normal could be the reason for such misdiagnoses.

[A] Obermeyer, Ziad et al. “Dissecting racial bias in an algorithm used to manage the health of populations.” Science 366 (2019): 447 - 453.

[B] Lakkaraju, Himabindu and Osbert Bastani. “"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations.” AIES (2019).

[C] Newman-toker, David E. et al. “Burden of serious harms from diagnostic error in the USA.” BMJ Quality & Safety 33 (2023): 109 - 120.

AuthorsOpenReview ↗

Q1: Which of results are new relative to Baer et al. (2025), Schlegel & Keim (2023), and Šimić et al. (2025)? (2/2)

We also compare our paper with other papers in Table 2.

Table 2: Ours vs. Baer et al. (2025), Schlegel & Keim (2023), and Šimić et al. (2025)

Baer et al. (2025)Schlegel & Keim (2023)Šimić et al. (2025)Ours
Targets the class levelYesNoNoYes
Meaningful–random checkNoNoNo (random map is only one candidate to be ranked)Yes
Controlled experiments with GTNoNoNoYes
New auditor for evaluatorsNoNoNoYes
Auditor per setup (no universal metric)NoNoNoYes
Audits LeRF/MoRF commonly used in TSC XAI and F-FidelityNoNoNoYes
Gate in the evaluation pipelineNoNoNoYes (per-class audit gate)
Links class failure to a concrete harmNoNoNoYes ([Note 1])
Shows a class failure hidden by a good dataset-level resultYes (cause only a hypothesis, no meaningful–random check)NoNoYes

In summary of Q1, our paper brings three points that are novel to these works. First, we go beyond a hypothesis and an analysis with a class penalty. We directly audit whether the evaluator separates meaningful from random explanations within each class, we show that this separation can fail even when the evaluator looks good on the whole dataset, and we trace the failure to the perturbation baseline. Second, we propose an audit gate for case-specific scenario that keeps an evaluator that fails in a class out of high-stakes applications, where it could cause harm as in [B]. Lastly, we propose NULL and CF as candidate fixes for the baseline problem.

[A] Obermeyer, Ziad et al. “Dissecting racial bias in an algorithm used to manage the health of populations.” Science 366 (2019): 447 - 453.

[B] Lakkaraju, Himabindu and Osbert Bastani. “"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations.” AIES (2019).

[C] Newman-toker, David E. et al. “Burden of serious harms from diagnostic error in the USA.” BMJ Quality & Safety 33 (2023): 109 - 120.

AuthorsOpenReview ↗

W1: Unclear Novelty

W1: Unclear Novelty

We agree that some closely related works are not cited, and we are adding them to the revised version. Our novelty beyond Baer et al. (2025), Schlegel & Keim (2023) and Šimić et al. (2025) is addressed in our answer to Q1. Sturmfels et al. (2020) study the baseline of path attribution methods such as integrated gradients, and they compare several baseline choices and their effects. Their study concerns the baseline inside the attribution method, while we audit the baseline inside the evaluator and in the class scope.

Hooker et al. (2019) propose RemOve And Retrain (ROAR) to evaluate attribution maps. They show that a retrained ResNet-50 still reaches 63.53% accuracy on ImageNet with 90% of the input features removed. They also find that some attribution methods are no better than a random assignment of importance, while others are clearly better than random guess. This comparison is close to our meaningful–random check, but they compare against random only over the whole test set to rank attribution methods. Our findings show that an evaluator can separate meaningful from random explanations well at the dataset level and still fail within a class. Such a failure is dangerous because a reliable dataset-level result hides it, as [Note 1] shows.

Rong et al. (2022) show that the shape of the removed region can leak class information and mislead the evaluation. They propose Remove and Debias (ROAD), which avoids the cost of retraining and fills the removed points with an imputation from which the removed region is hard to recognize. This leakage is related to our study of perturbation effects, and we test its effect on our results in our reply to Reviewer 8Wvy. However, they do not check whether the evaluation fails within a class.

Theissler et al. (2022) review XAI for TSC and point out that perturbation needs "a non-information holding value as a baseline". They also note that such values are hard to find in time series, because "values like 0 could have a specific meaning altering the class membership" (Sec. VI-B). We go further and show that even a baseline that looks uninformative can be predicted as one class, so class collapse can still happen. For this reason, an evaluator should pass the audit gate before the next step.

Reviewer_8WvyOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether perturbation-based faithfulness evaluators can distinguish informative from random time-series explanations within individual classes, even when aggregate evaluations appear reliable. The authors introduce a class-conditional audit and investigate two strategies for improving evaluation reliability: an additional NULL response trained on frozen representations and input-dependent counterfactual (CF) baselines. Experiments on synthetic datasets, MIT-ECG, and UCR benchmarks demonstrate class-conditional evaluation failures and show that NULL and CF can partially mitigate them.

Critical strengths
  • The paper identifies an important limitation of aggregate faithfulness evaluation, demonstrating that favorable overall scores can conceal systematic failures within individual classes. The controlled experiments across multiple architectures and datasets provide convincing evidence that class-conditional evaluation deserves greater attention.

  • The investigation of baseline prediction bias offers useful insights into the behavior of perturbation-based evaluators. The complementary effects of NULL responses and counterfactual references highlight the importance of examining evaluation configurations rather than assuming that a particular perturbation strategy is universally reliable.

Critical weaknesses
  • The proposed audit relies on the assumption that reference attributions should consistently outperform random rankings. However, ground-truth regions in synthetic datasets represent the data-generating mechanism rather than necessarily the features used by a trained model. On real-world datasets, reference attributions such as Integrated Gradients are themselves imperfect. Consequently, failing the audit does not necessarily imply that an evaluator is unfaithful to the model, limiting the validity of the proposed criterion as a mandatory evaluation gate.

  • The NULL response modifies the evaluated class probabilities even though the original classifier's representations and real-class logits remain unchanged. Thus, improved attribution–random separation may primarily reflect the additional head's ability to recognize perturbed inputs rather than improved measurement of the original model's feature dependence. The experiments do not sufficiently disentangle these effects, raising concerns about whether NULL genuinely improves faithfulness evaluation.

  • The counterfactual baseline construction relies on true test labels and optimization over the entire test set, making it inherently transductive and label-dependent. Although this is acknowledged, the resulting evaluation procedure cannot directly generalize to unseen or unlabeled samples. This substantially limits its applicability to practical settings where explanation reliability must be assessed without access to ground-truth labels for every evaluated instance.

  • The proposed audit uses a fixed tolerance of 0.005 to categorize evaluations as Healthy, Blind, or Inverted, without systematically investigating its sensitivity to sample size, class imbalance, or statistical uncertainty. More importantly, the evaluation primarily examines informative–random separation rather than whether configurations passing the audit reliably rank multiple explanation methods according to their actual faithfulness. This leaves the practical effectiveness of the proposed evaluator-selection procedure insufficiently established.

Questions for authors
  • How do the authors justify treating reference–random separation as a necessary condition for evaluator reliability when reference attributions may not reflect the actual features used by the trained model? Can the audit distinguish evaluator failures from failures of the reference explanation itself?

  • Since the NULL response changes class probabilities, how can the authors establish that improved separation reflects more faithful evaluation of the original classifier rather than simply better detection of perturbed inputs? Can additional controlled experiments isolate these effects?

  • Can the counterfactual reference construction be extended to an inductive, label-free setting? How much of its reported improvement remains when the generator is trained on separate training data without access to test labels?

  • How sensitive are the audit outcomes to the tolerance threshold and class sample sizes? More importantly, do configurations passing the audit produce more reliable rankings among several established explanation methods, beyond distinguishing reference attributions from random rankings?

15 comments
AuthorsOpenReview ↗

W1/Q1: How do the authors justify treating reference–random separation as a necessary condition

We thank the reviewer for their constructive review.

W1/Q1: How do the authors justify treating reference–random separation as a necessary condition for evaluator reliability when reference attributions may not reflect the actual features used by the trained model?

On synthetic data, we follow the standard setup of time-series XAI methods such as TimeX (Queen et al., 2023), TimeX++ (Liu et al., 2024a) and Dynamask (Crabbé & van der Schaar, 2021). These works build synthetic datasets with a predefined binary attribution map that comes from the data design. They then score explanations against this map with AUPRC, AUP and AUR. We therefore use this ground truth (GT) as the meaningful reference. Our audit does not assume that GT is exactly what the model uses. It needs only a weak assumption, namely that GT is closer to what the model uses than a random ranking is. Under this assumption, an evaluator should rank GT above random within each class.

On real data without GT, we use Integrated Gradients (IG) and Feature Ablation (FA), two well-known attribution methods, to build the meaningful references. We agree with the reviewer that these references are imperfect, because no ground truth of what a black-box model uses is available. App. D.1 states this directly, since a failure to separate these references from random "may reflect limitations of the attribution method, the evaluator, or both." We only assume that these references are more meaningful than random.

On real data, the baseline can also weaken the reference itself, and App. D.2 studies this case. On FordA, replacing an input with its baseline lowers both logits of Mantis by similar amounts. As a result, the probability of the original class stays near 0.85 and 0.82 (Fig. 10). IG on class probabilities sums to the input–baseline probability difference, so this difference becomes small or even negative in the class the baseline is predicted as (Fig. 11). This can show the IG reference can be the reason for the evaluator failing. We therefore also generate IG and FA from NULL-augmented models, where the baseline is predicted as NULL (Figs. 14 and 15). These attributions do not remove the class failures by themselves but make the system providing more meaningful attribution maps. Combined with NULL evaluation, however, they separate from random in some settings. For example, on ECG200 class 0 rises from 0/2 healthy evaluations under MoRF and 1/2 under LeRF to 2/2 in both directions (Fig. 14).

In short, our audit only flags a failure to separate the meaningful reference from random within a class. The cause needs to be investigated, and it can come from the baseline or in the reference, which can guide how to fix it. We note this as future work for a more comprehensive auditing system.

AuthorsOpenReview ↗

W2/Q2: Since the NULL response changes class probabilities, how can the authors establish that improved separation reflects more faithful evaluation

W2/Q2: Since the NULL response changes class probabilities, how can the authors establish that improved separation reflects more faithful evaluation of the original classifier rather than simply better detection of perturbed inputs? Can additional controlled experiments isolate these effects?

To separate a more faithful evaluation from a better detection of perturbed inputs, we add a controlled experiment. If the NULL output only detected perturbed inputs, its response would depend on the shape of the perturbation. We therefore compare GT with the usual Random map, which ranks the points in a random order, and with a new map that has the same shape and size as GT, which we call Block-Random. For a GT with one segment, we choose a random start position outside GT and build a segment of the same size from it. For a GT with several segments, such as class 1 of MC-1v5, we do this for each segment. Block-Random thus perturbs the same number of points in the same shape as GT, but at a different place. Each score is the true-class confidence of the NULL model, averaged over the filling path as in the paper and over 3 architectures and 5 seeds. Tables 1 to 6 report the scores of GT, Random and Block-Random for MoRF and LeRF, in the overall, class 0 and class 1 scopes. The scores of Random and Block-Random are close, and they differ by at most 0.03 in 32 of the 36 cells. The four larger gaps are all on MC-1v5, where Block-Random moves further away from GT rather than toward it. In contrast, GT stays far from both random maps. The only exception is class 0 of FPH-FD, where GT does not beat either of them. This is expected, because the paper already reports that NULL BG fails on FPH-FD class 0, where only 2/15 class 0 evaluations pass (Section 5.2).This shows that the shape of the perturbation does not drive the NULL response. Instead, the score depends on where the perturbation falls, that is, on whether it removes the evidence. This result follows from the design of the NULL model. We freeze the base classifier and train only the NULL head, which is one linear layer on the penultimate features. The NULL head can therefore use only the information that these features already encode. If these features do not encode whether an input is perturbed, the NULL head cannot detect perturbed inputs as such.

Table 1: MoRF, overall

datasetGT scoreRandom scoreBlock-Random score
FPH0.1270.4440.463
RPH0.1040.4470.466
FPH-FD0.2820.3890.399
RPH-FD0.2460.3810.398
MC-1v50.1530.4530.515
PP0.1320.4480.465

Table 2: MoRF, class 0

datasetGT scoreRandom scoreBlock-Random score
FPH0.1640.4640.478
RPH0.1160.4880.505
FPH-FD0.4760.4560.456
RPH-FD0.3990.5200.525
MC-1v50.1530.4040.418
PP0.1730.4570.470

Table 3: MoRF, class 1

datasetGT scoreRandom scoreBlock-Random score
FPH0.0910.4230.447
RPH0.0920.4050.427
FPH-FD0.0880.3210.343
RPH-FD0.0940.2410.271
MC-1v50.1540.5010.612
PP0.0910.4390.460

Table 4: LeRF, overall

datasetGT scoreRandom scoreBlock-Random score
FPH0.8280.4420.427
RPH0.8390.4450.430
FPH-FD0.5300.3850.379
RPH-FD0.4950.3810.371
MC-1v50.7670.4520.390
PP0.8140.4450.431

Table 5: LeRF, class 0

datasetGT scoreRandom scoreBlock-Random score
FPH0.7490.4630.448
RPH0.8070.4870.469
FPH-FD0.4480.4530.450
RPH-FD0.5990.5180.508
MC-1v50.6890.4030.391
PP0.7200.4560.442

Table 6: LeRF, class 1

datasetGT scoreRandom scoreBlock-Random score
FPH0.9080.4210.405
RPH0.8720.4030.391
FPH-FD0.6110.3180.309
RPH-FD0.3900.2440.234
MC-1v50.8450.5000.389
PP0.9080.4350.420
AuthorsOpenReview ↗

W3/Q3:Can the counterfactual reference construction be extended to an inductive, label-free setting?

W3/Q3:Can the counterfactual reference construction be extended to an inductive, label-free setting? How much of its reported improvement remains when the generator is trained on separate training data without access to test labels?

Yes, the algorithm can extend to an inductive setting. We fit the same CF generator, with the same loss and settings, on the train split instead of the test split, and then apply it to the test series. This CF (train) uses the train labels, but it uses no test series and no test labels to build xrx^{r}. The test labels are used only to score the result. Tables 7 to 9 compare the GT–random gap of the background, CF (test) and CF (train) in the overall, class 0 and class 1 scopes. Each value is the mean gap (GT vs. radom) over the 3 architectures, and a positive gap means that GT beats random. On FPH, RPH, FPH-FD and RPH-FD, CF (train) stays close to CF (test), and the two differ by at most 0.03. On these datasets, CF (train) therefore keeps almost all of the results of CF (test). On MC-1v5 and PP, the differences are larger, up to 0.11 on MC-1v5 and 0.14 on PP, and they go in both directions. Across all 36 settings, CF (train) and CF (test) always give a gap with the same sign, so moving to the inductive setting never changes whether GT beats random.

Table 7: overall

datasetDir.backgroundCF (test)CF (train)
FPHLeRF+0.221+0.436+0.432
FPHMoRF+0.187+0.376+0.380
RPHLeRF+0.252+0.498+0.494
RPHMoRF+0.155+0.315+0.319
FPH-FDLeRF+0.045+0.413+0.420
FPH-FDMoRF+0.128+0.357+0.352
RPH-FDLeRF+0.064+0.456+0.463
RPH-FDMoRF+0.073+0.333+0.327
MC-1v5LeRF+0.166+0.148+0.112
MC-1v5MoRF+0.172+0.186+0.133
PPLeRF+0.185+0.206+0.295
PPMoRF+0.224+0.525+0.412

Table 8: class 0

datasetDir.backgroundCF (test)CF (train)
FPHLeRF−0.001+0.528+0.538
FPHMoRF0.000+0.281+0.271
RPHLeRF+0.007+0.604+0.601
RPHMoRF+0.065+0.205+0.209
FPH-FDLeRF−0.199+0.508+0.511
FPH-FDMoRF0.000+0.257+0.255
RPH-FDLeRF+0.025+0.513+0.503
RPH-FDMoRF+0.147+0.273+0.291
MC-1v5LeRF0.000+0.059+0.053
MC-1v5MoRF0.000−0.061−0.054
PPLeRF−0.001+0.251+0.332
PPMoRF+0.002+0.517+0.376

Table 9: class 1

datasetDir.backgroundCF (test)CF (train)
FPHLeRF+0.444+0.344+0.326
FPHMoRF+0.374+0.471+0.489
RPHLeRF+0.497+0.392+0.387
RPHMoRF+0.244+0.424+0.429
FPH-FDLeRF+0.288+0.319+0.330
FPH-FDMoRF+0.256+0.458+0.449
RPH-FDLeRF+0.103+0.399+0.423
RPH-FDMoRF0.000+0.394+0.364
MC-1v5LeRF+0.332+0.236+0.171
MC-1v5MoRF+0.344+0.433+0.320
PPLeRF+0.371+0.160+0.259
PPMoRF+0.446+0.532+0.448

We also count how many of the 3 models pass the audit for seed 0, where passing means being Healthy (gap ≥ ε = 0.005) overall, in class 0 and in class 1 (Table 10). The background passes for at most 1 of the 3 models in every setting. CF (train) passes for the same number of models as CF (test) in every setting except one. On MC-1v5 with LeRF, CF (test) passes for 2 of the 3 models and CF (train) for 1, and this difference comes from a single model. For the linear model, the class 0 gap changes from +0.0063 to +0.0039, a difference of only 0.0024. Both values lie within 0.0015 of the threshold ε = 0.005, so this one changes because of a very small shift in the gap.

Table 10: passes the audit (Healthy overall, in class 0 and in class 1)

datasetDir.backgroundCF (test)CF (train)
FPHLeRF0/33/33/3
FPHMoRF0/33/33/3
RPHLeRF1/33/33/3
RPHMoRF1/33/33/3
FPH-FDLeRF0/33/33/3
FPH-FDMoRF0/33/33/3
RPH-FDLeRF1/33/33/3
RPH-FDMoRF0/33/33/3
MC-1v5LeRF0/32/31/3
MC-1v5MoRF0/30/30/3
PPLeRF0/33/33/3
PPMoRF1/33/33/3
Reviewer_3958OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the reliability of perturbation-based faithfulness metrics for time-series explanations. The authors find that existing metrics can distinguish meaningful from random attributions at the dataset level while failing to do so for individual classes. To address this, they propose a class-conditional audit and investigate two mitigation strategies, NULL and counterfactual. Experiments across synthetic datasets, ECG, and UCR benchmarks demonstrate the class-conditional failures and show that NULL and CF can improve attribution separation in some settings.

Critical strengths

S1. The paper highlights an important issue with existing faithfulness metrics. I find this observation interesting and worth investigating, especially for applications where explanations for specific classes matter.

S2. The authors reproduce the class-conditional failures across multiple datasets, model architectures, and random seeds. The NULL response is also an interesting attempt to address baseline-related issues.

Critical weaknesses

W1. The class-conditional audit is a reasonable idea, but its core operation is essentially applying an existing meaningful-vs-random comparison at the class level. Given prior work on evaluator reliability and perturbation-based evaluation, I am not fully convinced that this constitutes a sufficiently substantial methodological contribution. More importantly, the experiments mainly show whether an evaluator can distinguish reference from random attributions, rather than whether passing the audit actually leads to more reliable rankings of explanation methods.

W2. The evaluation also focuses on a relatively narrow set of metrics (MoRF, LeRF, and F-Fidelity). While this is consistent with the paper's scope, stronger evidence linking audit outcomes to actual evaluation reliability would make the contribution much more convincing.

W3. Adding a NULL output effectively introduces a baseline/OOD detection mechanism and changes the class probabilities used for evaluation, even though the original classifier's logits remain unchanged. This raises a question: are we measuring an explanation's faithfulness to the original classifier, or its ability to produce perturbations that the NULL detector recognizes? An improved GT-Random separation could partly reflect the NULL detector's sensitivity to different masking patterns, rather than a more faithful evaluation of the original model.

W4. The optimization objective encourages generated references to be closer to different-label samples in L1 distance, but does not explicitly ensure that they flip the classifier's prediction or preserve the intended signal semantics. The construction also relies on the labels of other test samples, which limits its practical applicability. More importantly, label-informed references may introduce additional discriminative information, making it difficult to tell whether prediction changes are caused by the attribution ranking or by the replacement itself. Without stronger validation of the generated references, it is unclear whether CF actually reduces evaluation bias or simply introduces a different source of bias.

Questions for authors

Q1. What does the class-conditional audit add beyond existing meta-evaluation approaches? Prior work has already investigated evaluator reliability, meaningful-vs-random attribution comparisons, and perturbation-induced distribution shifts. Could the authors clarify what new insights the class-conditional audit provides beyond applying these existing ideas separately to each class?

Q2. Does passing the audit translate into more reliable explanation rankings? Could the authors evaluate whether configurations that pass the audit also produce more reliable rankings across different explanation methods, or across explanations with controlled levels of degradation?

Q3. For NULL, could the authors provide more analysis of what drives the improvement? E.g., how does the NULL rejection probability change with the retention ratio? How do the original class logits and NULL probabilities behave across classes? Is there a systematic difference in NULL responses between GT-guided and random masking?

Q4. For CF, how valid are the generated counterfactual references?*Could the authors report metrics such as classifier flip rate and reference realism, and examine whether the generated references introduce unintended discriminative information?

15 comments
AuthorsOpenReview ↗

W1 and Q1. What does the class-conditional audit add beyond existing meta-evaluation approaches?

We thank the reviewer for their constructive review.

W1 and Q1. What does the class-conditional audit add beyond existing meta-evaluation approaches?

As the paper states, we propose a simple class-conditional audit, and its novelty comes from three points. First, to our knowledge, this is the first work to audit whether a faithfulness evaluator separates meaningful from random explanations on the class level. Second, we go beyond the hypothesis that class bias affects evaluation. Third, we convert the principle of meaningful–random separation into a practical recommendation in overall and class scopes, an audit gate for faithfulness evaluation. This matters because the class-level failure is a dangerous failure in XAI evaluation, and it can harm people. Many problems hide behind a method that looks good overall, and they appear only when we check each group. For example, [A] studied a widely used commercial algorithm that affects millions of patients. At a given risk score, Black patients were considerably sicker than White patients, because the algorithm predicted health care costs rather than illness. Unequal access to care causes that less money is spent on Black patients, so Black patients with the same need were less likely to receive extra care. This failure came from a convenient setup. The study teaches us that a simple group-by-group check can reveal serious problems hidden inside a system, which can harm our community.

Our paper shows the same pattern in XAI evaluation. When an evaluator separates meaningful from random explanations well overall but fails in one class, the average scores still makes meaningful attribution maps look much better than random ones. The report then looks good, a method that scores well may be used, and the failure in that class can lead to serious consequences.

To see the risk, we can look closer to time-series classification on ECG data. ECG monitoring is very important for patients, and automated ECG classification can speed up this task and monitor many patients at the same time. In such a high-stakes application, clinicians need explanations from AI, especially when its decision does not agree with their own judgment (Tonekaboni et al., 2019). Our audit shows that on MIT-ECG, the evaluator fails the simple audit test for class 0, the normal class, in some setups.

Why is this dangerous? [B] show that explanations can be manipulated to look plausible and still pass a fidelity check, even when they do not reflect what the model uses. In a class that fails our audit, meaningful and random explanations receive the same faithfulness score, so this evaluator cannot detect whether the explanations in that class are faithful. The worst case in ECG classification is an abnormal ECG classified as normal, with an explanation that makes clinicians trust the AI. Such misdiagnoses are costly. The paper [C] estimates that 795,000 Americans become permanently disabled or die each year because dangerous diseases are misdiagnosed. In ECG monitoring, misleading explanations that make clinicians trust an abnormal ECG as normal could be the reason for such misdiagnoses.

In summary, we can compare our method with existing meta-evaluation approaches:

Existing meta-evaluatorsOurs
ScopeDataset levelClass level
InsightEvaluation is good or not at the dataset level.An evaluator can score well on the whole dataset but fail the meaningful–random check in some classes. Without our audit, this good score can lead to its use in high-stakes applications while the issues stay hidden.
ImpactA good overall result of evaluator helps people trust it to evaluate XAI methods before using them.Our audit checks the evaluator within each class to reveal hidden issues. This helps keep bad evaluations out of high-stakes applications.
ResearchMeta-evaluation shows that XAI evaluators can be biased, so they should be tested first.We show that an XAI evaluator that looks good at the dataset level can still fail in some classes. This failure should not be ignored, and the audit is a necessary first check.

In short, our class-conditional audit comes from the insight that an issue visible in the overall score is less dangerous than the one hidden behind a good overall score so that people trust and act on. Our recommended audit can reveal such hidden issues and stop them from reaching practice, which could help the community.

[A] Obermeyer, Ziad et al. “Dissecting racial bias in an algorithm used to manage the health of populations.” Science 366 (2019): 447 - 453.

[B] Lakkaraju, Himabindu and Osbert Bastani. “"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations.” AIES (2019).

[C] Newman-toker, David E. et al. “Burden of serious harms from diagnostic error in the USA.” BMJ Quality & Safety 33 (2023): 109 - 120.

AuthorsOpenReview ↗

W2/Q2: Does passing the audit translate into more reliable explanation rankings?

W2/Q2: Does passing the audit translate into more reliable explanation rankings? Could the authors evaluate whether configurations that pass the audit also produce more reliable rankings across different explanation methods, or across explanations with controlled levels of degradation?

We agree to check whether passing the audit leads to more reliable rankings of explanations. To investigate this, we build degraded attribution maps from the ground-truth (GT) map. Starting from the GT attribution scores AGTA_{GT}, we randomly choose k% of the positions and shuffle their scores, which gives a degraded map AkA_{k}. We use k = 0, 20, 40, 60, 80 and 100%, so each example has 6 maps. As k increases, the quality of AkA_{k} should decrease, so we know the ground-truth ranking of the 6 maps. Each evaluator then ranks the 6 maps by the score. Finally, we compare this ranking with the true ranking using Spearman's rank correlation coefficient. The coefficient ranges from −1 to 1, and a higher value means a better ranking.

Table 1: overall

methodFPHRPHFPH-FDRPH-FDMC-1v5PPmean
MoRF, BG+1.00+0.99+1.00+0.97+1.00+1.00+0.99
LeRF, BG+1.00+1.00+0.95+1.00+1.00+1.00+0.99
F+, BG+1.00+1.00+0.61+0.36+0.90+0.93+0.80
F−, BG+0.87+0.92+0.33+0.33–+0.94–
MoRF, NULL BG+1.00+1.00+0.81+0.82+1.00+1.00+0.94
LeRF, NULL BG+1.00+1.00+1.00+0.98+1.00+1.00+1.00
MoRF, CF+1.00+1.00+1.00+1.00+0.91+0.97+0.98
LeRF, CF+1.00+1.00+1.00+1.00+0.88+0.37+0.87

Table 2: class 0

methodFPHRPHFPH-FDRPH-FDMC-1v5PPmean
MoRF, BG−0.76−0.45−0.78−0.47−1.00−0.33−0.63
LeRF, BG−0.73−0.44−0.72−0.39−1.00−0.32−0.60
F+, BG–+1.00−1.00−0.68–––
F−, BG–+0.95−1.00−0.87–––
MoRF, NULL BG+1.00+1.00+0.33+0.22+1.00+1.00+0.76
LeRF, NULL BG+1.00+0.96+0.33+0.32+1.00+1.00+0.77
MoRF, CF+1.00+1.00+1.00+1.00−1.00−0.32+0.45
LeRF, CF+1.00+1.00+1.00+1.00−1.00−0.33+0.45

Table 3: class 1

methodFPHRPHFPH-FDRPH-FDMC-1v5PPmean
MoRF, BG+1.00+0.95+1.00+0.80+1.00+1.00+0.96
LeRF, BG+1.00+1.00+1.00+1.00+1.00+1.00+1.00
F+, BG+1.00+1.00+1.00+1.00+0.90+0.93+0.97
F−, BG+0.87+0.88+0.98+0.98–+0.94–
MoRF, NULL BG+1.00+1.00+1.00+1.00+1.00+1.00+1.00
LeRF, NULL BG+1.00+1.00+1.00+1.00+1.00+1.00+1.00
MoRF, CF+1.00+1.00+1.00+1.00+1.00+1.00+1.00
LeRF, CF+1.00+1.00+1.00+1.00+1.00+1.00+1.00

Tables 1, 2 and 3 report Spearman's rank correlation for the 6 datasets in the overall, class 0 and class 1 scopes. We average the correlation over 3 architectures and 5 training seeds. A value of +1 means that the evaluator ranks the maps in the true order, and −1 means that it ranks them in the reverse order. When all 6 maps get the same score, the correlation cannot be computed. The mark "–" shows the dataset that the correlation can not be estimated. In the mean column, "–" shows that at least one of the 6 datasets is "–".

Overall (Table 1), our NULL BG (the NULL with the background baseline) and CF (counterfactual) evaluators rank the maps about as well as MoRF, BG and LeRF, BG. This shows that passing the audit does not harm the overall ranking ability of an evaluator. The only clear exception is LeRF, CF on the dataset PP, which reaches +0.37. In class 0 (Table 2), MoRF, BG and LeRF, BG fail to rank the maps, with mean correlations of −0.63 and −0.60. In contrast, the NULL BG evaluators reach good correlations, with means of +0.76 and +0.77. They are weaker on FPH-FD and RPH-FD, where their correlations lie between +0.22 and +0.33. The CF evaluators rank perfectly on FPH, RPH, FPH-FD and RPH-FD, but they fail on MC-1v5 (−1.00) and PP (−0.32 and −0.33). In class 1 (Table 3), all evaluators rank well, with correlations of at least +0.80.

In summary, evaluators that pass our audit rank the maps as well as the other evaluators. However, an evaluator that ranks well in the overall scope can still fail within a class. In contrast, evaluators that pass the full audit, such as NULL BG, still rank well within each class. NULL BG ranks weaker only on FPH-FD and RPH-FD, which are also the datasets where it fails the audit in class 0. The one exception is CF on PP. It is Healthy in class 0 in 14 of 15 runs, but it still ranks poorly in class 0 (−0.32 and −0.33). Passing our audit therefore does not harm ranking, and in most cases it also gives good ranking in the class where the other evaluators fail. We thank the reviewer for this question, because this ranking test can extend our audit as the next gate after the meaningful-versus-random gate.

AuthorsOpenReview ↗

W3/Q3: For NULL, could the authors provide more analysis of what drives the improvement? (1/2)

W3/Q3: For NULL, could the authors provide more analysis of what drives the improvement? E.g., how does the NULL rejection probability change with the retention ratio? How do the original class logits and NULL probabilities behave across classes? Is there a systematic difference in NULL responses between GT-guided and random masking?

3.1. How does the NULL rejection probability change with the retention ratio? And 3.2. How do the original class logits and NULL probabilities behave across classes?

To study how the null probability changes with the retention ratio, we use the CNN with NULL with baseline as background on FPH and MC-1v5. The null probability is the softmax probability of the NULL class. Tables 4 and 5 show FPH classes 0 and 1, and Tables 6 and 7 show MC-1v5 classes 0 and 1. Each cell is the mean over the class's test examples and 5 training seeds. In all four tables, the null probability is very high at retention 0.0 and very low at retention 1.0. Between these two ends, with LeRF, the null probability falls faster for the GT map than for the random map. This happens because LeRF keeps the GT points first, so the most important features come back early. With MoRF, in contrast, the GT map keeps the null probability high for longer. At retention 0.9, MoRF has already removed the GT points, so from 0.9 so the input does not contain important features and looks like the background to the model. The only exception is MC-1v5 class 1, which has 5 motifs. MoRF removes all of them only at retention 0.7, so the null probability is 0.522 at 0.8 and 0.946 from 0.7 down.

Table 4: FPH, null class confidence, class 0, by retention ratio

Dir.ranking0.00.10.20.30.40.50.60.70.80.91.0
LeRFGT.960.044.044.044.044.044.044.044.044.044.044
LeRFrandom.960.939.889.804.683.537.389.258.155.086.044
MoRFGT.960.960.960.960.960.960.960.960.960.960.044
MoRFrandom.960.940.888.801.680.535.385.254.153.085.044

Table 5: FPH, null class confidence, class 1, by retention ratio

Dir.ranking0.00.10.20.30.40.50.60.70.80.91.0
LeRFGT.960.000.000.000.000.000.000.000.000.000.000
LeRFrandom.960.862.654.416.226.105.040.012.003.000.000
MoRFGT.960.960.960.960.960.960.960.960.960.960.000
MoRFrandom.960.855.648.416.223.099.038.012.002.000.000

Table 6: MC-1v5, null class confidence, class 0, by retention ratio

Dir.ranking0.00.10.20.30.40.50.60.70.80.91.0
LeRFGT.946.086.086.086.086.086.085.085.085.085.085
LeRFrandom.946.933.895.831.740.621.489.358.236.148.085
MoRFGT.946.946.946.946.946.946.946.946.946.946.085
MoRFrandom0.9460.9320.8960.8280.7340.6170.4830.3520.2360.1460.085

Table 7: MC-1v5, null class confidence, class 1, by retention ratio

Dir.ranking0.00.10.20.30.40.50.60.70.80.91.0
LeRFGT.946.009.000.000.000.000.000.000.000.000.000
LeRFrandom.946.843.451.106.014.002.000.000.000.000.000
MoRFGT.946.946.946.946.946.946.946.946.522.000.000
MoRFrandom.946.845.452.106.015.002.000.000.000.000.000

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response to all reviewers and the AC

We thank all reviewers and the AC for their careful reading and constructive feedback. The reviewers agree that the paper studies an important problem (3958, 8Wvy, iW7Y, YzHc), that the audit is a simple and useful diagnostic (iW7Y), the controlled experiments across datasets, architectures and seeds are convincing (3958, 8Wvy), and the baseline-collapse analysis explains where the failures occur (iW7Y).

What the audit tests. Part of our writing may have suggested that the audit judges whether an attribution is informative, and we apologize for this confusion. Our audit tests the faithfulness evaluator, meaning the metric, the perturbation baseline and the direction, not the attribution method. It asks whether the evaluator separates a meaningful reference from a random ranking within each class. Our main finding is that an evaluator can pass this check on the whole dataset and still fail within one class, and the good overall score hides this failure. We therefore propose the class-level audit as a gate that an evaluator must pass before it is used to rank explanations.

New experiments in this rebuttal. Following the reviewers' suggestions, we ran the experiments below. The details and tables are in the individual responses.

concernwhat we addedreviewers
Does passing the audit give reliable rankings?Ranking of degraded GT maps, and ranking of IG, Saliency, LIME and FA together with a random ranking3958, 8Wvy, iW7Y
What drives NULL?NULL probability, true-class probability and logits along the path, a Block-Random control with the same shape as GT, and a shortcut-model sanity check3958, 8Wvy, iW7Y
Is CF valid and practical?Flip rate, realism, flips inside and outside the GT region, and CF fitted on the train split with true or predicted labels3958, 8Wvy, iW7Y
Is the threshold justified?Spread across random draws, sensitivity to ε, and class sample size8Wvy, iW7Y, YzHc
Other baselinesBaseline-corrected response and a near-uniform baselineiW7Y
SOTA models and more real dataMantis, HYDRA and ModernTCN on CricketX, UWaveGestureLibraryX and EpilepticSeizureYzHc

Main findings. Evaluators that pass the audit rank explanations as well as the others over the whole dataset, and in most cases they also rank well within the class where the others fail. The NULL response depends on where the perturbation removes evidence, not on the shape of the perturbation. The CF still works when it is fitted on the train split, without test data or test labels. The threshold ε is larger than the spread across random draws, and our conclusions do not change with ε or with the class sample size. Finally, the class-level failure also appears in state-of-the-art models on new real datasets.

Limitations we acknowledge. We also report what does not work. First, with NULL, a plausible pattern that the classifier encodes but does not need for its decision can look better than random, as Reviewer iW7Y suspected. This does not happen when the classifier truly ignores the pattern. Second, the CF is less realistic on MC-1v5 and PP, and there it also changes points outside the GT region. Third, IG and FA are imperfect references on real data. These results are the reason why we recommend auditing each configuration instead of trusting one fix.

Revision plan. In the revised version, we are making the following changes.

  1. We will add the missing related work (Baer et al., 2025; Schlegel & Keim, 2023; Šimić et al., 2025; Sturmfels et al., 2020; Hooker et al., 2019; Rong et al., 2022; Theissler et al., 2022) and explain how our audit differs from it.
  2. We will clarify that the audit tests the evaluator, not the attribution.
  3. We will add all new experiments to the appendix.
  4. We will state the limitations above explicitly.
  5. We will present the extended audit as three gates, which are the meaningful-versus-random check, the ranking test and the shortcut test.

We hope these additions address the reviewers' concerns, and we are happy to answer any further questions.

AuthorsOpenReview ↗

Final summary of the discussion

Dear Reviewers and ACs,

We thank all Reviewers and the ACs again for their time and for the constructive discussion. All four Reviewers see value in the problem that our paper studies. Reviewers 3958, 8Wvy and YzHc call it important, and Reviewer 3958 finds it "interesting and worth investigating". Reviewer iW7Y writes that the audit "exposes a real blind spot", and after our reply, notes that our comparison with prior work "makes the novelty clearer".

Since our first message, the reviewers have also recognized the value of the new experiments. Reviewer 8Wvy writes that the new CF, sensitivity and ranking experiments "strengthen the paper" and recognizes "the value of the class-conditional diagnostic findings". Reviewer 3958 finds "the new experiments valuable".

We are grateful that Reviewers YzHc and iW7Y raised their scores, as the new experiments address most of their concerns. These experiments include state-of-the-art models on new UCR datasets, the sensitivity to ε, the shortcut test, the CF fitted on the train split, and the ranking test on FPH and MIT-ECG.

We sincerely thank Reviewers 8Wvy and 3958 for their expert reviews and their close engagement throughout the discussion, which have improved our paper. Their remaining concerns are whether GT from the data design or from experts matches what the model actually uses, whether NULL and CF stay faithful to the original classifier rather than adding new biases, and how much our audit adds beyond existing meta-evaluation.

We fully acknowledge these concerns. To our knowledge, the first two are open problems in XAI evaluation, and especially in time-series XAI. No benchmark can yet show what a black-box model truly uses, and the evaluators that the community already uses, such as MoRF and LeRF with standard baselines and F-Fidelity, can carry their own biases. We therefore state these points as limitations. We also present our audit not as a full meta-evaluation but as a simple first step. Without this step, an evaluator that looks good on the whole dataset can hide a failure within one class. Its scores can then lead users to trust misleading explanations in that class, which is dangerous in high-stakes settings such as ECG monitoring, where our audit finds such failures in the normal class. Our audit warns users about these failures before they trust the scores.

We have uploaded the revised paper, where all new content appears in blue. Because the time for the revision was short, this version keeps the new results in full. We appreciate Reviewer iW7Y's suggestion, and we will make this content more compact and easier to overview in the final version.

We thank all Reviewers and ACs once more for their insightful feedback.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.