Faithfulness Is Conditional: Auditing Perturbation-Based Explanation Evaluation in Time Series

Deep learning plays an important role in time-series classification (TSC), but its black-box nature motivates explanations that help users understand and inspect predictions. *Faithfulness* is a central criterion for evaluating these explanations, yet the suitability of the evaluator itself also requires examination. In particular, dataset-level results do not establish whether an evaluator remains informative within each class. We propose a *simple class-conditional audit that compares informative reference attributions with random rankings*. Across controlled synthetic datasets and an ECG benchmark, we find that *an evaluator can distinguish these attributions overall while failing the same comparison within particular classes*. Motivated by these findings and studies of animal perceptual decision-making, we examine perturbation baseline predictions and representations, identifying baseline class preference as a possible contributor to these failures. We introduce a *NULL response* that changes how the model evaluates baselines by training an additional output while keeping the original representation and real-class logits frozen. We also investigate *input-dependent counterfactual (CF)* as baselines that change the replacement input itself, using a transductive, label-informed construction. Both strategies improve meaningful–random separation in several settings, but neither passes the audit across all tested configurations. We therefore treat NULL and CF as two options whose use is guided by the class-conditional audit. *This audit acts as an initial gate for selecting evaluation configurations* for each trained model, dataset, and intended class scope before their scores are used to compare explanation methods. Although the audit does not certify explanation correctness or replace more comprehensive evaluation, this audit provides evidence of simple but mandatory meaningful–random explanation separation.