acceptodds
Under review as a conference paper at ICLR 2027

An Empirical Evaluation Framework for Auditing Explainable AI in Network Intrusion Detection

Abstract

Deep learning Network Intrusion Detection Systems (NIDS) suffer from a transparency gap that hinders operational trust. While Explainable AI (XAI) methods aim to bridge this, they often exhibit domain blindness, generating semantically invalid network states, such as fractional TCP flags that distort evaluation metrics. This paper proposes a modular evaluation framework centered on a novel Protocol-Aware XAI (PA-XAI) engine. By enforcing protocol invariants and hierarchical constraints, PA-XAI ensures that explanations are grounded in physically realizable telemetry. Our framework employs four auditing modules: functional faithfulness, adversarial robustness, algorithmic consensus, and dimensionality reduction utility. Empirical results across four benchmark datasets reveal that standard XAI metrics often mask significant instability; for example, LIME’s apparent stability is frequently an artifact of invalid neighborhood construction. Conversely, our PA-variants, specifically PA-DeepLIFT and PA-IG, demonstrate superior robustness against adversarial evasion. Furthermore, PA-XAI guides dimensionality reduction to achieve up to 90.5% feature compression with less than a 1% drop in detection fidelity. These findings demonstrate that domain-aware auditing is essential for transitioning XAI from a theoretical enhancement to a reliable component of operational network defense.

Primary area:
interpretability and explainable AI
Keywords:
Network Intrusion Detection Systems, Explainable AI, Protocol-Aware XAI, Deep Learning, Network Security, Adversarial Robustness, Model Interpretability, Feature Reduction, DeepLIFT, Integrated Gradients

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_E2ZFOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a new explanation framework PA-XAI, which enforces data constraints in explanations for network intrusion detection systems. The core hypothesis is that feature attribution methods don't consider validity constraints on inputs, and therefore produce misleading explanations that operators cannot trust. The paper modifies LIME, SHAP, IG, and DeepLIFT to consider these constraints, and compares them to their base-variants in different settings.

Critical strengths

The paper raises an interesting and underexplored problem. Without constraints, perturbation-based explainers evaluate models on inputs that can be outside the valid data range. This seems a relevant issue, even beyond the particular chosen setting. The paper shows that envofcing these invariants changes which explainers score best in the evaluation.

The empirical evidence is broad, including four datasets, three classifiers, and four explainers.

Critical weaknesses

The experiments don't support the central claim. The paper claims that PA outputs are "the corrected measurement," but PA changes the explainers themselves. Without ground-truth attributions, we cannot measure which explainer is better. PA-LIME’s negative faithfulness is equally consistent with PA-LIME being a worse explainer as with the paper’s reading that PA exposes LIME’s true unfaithfulness. The only thing the experiments can establish is that the two explainers differ, in both directions. Which one is better is assumed by definition without evidence.

The significant is limited. The paper motivates operator trust and practical application constraints in multiple places. However, it is never defined who the operator is, which decisions are influenced by the explanations, and how those decisions improve with the proposed framework. Furthermore, the feature extraction setting for NIDS is outdated. It relies on hand-crafted flow features from old lab-generated datasets (NSL-KDD was derived from 1998 traffic), and much of the recent work has moved towards models trained on raw traffic. It is unclear if the paper's conclusions transfer to more up-to-date settings.

Several headlines results look like artifacts. For example, LIME's perfect stability of J=1.0 on 2 datasets for all three classifiers. The consensus result is across related methods, especially IG and DeepLIFT, so high correlation is expected. Additionally, it is unclear which exact constraints were used, and if they only cover theoretically impossible aggregates, or also consider protocol validity (which can and often is violated by attackers).

Questions for authors

Please list the full constraint set per dataset. Are constraints enforced on the raw or the normalized feature space? Are there real training flows that violate these constraints, or could there be observed traffic that does, even malformatted?

Is the faithfulness measurement subject to the same constraints? If not, the faithfulness metric itself evaluates the model on invalid flows, which PA-XAI is meant to address.

How can we conclude that the explanations actually improved?

Reviewer_5aC3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a framework for auditing post-hoc XAI methods (SHAP, LIME, IG, DeepLIFT) applied to ML-based NIDS. Its core component, PA-XAI, modifies each explainer so that the inputs it generates respect protocol and feature constraints.

Critical strengths

The paper addresses a relevant and underexplored problem: the validity of post-hoc XAI evaluations when perturbations generate inputs that may violate domain constraints in network intrusion detection. Its broad empirical evaluation across four datasets, three model families, and multiple explainers provides a useful starting point for investigating this issue.

Critical weaknesses

The paper’s central claim that PA-XAI provides a “correct” evaluation framework is not fully supported. Since PA-XAI modifies the explainers themselves, the observed differences of standard methods and PA do not establish that the standard evaluations are distorted; they may instead reflect differences in explainer behaviour, potentially including worse explanations. Examples include the decrease in LIME stability on UNSW-NB15, contrasting with its increase on NSL-KDD, and the negative faithfulness scores of PA-LIME with DNNs. Without validation against ground-truth attributions or known decision rules, and without establishing that the occlusion-based faithfulness metric itself uses valid inputs, the main findings rely on an blurry interpretation of these differences. Furthermore, the constraints defining PA-XAI are not formally specified. Some motivating examples also appear inconsistent with the flow-level features used: simultaneous SYN and RST counts can represent valid and common connection-level patterns

Questions for authors
  1. Are the inputs used by the occlusion-based faithfulness metric guaranteed to satisfy the domain constraints enforced by PA-XAI? If not, how to justify this evaluation?

  2. What constraints are enforced for each dataset, how were they validated, and how do they account for valid flow-level patterns such as simultaneous SYN and RST counts?

  3. Can the authors validate PA-XAI against ground-truth attributions or known decision rules to demonstrate that it improves explanation validity rather than merely changing explainer behaviour?

Reviewer_yrm5OpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The study proposes a modular evaluation framework for auditing Explainable Artificial Intelligence (XAI) methods in machine-learning-based Network Intrusion Detection Systems (NIDS). The central motivation is that conventional XAI methods may generate misleading explanations because their perturbation and attribution procedures do not adequately respect the semantic and structural constraints of network traffic. The authors refer to this limitation as domain blindness and introduce a Protocol-Aware XAI (PA-XAI) engine designed to incorporate domain-specific constraints into explanation generation. The authors report that protocol-aware explanation generation substantially changes the measured faithfulness and stability of LIME. They also report improved attribution robustness for PA-IG and PA-DeepLIFT under adversarial perturbations and substantial dimensionality reduction with relatively small decreases in classification performance.

The authors try to emphasize that domain-aware explanation generation is valuable. However, I have significant concerns regarding the formal definition and validation of the proposed constraints, the lack of controlled experiments establishing improvements in explanation correctness, insufficient algorithmic specifications, and the reproducibility and interpretation of several experimental results. These concerns affect the validity of the central claims and should be addressed before the framework can be considered convincingly validated.

Critical strengths
  • Authors identify an important limitation of conventional explanation methods when applied to highly structured network telemetry.
  • The proposed evaluation is potentially valuable for systematic auditing.
  • The results show interesting observations such as Tables 1 and 2 reveal substantial changes in LIME faithfulness and stability under different explanation-generation procedures. Table 3 reports differences between constrained and unconstrained adversarial perturbations, while Figure 3 suggests that attribution-based feature selection can substantially reduce dimensionality.
Critical weaknesses
  • The central technical contribution of this paper is the PA-XAI engine. The authors claim that this engine enforces protocol invariants and generates explanations grounded in physically realizable network states. However, Section 3.2 describes the underlying mechanisms only at a high level, without specifying the complete admissible feature space, the constraint satisfaction procedure, or the guarantees of the repair operation.

  • Section 3.2 introduces Decision Boundary Anchoring, selecting a valid per-class sample nearest the decision threshold. However, the manuscript does not explain how the distance to the decision boundary is calculated, how candidate samples are selected, or how the procedure handles multiclass classification.

  • PA-XAI does not simply remove invalid samples from an otherwise unchanged evaluation procedure. It modifies the underlying explanation algorithms. These modifications can alter the attribution quantity being estimated, even when all evaluated samples are valid.

  • In Tables 1 and 2, the authors interpret faithfulness changes as evidence that conventional evaluation masks LIME failure. However, the reported results do not distinguish whether the differences originate from changes in neighborhood sampling, changes in feature-ablation procedures, or interactions between the explanation and evaluation distributions. Also, the reported averages in the tables are without sufficient uncertainty estimates or repeated-trial statistics

  • In Table 3, the results are potentially interesting, but the manuscript does not demonstrate that constrained and unconstrained examples satisfy comparable perturbation budgets or preserve the same attack objectives.

  • In Figure 2, the cosine similarity alone cannot establish whether explanations are correct or resistant to purposeful explanation manipulation. Attribution similarity can also be difficult to interpret when the adversarial example changes the classifier's prediction. Authors should quantify how constraint repair changes perturbation magnitude and attack success.

  • Table 5 shows mean Spearman correlations ranging from 0.256 to 0.309 across datasets; the authors report that PA-IG and PA-DeepLIFT achieve correlations above 0.91 on three datasets. This high agreement may partly result from using similar reference inputs rather than from independently identifying the classifier's important features. So, the authors should distinguish explanation consensus from faithfulness and avoid interpreting agreement as direct evidence of better explanations.

  • Section 6.2 claims that DNNs exhibit 1.5–3.3 times stronger interactions than Random Forests. However, the reported results in Table 6 for DNN and RF values do not directly substantiate this magnitude claim.

  • The manuscript should explicitly distinguish its methodological contributions from existing constraint-aware explanation and XAI auditing approaches in the literature.

Questions for authors
  • What complete set of protocol and cross-feature constraints is implemented for each dataset, and what independent evidence demonstrates that repaired samples represent physically realizable network traffic?

  • How can the authors distinguish improvements due to valid neighborhood generation from changes caused by different attribution baselines, coalition definitions, or evaluation distributions?

  • How are decision-boundary anchors selected, and how sensitive are PA-IG and PA-DeepLIFT to the anchor-selection strategy? Are the theoretical completeness properties of the original algorithms preserved?

  • How do the authors ensure that constrained FGSM and PGD attacks remain within the intended perturbation budget and preserve attack functionality? How are gradients obtained for Random Forest and XGBoost?

  • How are the 66 explainer pairs in Table 5 constructed, and what evidence shows that improved inter-explainer agreement corresponds to improved attribution correctness?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.