acceptodds
Under review as a conference paper at ICLR 2027

When Graph Ablations Change Nothing: Locating Where Time-Series Anomaly Detectors Absorb a Graph Replacement

Abstract

Graph-based time-series anomaly detectors are often ablated by replacing their graph with a reversed or random one; when detection is unchanged, the graph is declared irrelevant. Similar ablation outcomes, however, can arise from different internal responses: the replacement can be absorbed at execution, predictive information, utilization by the fitted model, or anomaly scoring. We introduce layered graph-ablation diagnosis, an executable protocol with one measurement and one decision rule per layer that distinguishes these responses and names the component to examine next. At the information layer, we identify a history information retention mechanism: older states of a replacement neighbor can still predict the current target, and in Gaussian directed -cycles under the specified forecasting interface a reversed neighbor first becomes predictive at lag , as window and lag interventions on trained forecasters confirm. On a pre-registered benchmark of twelve planted absorbers, four cases have ablation contrasts within yet absorbers at three different layers; blind rules recover 101 of 120 labels, with all misses in two partial-utilization cases, whereas importance readouts such as GraphMask confound the layers. Applied to public detectors, the execution check finds that four of seven implementations ignore a supplied graph when they learn the graph. On a directed cycle, the diagnosis locates GDN's near-chance detection in its source selection and static attention, and replacing both lifts AP to 0.967. On SMD, it uncovers and repairs a prediction deficit hidden behind a null learned-versus-random contrast, a contrast that stays unresolved after the repair, consistent with redundant sources.

Primary area:
learning on time series and dynamical systems
Keywords:
time-series anomaly detection, graph neural networks, graph ablation, evaluation methodology, layered diagnosis, history information retention

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_VmGfOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates why replacing the graph in a time-series anomaly detector sometimes leaves detection performance unchanged. It proposes a four-layer diagnostic procedure covering graph execution, predictive information, model utilization, and anomaly scoring. The authors provide theoretical analysis for directed cycles, test the procedure on planted failure cases, and apply it to several existing detectors and real datasets.

Critical strengths
  1. The paper addresses an important methodological problem. An unchanged anomaly score does not necessarily mean that graph information is useless. Separating graph execution, information availability, model utilization, and scoring offers a more informative alternative to conventional ablation analysis.

  2. The study goes beyond conceptual discussion. The diagnostic rules recover 101 of 120 planted labels, identify graph interfaces that ignore supplied replacements, and lead to concrete changes in GDN. The authors also report unsuccessful interventions rather than presenting every diagnosis as a performance improvement.

Critical weaknesses
  1. The diagnostic rules are less reliable when multiple mechanisms influence the same outcome. All nineteen errors in the planted benchmark occur in partial-utilization cases. Since real detectors can contain several interacting bottlenecks, the accuracy of the diagnosis outside single-absorber settings remains uncertain.

  2. The evidence for practical intervention benefits is mixed. The controlled-cycle experiment shows a substantial improvement, but the SMD repair mainly improves prediction without a corresponding significant improvement in anomaly detection. This limits the evidence that the proposed diagnosis consistently leads to better detectors.

Questions for authors
  1. How should the diagnostic procedure interpret cases in which two or more layers simultaneously absorb a graph replacement?

  2. When the identified repair improves prediction but not anomaly ranking, what criterion determines whether the diagnosis has been practically useful?

5 comments
AuthorsOpenReview ↗

Response to Reviewer VmGf (1/2)

Thank you for the positive assessment and for two pointed questions. They led to the two additions we consider most important: per-layer verdicts when several layers absorb at once, and a criterion for when a diagnosis has been useful.

W1 / Q1. How should the procedure interpret cases where two or more layers absorb the same replacement?

Every layer now returns its own verdict, and the report keeps the full verdict vector; the first absorbing layer gives only a summary label. With two absorbers planted at once, both are flagged in 29 of 30 runs.

How the procedure reads it. The summary label is the first absorbing layer in the order execution → information → utilization → scoring; the report keeps every layer's verdict, including partial retention at the information layer, together with u and ΔAP (§3.1, Algorithm 1). The submitted rules let any AP change override the earlier verdicts (a "scoring-route precedence"). We removed this override, because it caused one of the two partial-utilization misses.

Two absorbers at once (new, App. I.1). Three cases plant two absorbers: diluted static attention plus noise bursts (utilization and scoring), history retention plus bursts (information and scoring), and a symmetrizing detector on a retaining graph (execution and information). The per-layer verdicts flag both planted layers in 29/30 runs (pre-registered criterion: at least 24). The summary label names the first planted layer in every run of the first and third cases. In the retention case the replacement keeps 26% of the gain while AP still falls; the frozen rule R2 summarizes this as "retained" and names information (30/30 overall), whereas the rule of §3.1 reports partial retention and names a later layer (20/30 overall; Table I.3). Both readings are in the verdict vector, which is why we report the vector, not a single label.

Accuracy with single absorbers. All 19 planted misses came from how the rules read the utilization measurement: an absolute threshold, and the precedence. Reading utilization relative to the available information, u = e/g < 0.5, was frozen after the first run and confirmed on fresh seeds (20/20 partial-utilization runs; R2 120/120 overall, against 101/120 for the pre-registered rules, which we keep as the reported benchmark result) and on partial-utilization mechanisms outside the benchmark whose ground truth does not use u (120/120, against 80–100/120 for the absolute threshold; App. I.1). The rule of §3.1 (fixed after these runs) applies the same relative criterion to the gain the replacement removes.

Utilization as a degree. Over a sweep of distractor strength, u falls with the replacement's effect on AP (Spearman 0.97). At seven distractors the graph is utilization-limited while the replacement still lowers AP by 0.47, so every label is reported with u and ΔAP rather than alone (Table I.4).

A real detector with several absorbers. SMD's GDN is such a case, and the revised §6.2 reports it layer by layer rather than with one label: 4 of 25 objects have no available cross-channel information, on 6 GDN's selector misses it, on the other 15 GDN does not use it (its graph path degrades normal prediction), and the score hides cross-channel deviations. Each verdict names the component to examine next.

AuthorsOpenReview ↗

Response to Reviewer VmGf (2/2)

W2 / Q2. When a repair improves prediction but not ranking, what criterion determines whether the diagnosis was useful?

A diagnosis is useful when it predicts, before the run, which change will move the outcome and which will not (§7). We fixed 11 such predictions in advance and count all of them; 9 held.

The case you describe: the root weight on SMD. It closes at least 91% of GDN's normal-risk gap, yet SMD's AP does not rise (0.380 against 0.394). Rerun after the repair, the diagnosis says why and where to act next: the graph is now used on only 5 of 25 objects (rule R0), 4 objects have no available cross-channel information and on 6 GDN's original sources miss it, and on injected cross-channel anomalies the official max-over-channels score absorbs most of the signal (+0.108 on the injected channels, +0.018 under the official score). This explains why improving prediction alone did not move the ranking, and it predicted, before the run, that changing the score would. Changing the score then moved the outcome on the anomalies it targets, as predicted (+0.094 [+0.022, +0.177]). SMD's own labels are mostly single-channel deviations, where a cross-channel score helps only on objects that self-history detects poorly (Spearman −0.60 with the self-history AP), so we recommend a secondary score that keeps both streams (App. I.3).

All predictions tested (11 in three studies; also listed in App. I.8).

  • Cycle: a selector gradient restores detection (held: +0.417 [+0.399, +0.434], 20/20 seeds); removing only the gradient from the trainable selector loses it (held: −0.278); the attention change alone does not restore it (held: +0.001 [−0.011, +0.013]).
  • SMD, injected cross-channel anomalies: a label-free contrast score lets informative sources reach the ranking (held: +0.094), widens the learned-minus-random contrast of GDN's sources (held: +0.011 to +0.048), and does so beyond the official score (held: +0.037); for repaired GDN, whose learned graph has no advantage even when only the injected channels are read (+0.002), changing the score does not create one (held: +0.001 [−0.009, +0.012]).
  • SMD, GDN utilization repair: an attention-free linear source path trained jointly with GDN would lift u above 0.5 (not held: median u from −8.13 to 0.13) and let GDN's learned graph beat a random one (not held: +0.006 [−0.007, +0.025]); under the official score the contrast stays within ±0.01 (held), and without the path the learned graph does not beat the random one on the injected channels (held at the edge of its tolerance: +0.010).

What the two misses show. The repair moved the measured gap without crossing the threshold, and the outcome did not move; with the path, GDN's selector also picked weaker sources (App. I.4). A located layer tells where to act; it does not guarantee that the first repair closes it. To place the remaining absorption, we kept the sources fixed and changed only the path that reads them (App. I.8): read by a ridge path fitted on normal data, informative (Granger-selected) sources are used (median u 0.73) and beat random ones under a full-channel contrast score on fresh injections (+0.105 AP, p = 0.002), and GDN's own learned sources beat random ones on the injected channels (+0.070, secondary endpoint). Under the same score, three changes to GDN's own message path do not, so the absorption sits in GDN's message path, not in its sources. These later repairs met 2 of their 6 primary endpoints; App. I.8 lists all of them.

In short: by this criterion the diagnosis was useful on the cycle and on SMD's injected anomalies, and where a first repair fell short it still located the remaining absorption. On SMD's own labels the learned graph does not beat a random one after any repair; the abstract, §6.2, and the conclusion say so.

We thank the reviewer again for questions that sharpened the paper, and we hope the revision strengthens your confidence in it.

AuthorsOpenReview ↗

Gentle Follow-up Before the Discussion Closes

Dear Reviewer VmGf,

Thank you again for your constructive review and positive assessment of our work. As the author–reviewer discussion period closes soon (November 18 AoE), we gently ask whether our response and the revised PDF address your two questions—on interpreting simultaneous multi-layer absorption via per-layer verdicts, and on the pre-registered intervention criterion when a repair improves normal prediction without moving the ranking. We would be happy to clarify any remaining point before the window closes.

Best regards, The Authors

Reviewer_WcKJOpenReview ↗
Rating
1(−1)
Confidence
3(+1)
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper investigates why learned graphs and random graphs can yield similar anomaly detection performance. The authors propose a four-stage diagnostic framework and evaluate it through simulations and real-world experiments. They identify several implementation and model limitations but do not fully explain the similar detection performance observed on real-world data.

Critical strengths
  1. The paper provides some theoretical support for its claims.

  2. The paper addresses a research question that is worth exploring.

Critical weaknesses
  1. Although I acknowledge the motivation, the authors do not provide sufficient evidence to establish how widespread this issue is. The phenomenon appears to exist, but how frequently does it occur? Is it observed in most settings?

  2. The paper’s organization makes it very difficult to follow. Many technical terms are used without being defined. For example, what is meant by “refitted AP contrast”?

  3. The authors attribute GDN’s unsatisfactory performance to the lack of direct gradients for optimizing the selected neighbors. They therefore modify the scoring mechanism, but this modification changes the attention computation rather than isolating the effect of these gradients.

  4. The authors attribute the similar performance of learned and random graphs to the presence of information. However, they do not rule out alternative explanations, such as poor graph quality, model collapse, or insufficient detector sensitivity.

  5. The Utilization results in Table D.3 are unsatisfactory. Do these results suggest that Utilization is not sufficiently reliable?

  6. In my view, the central objective of this paper is to explain the reasons behind the observed phenomenon. However, the paper does not provide a complete explanation or reach a conclusive answer.

Questions for authors

Please refer to Weaknesses.

4 comments
AuthorsOpenReview ↗

Response to Reviewer WcKJ (1/3)

Thank you for the detailed critique; it led to the largest changes in the revision. In brief:

  • How common (W1): null graph ablations are common, and most are unanswered rather than shown equal.
  • Presentation (W2): a Terms paragraph, consistent rule names and notation, and a one-row-per-case roadmap.
  • The GDN modification (W3): a 2×2 design isolates the selector gradient as what restores detection; the attention change alone has no effect.
  • Alternative explanations (W4): graph quality, collapse, and detector sensitivity were each measured on SMD; none explains the null on its own.
  • Utilization (W5): the measurement was reliable; the submitted rule read it too strictly.
  • A complete explanation (W6): on SMD, each of the 25 objects now has a located absorber, and each located layer was tested by changing it.

Each answer opens with a one-sentence summary in bold; confirmatory runs used protocols and analysis scripts frozen beforehand.

W1. How widespread is the issue?

Null graph ablations are common, and most of them are unanswered rather than shown equal.

Prevalence (new §6.1, Table 2, App. I.5). We ran the standard learned-versus-random ablation for official GDN and MTGNN on SMD, PSM, MSL, SMAP, SKAB, and SWaT (12 detector–benchmark pairs): ten initializations per arm, the better of two learning rates, up to 200 epochs, a two-stage bootstrap, Holm correction, and an equivalence test (90% interval inside ±0.01 AP). The two SMD pairs reuse the fits of our 25-object study (three initializations per object, intervals over objects). A pair is valid only if its detector beats chance (AP above prevalence and AUROC above 0.5); SMD, with AP 0.39–0.41 at prevalence 0.04, counts as valid without the formal test.

  • Valid pairs (SMD, SKAB, SWaT): the learned graph wins none of 6. Only 2 contrasts are equivalent (MTGNN on SMD and on SKAB); the other four still allow a learned-graph advantage of 0.015–0.247 AP. With the relation-based GReLeN on SKAB and SWaT (App. I.6), it wins none of 8: GReLeN's per-window graph loses to a fixed random graph on SKAB (−0.023 [−0.053, −0.005]) and is unresolved on SWaT.
  • Pairs at chance (PSM, MSL, SMAP). Neither detector beats chance, so these 6 ablations cannot speak to the graph at all. On MSL and SMAP this agrees with Kim et al. (AAAI 2022): without point adjustment, most detectors they reproduced score close to random and untrained baselines.
  • Execution. Handing a learned-graph checkpoint a different graph changes no prediction in any of the 10 newly trained pairs.
  • Location. From normal data alone, the diagnosis places the 6 valid nulls at information (4) and utilization (2), and GReLeN's SWaT null at information. GDN's learned graph predicts normal data worse than self-only support on 5 of the 6 benchmarks.

Reporting practice (App. I.7). Of 24 graph-based detectors whose ablations we could read, 15 ablate the graph and 13 of these judge it by an endpoint detection score alone; none reports normal-data prediction error, the information available, or whether the replacement was executed. In 16 public implementations that learn their graph, a supplied graph is ignored when graph learning is on (6), used only as an initialization (2), or not accepted (8).

So an unchanged score is common, and on its own it cannot say whether information was missing, unused, or hidden by the score, or whether the ablation could detect anything at all. That is the gap the diagnosis fills.

W2. Hard to follow; technical terms used without definition (e.g., "refitted AP contrast").

We agree, and revised as follows.

  • §3 opens with a Terms paragraph. The refitted AP contrast is ΔAP = AP(original graph) − AP(replacement), each detector trained and calibrated on its own graph with the same data, budget, and score. A layer absorbs a replacement when it limits the replacement's effect on what the layer passes on (possibly partly). A replacement reaches the ranking when ΔAP is resolved.
  • The introduction defines the degree-matched random graph and the refitted contrast at first use; §3.1 defines executed edges, the native graph, and self-only support where they first appear; Table C.1 lists the rest.
  • Real-data labels use the one rule of §3.1 unless marked; Table I.3 compares it with the earlier versions. The guard length and switch time, which clashed with the gain g and the tolerance τ, are now h and s.
  • The paper follows one path: the four layers and their rules (§3), validation on planted absorbers with known answers (§4), a controlled cycle where we locate and repair GDN's absorption (§5), and public benchmarks, then SMD layer by layer (§6). Table 1 previews it with one row per case.
AuthorsOpenReview ↗

Response to Reviewer WcKJ (2/3)

W3. The GDN modification changes the attention computation rather than isolating the gradient.

The gradient, not the attention change, restores detection. The reviewer is right that the trainable selector changed two things at once: it gave the source scores a loss gradient and added a gate term to the attention logit. We separated the two in a 2×2 design (gradient × gate term), plus one arm that removes only the gradient, on the same 20 data seeds and shared initial parameters (§5.2, App. I.2, Table I.6):

ArmSelector gradientGate term in attention logitAP
Native GDNnono0.531
Native selection + gatenoyes0.532
Trainable selector, gradient removednoyes0.672
Straight-through selector, GDN's static attentionyesno0.947
Trainable selectoryesyes0.949

Gradient only: +0.417 [+0.399, +0.434], better on 20/20 seeds. Gate only: +0.001 [−0.011, +0.013]. Removing only the gradient from the trainable selector: −0.278 [−0.287, −0.269], worse on 20/20 seeds. Contract checks confirm that the no-gradient arms receive zero selector gradient and that the straight-through forward pass equals GDN's static attention (bitwise on CPU). The no-gradient arm sits above native GDN because its random initial sources stay fixed during training, and fixed sources within a cycle retain some lagged information. (Native GDN's AP on this cycle is 0.531 here and 0.537–0.545 in our other studies; every contrast above is paired within this study.)

W4. Alternative explanations: poor graph quality, model collapse, insufficient detector sensitivity.

We measured all three on SMD. None explains the null on its own: graph quality is what the information and utilization layers quantify, and the sensitivity test shows that cross-channel anomalies are visible to a forecaster reading GDN's sources, but GDN's message path and the max score hide them.

  • Graph quality: partial, not redundant (App. I.1 and I.3; Tables I.3, I.7). On 4 of 25 objects no source set (all 38 channels or the best four) adds 1% to self-history, so there is nothing for any graph to add. On the 14 objects where a ridge fit on all 38 channels adds at least 1%, GDN's learned sources keep a median 40% of the best four sources' gain, against 32% for four random sources and 83% for Granger-selected ones: better than random, but only part of the information. GDN's graph path then predicts normal data worse than self-only support on all 25 objects (MSE 0.368 against 0.194). This replaces the submitted reading that SMD's sources are redundant: 0 of 25 objects are.
  • Collapse: none. For the original GDN on its learned graph, the median per-node correlation between prediction and target on normal data is 0.86 (lowest object 0.45); for the repaired model it is 0.97, with 84–86% of the target variance kept (App. I.3).
  • Detector sensitivity: the anomalies are visible, the score hides them. We injected anomalies that only break cross-channel relations into SMD's normal data. Read by a ridge forecaster on the injected channels, informative sources beat self-history by +0.108 AP [+0.032, +0.205], and GDN's learned sources beat its random ones by +0.054 [+0.011, +0.123]; repaired GDN's own learned graph beats its random graph by only +0.002. The official score, a maximum over 38 channels, shrinks the two ridge contrasts to +0.018 and +0.012. GDN's message path and the score hide the signal, which the diagnosis labels absorption at utilization and at scoring. In the atlas, ablations whose detector does not beat chance are reported as uninformative rather than as nulls (W1).
AuthorsOpenReview ↗

Response to Reviewer WcKJ (3/3)

W5. The utilization results (Table D.3 of the submission) are unsatisfactory. Is utilization not reliable?

The utilization measurement was reliable; the submitted decision rule read it too strictly (details in our reply to Reviewer y9vN, W1). Read relative to the available information, u = e/g < 0.5, the utilization layer flags 20/20 partial-utilization runs (10/20 for the absolute threshold e < τ), with 0/30 false alarms. Frozen after the first run, this reading was confirmed on fresh seeds 11–20 (120/120 overall for the frozen rule R2, against 101/120 for the pre-registered rules, whose misses again all fall in the two partial-utilization cases), on new mechanisms whose ground truth does not use u (120/120, against 80–100/120 for the absolute threshold), and on multipath systems where utilization is set exactly (464/480 conditions for the §3.1 rule, every miss within 0.1 of the threshold; App. I.1). Table D.3 keeps the pre-registered labels; Table I.3 lists every result under every rule version.

W6. The paper does not fully explain why learned and random graphs perform similarly on real data.

For SMD, the revision now gives a full account: each of the 25 objects has a located absorber (4 + 6 + 15), and each located layer was tested by changing it. We agree that the submitted version stopped short of this. The null has no single cause; the learned graph's possible advantage is lost at successive layers (§6.2; App. I.1, I.3, I.4, I.8, F.3):

LayerFinding on SMD (25 objects)Changing that layer
Information4 objects: no source set adds 1% to self-historynothing for a graph to add
Selection6 objects: GDN's sources miss the information that existstrainable selector + GATv2 + root weight: graph used on 9 objects (rule R0), but SMD's AP does not rise (0.358 against 0.394, unresolved)
Utilization15 objects: GDN does not use what its sources carry; its graph path predicts normal data worse than self-only on all 25a ridge path fitted on normal data reads Granger-selected sources at median u 0.73, and they beat random sources by +0.105 AP (p = 0.002) under a full-channel contrast score; under the same score, three changes to GDN's message path do not
Scoringthe max over 38 channels shrinks the injected cross-channel signal from +0.108 to +0.018 APa label-free contrast score recovers it: +0.094 [+0.022, +0.177], predicted before the run
Anomaly typeSMD's labels mostly mark single-channel deviationswith the fitted ridge path, learned minus random sources on these labels: −0.003 [−0.033, +0.022]

The utilization row in detail. A first repair, a linear path trained jointly with GDN, moved GDN's median u from −8.13 to 0.13, short of the 0.5 fixed in advance (App. I.4). With the fitted ridge path, GDN's learned sources also beat random ones on the injected anomalies (+0.070 [+0.031, +0.117], secondary endpoint; App. I.8). The remaining absorption therefore sits in GDN's message path, not in its sources or its score.

SMD's own labels. The contrast score lowers AP on the half of the objects whose anomalies self-history already detects best and raises it on the other half (−0.085 and +0.138; Spearman −0.60 [−0.81, −0.25] with the self-history AP). A secondary score that takes the larger of the official and contrast streams keeps both (−0.010 and +0.052; App. I.3).

In short: on SMD the learned graph does not beat a random one because many objects offer little cross-channel information, GDN finds only part of it and does not use what it finds, the score hides cross-channel deviations, and SMD's labels mostly do not need them.

We hope the revision now addresses your concerns about soundness and presentation. If it does, we would be grateful if you would consider updating your assessment; we would be glad to clarify any point further.

Thanks for authors' rebuttal. Yet I found that in Appendix F.3, the number of information-limited cases varies substantially with the choice of information estimator and diagnostic threshold. I think this suggests that the identified absorption mechanisms may depend heavily on specific experimental configurations rather than reflect intrinsic properties of the detectors. What evidence supports the claim that the proposed framework identifies general mechanisms underlying unchanged anomaly detection performance after graph replacement, rather than merely producing classifications that depend on specific diagnostic configurations?

Reviewer_y9vNOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates why graph replacement may leave the performance of graph-based time-series anomaly detectors unchanged. It proposes a four-layer diagnostic framework covering execution, information, utilization, and scoring, and theoretically analyzes history information retention in directed cycles. Synthetic and real-world experiments validate the framework's ability to distinguish different absorption mechanisms and identify architectural bottlenecks.

Critical strengths

Theoretically grounded explanation. The paper provides a theoretical characterization of history information retention, including the exact predictive lag in directed cycles. This offers a principled explanation for why graph replacement may preserve predictive information despite changing the graph structure.

Novel diagnostic perspective. The paper distinguishes four mechanisms behind unchanged graph-ablation results, challenging the conventional interpretation that unchanged performance implies graph irrelevance.

Clear distinction between information availability and utilization. The framework explicitly separates the predictive information available in graph structure from the fitted model's ability to exploit it. This distinction enables a more precise interpretation of graph contributions and avoids conflating model limitations with information irrelevance.

Critical weaknesses

Limited diagnostic accuracy in partial-utilization cases. The framework correctly identifies only 1 of 20 partial-utilization cases. Although it detects some utilization issues, its final decision rules often assign incorrect labels, limiting its diagnostic accuracy.

Limited applicability across detector architectures. The information and utilization layers are formulated around forecasting-based risk comparisons and self-history baselines. This makes the complete diagnostic protocol less directly applicable to reconstruction-based or relation-based anomaly detectors, limiting its generality across different detection architectures.

Insufficient evaluation of computational scalability. Although Appendix D.2 outlines the computational requirements of individual diagnostic steps, the paper does not provide a quantitative comparison of the overall runtime or computational cost against standard graph ablation. This leaves the scalability of the complete diagnostic protocol to larger and more complex detectors insufficiently evaluated.

Questions for authors
  1. Clarification, Reliability of intermediate diagnoses. Given that only 1 of 20 partial-utilization cases receives the correct final label, can the authors clarify how reliably the intermediate layer-level diagnostics identify the underlying utilization bottlenecks, and what evidence supports this reliability?

  2. Clarification, Scope of the diagnostic framework. Does the proposed four-layer protocol currently support reconstruction-based or relation-based anomaly detectors, or is its complete implementation restricted to forecasting-based detectors? Please clarify the intended applicability of the information and utilization criteria.

5 comments
AuthorsOpenReview ↗

Response to Reviewer y9vN (1/2)

Thank you for the review. We are glad you found the four-mechanism perspective novel and the separation of information availability from utilization clear. Your three concerns, the reliability of the utilization verdict, scope beyond forecasting, and cost, each now have an answer from a study whose protocol was frozen before the confirmatory run (Appendix I of the revised PDF).

W1 / Q1. Partial utilization: 1 of 20 final labels correct. How reliable are the layer-level diagnostics?

The layer-level measurement identified the bottleneck in all 20 partial-utilization runs once utilization is read relative to the available gain. The misses came from the submitted decision rule, not from the measurement.

Layer-level evidence (Q1). Let g be the predictive gain the graph makes available on normal data and e the normal-risk gain the fitted detector realizes over self-only support. On the utilization layer alone, the submitted absolute threshold e < τ flags 10/20 partial-utilization runs (95% Wilson interval [0.30, 0.70]); the relative criterion u = e/g < 0.5 flags 20/20 ([0.84, 1.00]). Neither raises a false alarm among the 30 control runs that reach this layer.

Why the final labels missed. One rule effect per case:

  • P08 (GDN's top-k selector): the utilization layer flagged it in 9 of 10 seeds, but AP still moved (mean 0.041), and a precedence rule then relabeled the case a scoring route.
  • P09 (static attention among 23 distractors): the model realizes 1.3% of a 33% available gain, just above the absolute 1% threshold.

Confirmation. We now read u < 0.5, drop the precedence, and report every layer's verdict, with the first absorbing layer as the summary label. Because we proposed this after seeing seeds 1–10, we froze it and tested it three ways:

  • Fresh seeds 11–20, unchanged benchmark. The pre-registered rules again give 101/120 with the same misses, so the misses are systematic; the relative rule (R2) gives 120/120 ([0.97, 1.00]).
  • Mechanisms outside the benchmark (noisy messages, an attention penalty, early stopping), with a ground truth that does not use u: a run is limited when the detector realizes less than half of the gain the same detector realizes without the mechanism. The relative rule classifies 120/120 runs; the absolute threshold classifies 80, 90, and 100 at τ = 1, 2, and 5% (exact McNemar p ≤ 2×10⁻⁶ at every τ).
  • Exact references (Table I.5). On two multipath VAR systems whose utilization is set analytically, including u = 0.5, the §3.1 rule's normal-data verdicts are correct in 464 of 480 conditions: all 360 away from the threshold, with every miss within 0.1 of u = 0.5. The measured u has a mean absolute error of 0.0095.

With two absorbers planted at once, both are flagged in 29/30 runs. Across distractor strengths, u falls with the replacement's effect on AP (Spearman 0.97), so we report u with every label (Table I.4). The pre-registered 101/120 remains the reported benchmark result (§4, Figure 2).

Rule versions in one line. The §3.1 rule uses the same u < 0.5 criterion and differs from R2 only at the information layer, where it reports partial retention: in P06 and P07 the replacement keeps 26% and 70% of the gain while AP still falls (by 0.04 and 0.10), so it scores 100/120 against a planted key that calls these cases "retained" (Table I.3).

Where: §3.1, Algorithm 1, §4, App. I.1 (Tables I.1–I.5).

AuthorsOpenReview ↗

Response to Reviewer y9vN (2/2)

W2 / Q2. Does the protocol support reconstruction- or relation-based detectors?

Yes. Each layer is measured through the detector's own interface and loss, not through forecasting, and we ran the full protocol on one relation-based and one reconstruction-based detector (App. I.6). Execution (executed edges and predictions in a frozen checkpoint) and scoring (residuals → score → AP) are model-agnostic. Utilization compares the detector's own training loss on its graph and on self-only support. The information probe reads what the interface passes: lagged values for a forecaster, compressed codes for a reconstruction model (§3.2).

  • Relation-based GReLeN, planted cycle (utilization on GReLeN's own NLL). Parent and reversed graphs execute the same neighbor sets through bidirectional diffusion, so AP does not change (+0.0005): absorbed at execution. A random graph reaches the ranking (+0.719). Native GReLeN learns the cycle (edge probability 0.91 on parent and child edges, 0.55 otherwise), yet its relation score stays at prevalence: a reversal leaves the undirected relations unchanged, so the graph is used and the anomaly is absorbed at scoring.
  • Relation-based GReLeN, real SKAB and SWaT (new; Table I.12 and the GReLeN rows of Table 2). Released model and settings, ten initializations per arm, protocol and analysis frozen before any test score. GReLeN beats chance with its residual score (AP 0.757 at prevalence 0.36; 0.466 at 0.12), while its own relation score stays at chance without point adjustment (AUROC 0.50 and 0.28). Its per-window graph does not beat a fixed random graph on either benchmark (SKAB −0.023 [−0.053, −0.005], in all ten initializations; SWaT −0.169 [−0.331, +0.016]). Normal data place both at the information layer. SKAB offers no cross-channel information (best-four ridge gain −0.002), and the fixed learned and random graphs are equivalent within ±0.01 AP; the per-window graph's loss there is a scoring route that normal data do not explain. On SWaT, diffusion along both edge directions lets the random graph execute as much information as the learned one (0.274 against 0.265), and removing the graph raises normal risk (u = 1.93): the graph is used, and the removal is absorbed at scoring. At execution, the reversed graph with the two diffusion directions' weights exchanged gives identical predictions in all 20 checkpoints.
  • Reconstruction-based graph autoencoder. On four planted replacements (execution; information, by history retention; scoring; reaches the ranking), blind labels match in 19/20 runs (Table I.11). Matching the probe to the interface matters: a probe reading raw source windows, which the autoencoder never receives, would misread it as utilization-limited.

The reconstruction-based detector was run on synthetic cycles only; the contributions and §7 state the scope.

W3. No quantitative runtime comparison with a standard graph ablation.

The diagnosis costs 1.43–1.50 times the ablation, and the ratio stays flat from 32 to 256 channels. We timed every step of Algorithm 1 for GDN (GATv2 score) on directed cycles, median of three repetitions, GPU idle at the start (App. I.6):

ChannelsAblation (2 fits + scores)Added by the diagnosisof which self-only fitAll other checksDiagnosis / ablation
3222 s11 s10 s0.7 s1.48×
6422 s11 s10 s0.9 s1.49×
12823 s12 s10 s1.5 s1.50×
25630 s13 s10 s2.6 s1.43×

The diagnosis adds one fit, on self-only support; all other checks together take under 3 s. An optional search for the best four sources, used only to split an information-limited label into "no information available" and "available but not selected", is timed separately (3.2 times the ablation at 64 channels, 22 times at 256). For GReLeN the diagnosis again adds one fit, about 1.5×. The cost appears in the abstract and §7.

We hope these answers resolve the three concerns. If they do, we would be grateful if you would consider revisiting your assessment; we are glad to clarify anything further.

AuthorsOpenReview ↗

Gentle Follow-up Before the Discussion Closes

Dear Reviewer y9vN,

Thank you again for your time and thoughtful review. As the author–reviewer discussion period closes soon (November 18 AoE), we gently ask whether our rebuttal and revised PDF address your concerns regarding the reliability of the utilization verdict, applicability to relation- and reconstruction-based detectors, and computational cost. We would be glad to clarify any remaining point while there is still time.

Best regards, The Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response to all reviewers and the AC

We thank the reviewers for three careful reviews; the four questions they share are answered below.

The claim. A graph ablation that leaves detection unchanged is usually read as "the graph does not matter." An unchanged score says only that the replacement was absorbed somewhere, and there are four places: the code never executes it, it keeps the predictive information, the model does not use the information, or the score hides the change. Layered graph-ablation diagnosis tells these apart with one measurement per layer, at about 1.5 times the cost of the ablation, and each verdict names the component to examine next.

Question in the reviewsShort answer
How common are null ablations? (WcKJ)Common, and mostly unanswered rather than shown equal (§6.1)
Is the utilization verdict reliable? (all)The measurement was; the submitted rule read it too strictly (§3.1, App. I.1)
Why do learned and random graphs tie on SMD? (WcKJ)Each of the 25 objects now has a located absorber (§6.2)
Does acting on the located layer help? (WcKJ, VmGf)Yes where the located component was replaced; a substitution test locates the rest (§5.2, App. I.8)

Every new confirmatory run used a protocol and analysis script frozen beforehand (Appendix I).

1. Null ablations are common and mostly unanswered. In a new atlas of GDN and MTGNN on six benchmarks, the learned graph never beats a degree-matched random graph in the 6 pairs whose detector beats chance, yet only 2 contrasts are equivalent within ±0.01 AP; the rest leave learned-graph advantages of up to 0.247 AP open. In the other 6 pairs (PSM, MSL, SMAP) the detector itself is at chance, so the ablation cannot speak to the graph.

2. The measurement was right; the reading was too strict. The pre-registered benchmark gave 101/120, all 19 misses in the two partial-utilization cases, and all from how the rules read utilization: an absolute threshold, and a precedence that let any AP change override it. Read relative to the available gain, u = e/g < 0.5, the utilization layer flags all 20 partial-utilization runs (10/20 for the absolute threshold) with no false alarm in 30 controls. Frozen after the first run, this reading was confirmed on fresh seeds (20/20), on new mechanisms whose ground truth does not use u (120/120, against 80–100/120 for the absolute threshold), and on multipath systems with analytically set utilization (464/480 conditions). The pre-registered 101/120 stays the reported benchmark result.

3. On SMD, every object has a located absorber. From normal data alone, the diagnosis accounts for all 25 objects: 4 offer no cross-channel information, on 6 GDN's sources miss the information that exists, and on the other 15 GDN does not use what its sources carry (its graph path predicts normal data worse than self-history on all 25). On cross-channel anomalies injected into SMD's normal data, the official max-over-channels score then shrinks the advantage of informative sources over self-history from +0.108 AP (on the injected channels) to +0.018.

4. Changing the located component moves detection.

  • Selector, controlled cycle. A selector gradient alone lifts GDN from 0.531 to 0.947 AP (+0.417, better on 20/20 seeds); the attention change alone gives +0.001.
  • Score, SMD injections. A label-free contrast score lets informative sources reach the ranking (+0.094 AP [+0.022, +0.177]), as predicted before the run.
  • Message path, SMD injections. With the sources fixed and only the path that reads them replaced by a ridge path fitted on normal data, Granger-selected sources are used (median u 0.73) and beat random ones by +0.105 AP (p = 0.002), and GDN's own learned sources beat random ones on the injected channels (+0.070, secondary endpoint). Under the same score, three changes to GDN's message path do not, so the remaining absorption sits in that path.

Of 11 pre-registered intervention predictions, 9 held; both misses come from a first utilization repair, which narrowed the gap without closing it (median u −8.13 → 0.13); later repairs, not counted, met 2 of their 6 primary endpoints (App. I.8). On SMD's own labels, mostly single-channel deviations, the learned graph still does not beat a random one after any repair; the abstract and §6.2 say so.

5. Scope, cost, presentation. The full protocol now also runs on a relation-based detector (GReLeN, including real SKAB and SWaT) and a reconstruction-based graph autoencoder, at 1.43–1.50 times the ablation's cost. §3 opens with a Terms paragraph.

What the revision claims. A null graph ablation can be located layer by layer, and the location predicts which change will move detection. It does not claim that learned graphs help on SMD's own labels, or that directed cycles model real sensor graphs.

Dear Reviewers,

This is a friendly reminder to check the authors' responses to your reviews and leave a follow-up comment if appropriate. Your engagement in the discussion is greatly appreciated.

Thank you!

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.