- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper aims to address the limitation of point-level evidence scores, namely drift scores and reconstruction scores, which are often temporally misaligned with anomaly labels. The paper proposes a two-stage approach consisting of a base prediction and a bounded correction residual. Experiments on various temporal interaction datasets validate the effectiveness of the proposed approach.
Critical strengths
-
The issue of evidence-label mismatch in dynamic graph anomaly detection is well identified, and the paper provides theoretical claims and proofs to support this observation. This helps clarify the limitation of point-level evidence scores and provides a solid motivation for the proposed method.
-
The paper is well organized and easy to read, with a clear logical structure that makes both the motivation and the technical details accessible to readers.
Critical weaknesses
-
Figure 1 does not explicitly show the evidence-label mismatch. Instead, this part appears on the far right, making the sentence "As illustrated in Figure 1,..." difficult to follow. Since this mismatch is the central motivation of the paper, separating it into a new figure would help readers understand the problem more directly and better appreciate the need for the proposed approach.
-
The architectures of the dynamic graph encoders used for drift scores and reconstruction scores are not clearly stated, which makes it difficult to understand how these two types of scores are computed. In addition, the reason for describing the encoder as memory-based is unclear. The underlying rationale for the base prediction should be elaborated, for example, what is stored in the memory, how it evolves, and how it contributes to the prediction.
-
For the bounded correction, the features are simply concatenated, with insufficient explanation of why this is reasonable. It would be helpful to discuss the role of each feature and to justify why concatenation is adopted rather than other fusion strategies.
-
The computation of the stage-1 score is not provided. The authors should specify how is computed from the two point-level evidence scores.
Questions for authors
-
Could the authors clarify whether the model is trained in an end-to-end manner?
-
In the problem setup, the meaning of the source-side label is unclear. Could the authors clarify whether it is the anomaly label?
-
For a new dataset, how is the number of preceding events determined? The sensitivity experiments indicate that anomaly-related temporal patterns occur at different local scales across datasets, so it would be helpful to explain how is chosen in practice.
8 comments
W2: Encoder architecture, the role of memory, and the base score
Reconstruction and drift share one memory-based encoder following SLADE [1]. They measure two forms of consistency within the same architecture.
1. A persistent memory summarizes each node’s interaction history
Each node maintains a memory vector that summarizes its interaction partners and timing. Interactions are processed chronologically: an MLP constructs messages from the other endpoint’s memory and elapsed-time encoding, and a GRU combines these messages with the node’s previous memory. Both endpoints are updated through this procedure.
The encoder is memory-based because these node states persist across interactions and accumulate historical information. Temporal attention additionally reconstructs a node’s memory from historical neighbor memories and relative-time encodings.
2. Reconstruction and drift compare different aspects of consistency
| Score | Compared representations | Meaning of low compatibility |
|---|---|---|
| Reconstruction | Maintained memory and reconstructed memory | Accumulated behavior agrees poorly with recent neighborhood context |
| Drift | Current memory and reference memory (the node’s memory before the current update) | The updated state agrees poorly with its pre-update state |
The scores are
=\widehat I(\hat m_i,m_i;\mathcal N^-), \qquad c_i^{\mathrm{drift}} =\widehat I(m_i,m'_i;\mathcal N^-).$$ The compatibility measure $\widehat I$ in Section 4.1 contrasts the paired vectors’ cosine similarity with similarities to negative node memories. Stage-1 training encourages agreement in both comparisons without anomaly labels. **Low compatibility therefore provides evidence of departure from the learned behavioral pattern.** **3. Memory contributes through base scoring and temporal correction** **Base scoring.** The two compatibility scores determine the base anomaly score: $$b_i=1-\operatorname{clip}_{[0,1]}\left(\frac{c_i^{\mathrm{rec}}+c_i^{\mathrm{drift}}}{M_{\mathrm{rec}}+M_{\mathrm{drift}}}\right).$$ Each scale $M_q$ uses the training-set 95th percentile of its corresponding score, with a lower bound of $10^{-8}$. These scales stay fixed during validation and testing. **Low compatibility produces a high base anomaly score.** **Temporal correction.** The maintained and reconstructed memories of both endpoints enter Stage 2 alongside the two compatibility scores. The correction module uses these features from the current event and preceding same-source events to learn a bounded adjustment to the base prediction. [1] Lee et al. *SLADE: Detecting Dynamic Anomalies in Edge Streams without Labels via Self-Supervised Learning.* KDD 2024. https://arxiv.org/abs/2402.11933W3: Feature roles and the rationale for concatenation
1. What the features represent and how they are fused
The correction module combines two types of information: the reconstruction and drift MI scores, which summarize reconstruction compatibility and temporal consistency, respectively; and the current and reconstructed endpoint memory representations, which retain richer information about the interacting nodes.
Concatenation followed by a learned projection provides learnable linear fusion. Specifically, we compute
where contains the two standardized MI scores and contains the standardized endpoint representations. Concatenation makes both available to a shared 64-dimensional projection, with and learning their respective contributions.
This provides a simple way to retain the two precomputed evidence summaries alongside the endpoint representations. Including the MI scores requires only two additional input coordinates, and the projection can reduce their contribution when the endpoint representations already provide sufficient information. The resulting event representations are then processed by the temporal Conv1D.
2. Feature ablations and alternative fusion
We evaluated feature contributions through two input ablations. Evidence-only supplies only the MI scores to the correction module, while Memory-only supplies only the endpoint representations. Both variants retain the MI-derived base score . We also evaluated evidence-guided cross-attention as an alternative fusion strategy, using the MI scores to attend to the endpoint representations. The temporal Conv1D and bounded correction mechanism remain unchanged.
All variants share frozen Stage-1 features, the temporal window, and training budget. We use the same configurations as in the main experiments, select checkpoints based on validation performance, and report test mean ± standard deviation over three seeds.
Test AP (%):
| Dataset | Evidence-only | Memory-only | Concat (TECOS) | Cross-attention |
|---|---|---|---|---|
| Wikipedia | 1.41 ± 0.16 | 1.64 ± 0.17 | 1.70 ± 0.21 | 1.61 ± 0.28 |
| 0.26 ± 0.04 | 0.31 ± 0.06 | 0.31 ± 0.08 | 0.33 ± 0.06 | |
| BitcoinAlpha | 15.28 ± 0.33 | 15.52 ± 0.42 | 15.69 ± 0.24 | 14.93 ± 0.57 |
| BitcoinOTC | 19.99 ± 0.05 | 22.38 ± 0.80 | 22.34 ± 0.83 | 22.67 ± 0.69 |
| MOOC | 2.38 ± 0.04 | 2.80 ± 0.02 | 2.79 ± 0.01 | 2.66 ± 0.03 |
| EU-Core | 22.34 ± 2.38 | 55.06 ± 4.44 | 53.08 ± 3.97 | 55.12 ± 4.04 |
| Email-DNC | 48.47 ± 0.68 | 87.20 ± 0.27 | 87.13 ± 0.21 | 82.04 ± 2.25 |
Test AUC (%):
| Dataset | Evidence-only | Memory-only | Concat (TECOS) | Cross-attention |
|---|---|---|---|---|
| Wikipedia | 86.61 ± 0.42 | 87.67 ± 0.99 | 87.66 ± 0.93 | 87.36 ± 0.82 |
| 73.14 ± 1.01 | 72.42 ± 1.38 | 73.22 ± 1.36 | 73.85 ± 1.45 | |
| BitcoinAlpha | 76.28 ± 0.22 | 78.16 ± 0.07 | 78.24 ± 0.03 | 76.93 ± 0.84 |
| BitcoinOTC | 75.84 ± 0.10 | 76.73 ± 0.32 | 76.72 ± 0.34 | 76.47 ± 0.56 |
| MOOC | 62.43 ± 0.37 | 70.43 ± 0.29 | 70.69 ± 0.55 | 69.58 ± 0.59 |
| EU-Core | 74.97 ± 4.04 | 92.41 ± 0.37 | 92.27 ± 0.22 | 92.29 ± 0.25 |
| Email-DNC | 92.30 ± 0.26 | 98.31 ± 0.05 | 98.31 ± 0.04 | 97.69 ± 0.29 |
3. What the results show
The endpoint representations provide effective features for temporal correction, while MI scores also contribute through the base prediction. Adding endpoint representations to Evidence-only improves both mean AP and mean AUC on all seven datasets. Memory-only performs close to the full input, indicating that the endpoint representations retain substantial information useful to the correction module.
Memory-only still uses the MI-derived base score . The MI scores therefore continue to influence the final prediction even when they are omitted from the correction inputs. Removing the base score lowers AP on six of seven datasets (Table 4), supporting its contribution. The full-input design additionally exposes the two MI summaries directly to the correction module, where their weights are learned jointly with those of the endpoint representations.
Concatenation followed by a learned projection is competitive with the cross-attention alternative. It achieves higher mean AP on four of seven datasets and higher mean AUC on five of seven datasets. On Email-DNC, its mean AP is 5.09 percentage points higher (87.13% versus 82.04%).
These results support our choice of concatenation followed by projection: it combines both feature families through learned weights without an additional attention module, while cross-attention provides no consistent performance advantage under the matched experimental configuration.
W1
We have separated the motivation from the framework and added quantitative analysis across seven datasets. The revised Figure 1 is a standalone motivation figure in the Introduction, while Figure 2 presents the framework. Figure 1(a) illustrates the three evidence–label mismatch patterns.
Figure 1(b) analyzes base-score peak locations within the current event and its two preceding same-source events, using the original observed labels without imposed delays. Historical peaks account for 36.07%–65.25% of eligible positively labeled test events, averaged over three seeds. This quantifies how observed positive labels can occur at positions other than the local maximum of anomaly evidence.