- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SCAST, a framework for cross-system log anomaly detection under heterogeneous event vocabularies, graph structures, and limited target-domain anomaly supervision. The main idea is to replace direct alignment of mixed source and target representations with a decompose–screen–transfer pipeline. SCAST first constructs graph-based representations of log windows and decomposes their structural signals into multiple graph-spectral bands. Within each band, spatial and temporal priors are used to model candidate dependencies. A feature-wise router then separates the representation into a screened branch and a residual branch. Only the screened branch is dynamically fused across spectral bands, used for anomaly scoring, and subjected to cross-domain adversarial alignment. The paper motivates this design by arguing that domain shifts may differ across structural frequency bands and can partially cancel after premature feature fusion. The authors also provide a conditional domain-adaptation bound characterizing when selective alignment can improve a target-risk upper bound. Experiments are reported on HDFS, BGL, Thunderbird, and OpenStack. The paper includes single-system results, historical cross-system comparisons, matched HDFS↔OpenStack controls, component ablations, frequency-resolved perturbation analyses, efficiency studies, and spectral-basis sensitivity analyses.
Critical strengths
- The paper identifies a meaningful limitation of indiscriminate domain alignment. The distinction between aligning a mixed representation and first screening potentially transferable information is conceptually useful. The matched alignment-placement comparison provides direct evidence that the ordering of screening and alignment matters in the evaluated setting.
- The empirical study is reasonably thorough and includes useful diagnostics beyond headline performance. The paper includes component ablations, matched target-only and pre-alignment controls, spectral-band analyses, perturbation-based sensitivity diagnostics, efficiency measurements, and spectral-basis sensitivity tests. I particularly appreciate that the paper explicitly reports limitations of several analyses rather than overclaiming them.
Critical weaknesses
- The central notion of a “transferable dependency” is not independently identified or validated. The paper’s core scientific interpretation is that SCAST screens out system-specific dependencies and retains dependencies that are more suitable for cross-system transfer. However, there is no independent supervision or ground truth indicating which dependencies are actually transferable. Operationally, the router is trained using the same anomaly-prediction, structural regularization, and domain-alignment objectives that define the final task. The screened branch is then the only branch used for anomaly scoring and alignment. Therefore, what is directly established is that the optimization procedure learns a representation that is useful for the evaluated prediction and alignment objectives. This does not by itself establish that the screened branch corresponds to genuinely stable cross-system structural dependencies. The perturbation analysis does not fully resolve this issue. It shows that perturbing the screened branch causes larger anomaly-logit changes than perturbing the residual or random branches in the tested configuration. This establishes greater predictive sensitivity, but predictive importance is different from transferability. A feature can strongly affect anomaly prediction while still being system-specific. This distinction is important because “selective transfer of transferable dependencies” is the main conceptual claim of the paper. Without an independent notion of transferability, the interpretation is substantially stronger than the evidence currently supports.
- The learned spectral dependency representation is basis-dependent, which weakens its interpretation as a stable structural object. The appendix shows that the tested predictors are not invariant to eigenvector sign flips or rotations within repeated eigenspaces. These transformations can correspond to mathematically equivalent representations of the same eigenspaces, yet they produce non-negligible and sometimes large changes in model logits. This is particularly concerning here because the method interprets learned quantities in spectral coordinates as frequency-resolved structural dependencies. If model behavior depends materially on an arbitrary choice of spectral basis, then it is unclear whether the router is discovering stable graph-structural relationships or basis-specific coordinate patterns. The authors correctly note that the reported logit changes do not directly measure F1 degradation, but I do not think this eliminates the conceptual issue. The lack of basis invariance directly affects the interpretation of the learned spectral representations, which is central to the paper’s claimed mechanism.
Questions for authors
- Can the authors directly validate transferability rather than predictive importance? Is there an experiment in which dependencies selected in one source–target pair are evaluated for stability or usefulness in a different target system, without retraining the router for that pair? For example, if the screened representation truly captures transferable dependencies, I would expect it to generalize more consistently across multiple independently chosen target domains than the residual branch. Such an experiment would materially strengthen the central claim.
- How should the learned spectral dependencies be interpreted given the demonstrated basis sensitivity? Can the authors construct a basis-invariant or basis-canonicalized version of the method and show that the main transfer results remain intact? Alternatively, can the authors quantify whether the observed sign-flip and eigenspace-rotation sensitivity affects actual ranking metrics such as PR-AUC or ROC-AUC, rather than only logits? A convincing answer here would materially affect my assessment.
- How much of the gain comes from selective alignment itself, versus the particular spectral decomposition? The current results support the full pipeline, but the broader scientific idea seems to be “screen before align.” Can the authors test whether the same principle holds with a non-spectral decomposition or another representation family? If the advantage persists, that would support a more general selective-alignment principle rather than a method tied specifically to this graph-spectral parameterization.
- Can the authors clarify the strength of the cross-system evidence under matched protocols? The historical cross-system table mixes results from methods with heterogeneous preprocessing, data splits, label budgets, encoders, and thresholding protocols. The most controlled comparisons appear to be concentrated in HDFS↔OpenStack. How much of the claimed cross-system generality remains if the conclusions are restricted to only matched-protocol experiments?
3 comments
Follow-up on the Rebuttal
Thank you for the additional experiments. The frozen-model results on new target systems, along with the non-spectral and LogTAD comparisons, address my main questions about transferability and the broader usefulness of screening before alignment.
Please highlight these results in the revision and be more precise about the claim of discovering transferable structural dependencies. The new stability tests are also helpful, though stable predictive performance should be distinguished from strict spectral-basis invariance.
Thank you for the comments.
We have highlighted the above results in the revised manuscirpt. Meahwhile, we supplemented and revised the relevant content to ensure that the claims are stated more rigorously, comprehensively, and precisely.
As to transferability, we have complemented an entirely Section 4.6 dedicated to the discussion of transferability. And, the newly added transferability experiments and their detailed discussions are included therein.
As to basis sensitivity, we have complemented the experimental results on performance under spectral-basis transformations as well as relevant discussions in Table 21 of Appendix G.2. Furthermore, we explicitly state that the experiments provide empirical support for its task-level stability of predictive performance, rather than strict spectral-basis invariance.(In the the preceding paragraph of Appendix G.2 of the revised manuscript)
As to selective alignment, we have complemented an entirely Section 4.5 dedicated to the discussion of “screening before alignment”. The newly added experiments and their detailed discussions are included therein.
As to comparisons under matched protocols, we have complemented the related experimental results in Table 2.
All the above revisions are highlighted in blue in the revised manuscript.
Responses to Question 1: In order to validate transferability, we have specially designed the following experiment: to froze the representation, screening, routing, and selection parameters after training on HDFS → OS, then evaluated selected, residual, and gate‑location‑shuffled representations on BGL and TB. Identical linear readouts use only OS labels and share feature dimension, optimizer, stopping rule, and evaluation windows. External‑target access is limited to predefined normal reference data for graph construction and evaluation, with no parameter updates. Results are summarized below:
The selected representation achieves higher PR‑AUC than both controls on both unseen targets. All descriptive paired 95% intervals are above zero, supporting the predictive utility of the learned selection locations beyond the training pair. Furthermore, we have added a new Section 4.6 dedicated to the discussion of transferability.
Responses to Question 2: In order to verify the basis sensitivity, we extended the analysis to complete target test sets, using three independently trained checkpoints per direction and 20 pre‑specified transformations per type (Table 21, Appendix G.2). Graphs, inputs, learned parameters, and validation-selected thresholds were fixed. Results indicate that the proposed model largely preserves its average ranking performance under the tested spectral-basis transformations, providing empirical support for its task-level stability:
Responses to Question 3: First, to test screening before alignment beyond spectral decomposition, we replaced spectral features with one‑ to three‑hop propagation features in node space and compared screened versus direct alignment across five directions (Table 4, Section 4.5).
Screening improves F1 in both representation families across all five directions.
Second, to test the same principle in an external method, we added screening before alignment to LogTAD (Table 5, Section 4.5). The table reports BGL → TB.
Adding screening raises F1 by 10.49 percentage points, supporting its benefit in an external alignment model.
Third, to isolate alignment placement while retaining screening, we compared three alignment settings in the multi‑hop non‑spectral model (Table 17, Appendix E.3).
Post‑screen alignment achieves the highest F1, exceeding pre‑screen alignment by 0.78 percentage points and no alignment by 2.51 percentage points. This supports aligning the screened representation for thresholded detection.
Responses to Question 4: Owing to inherent differences in model architectures, some protocol components, such as label budgets and encoders are difficult to align perfectly. We adopt LogTAD (with released code) and add a new baseline ARMET, and provide new experimental results (F1$\uparrow$) under adapted protocol (Table 2). It can be observed that SCAST achieves competitive performance with the matched protocol.