- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the difficulty of detecting transient anomalies, such as bursts, in the presence of background noise and dynamically changing conditions. It proposes an anomaly detection framework that combines power spectral density (PSD)-matched colored-noise padding with conditional variational autoencoder (CVAE)-based data augmentation. The framework achieves strong performance on X-ray time-series signals from active galactic nuclei (AGN).
Critical strengths
- The paper explores a new application setting for anomaly detection: X-ray time-series signals from AGN, which present more challenging detection conditions.
- The paper proposes a novel anomaly detection method that combines a convolutional neural network (CNN) with CVAE-based data augmentation and achieves promising results in this setting.
Critical weaknesses
- Several design choices lack sufficient theoretical justification or experimental validation, including the use of a CNN, the choice of a CVAE, and the need for data augmentation.
- The paper lacks comparisons with existing baseline methods.
Questions for authors
- What motivates the use of a CNN, a CVAE, and PSD-matched colored-noise padding? Please provide theoretical justification or experimental evidence to support these design choices.
- How do existing state-of-the-art anomaly detection methods perform on this dataset?
Dear Reviewer VzbJ,
Thank you for constructive feedback on our paper. Here are our responses to your questions/comments/feedback:
Response to Critical Weakness (1): We used a 1D-CNN since 1D-CNNs are effective in capturing location invariant features in time-series. This is also the simplest deep learning based method that can be used for this purpose, and the usage of other methods such as transformer based models is left as future work. The goal of the CVAE is to do data augmentation, and the objective of data augmentation is to supplement the (very small) real training set in training the CNN for the final prediction (Please see paragraphs 1 and 5 in the main paper). By nature, the data generated by our CVAE should therefore be in-distribution to the real dataset. Our framework/pipeline incorporates known physics into the simulated (hand-crafted) time-series samples by injecting known forms of noise and known forms of mathematical stochastic burst profiles, and then employs a CVAE to make the transition from mathematically engineered time-series to continuous/seamless time-series that are more in-distribution. One of the novelties of this work is displaying the very fact that CVAEs can make this transition from mathematically engineered signals to real signals (for the astrophysical domain). Further work on (other domains) will support our hypothesis. Please see the discussion in Section 4.3 in the main paper for this and Table 7 for experimental evidence in Supplementary Materials in support of this. The goal of using PSD matched colored noise is explained in paragraph 3 of Section 3.1. In the time-series without stochastic bursts, the PSD assumes a single value of alpha to be governing the noise in the time-series signals, and extends the signal using the computed value of alpha for each signal. In the time-series with stochastic bursts, the PSD assumes the same - that the stochastic bursts on top of the background noise are modeled by a single value of alpha, and similarly extend the signal using the computed value of alpha for each signal. Experimental evidence, i.e. using methods other than PSD to extend the signal while inferencing on the test set showed a marked degradation in performance (signals without stochastic bursts extended by other methods were erroneously inferenced by the CNN to contain stochastic bursts). In other words, there were more false positives.
Response to Critical Weakness (2): A table (Table 2 in main paper) showing the zero-shot performance of (1) Classical Non Deep-learning based methods, and (2) Pre-trained Foundation Models on our WAX-XMM test set has been added to the main paper (rebuttal submission). Please note that all of these models did not require us to retrain them with our synthetic/generated datasets or any other datasets.
We request you to re-evaluate your rating for the paper based on the above-mentioned comments and the changes made to the paper (i.e. rebuttal submission).
Thank you for your continued evaluation of our paper.
Thank you.
Regards,
Authors