acceptodds
Under review as a conference paper at ICLR 2027

A representation learning perspective on revisiting reconstruction-based time-series anomaly detection

Abstract

Reconstruction-based approaches are one of the mainstream paradigms for time- series anomaly detection. Existing studies typically improve anomaly detection performance by designing sophisticated network architectures or introducing prior biases to capture specific characteristics of time-series data. However, we find that these approaches still have limitations in effectively capturing data characteristics and often incur substantial computational costs. To address these issues, we in- troduce a Transformer-based patch-level anomaly detection model that focuses on learning global representations of time-series data. By combining point-wise and interval-wise anomaly scores, our method achieves anomaly detection perfor- mance comparable to or even better than that of mainstream approaches, without requiring complex network architectures or manually designed prior biases. It also achieves a favorable trade-off between detection performance and computational cost. Furthermore, we investigate the scalability and transferability of our approach, providing a new perspective on reconstruction-based time-series anomaly detec- tion and suggesting that increasingly complex network architectures may not be necessary for effective anomaly detection

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
time series anomaly detection, representation learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_nkB2OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

As I understand, the paper wants to propose a relatively simple approach to timeseries anomaly detection, "without requiring complex network architectures or manually designed prior biases". The approach involves a Transformer architecture (which is to me a relatively complex network architecture) and a manually designed loss function. The final claim is that "complex representation learning designs are not always necessary". The results show that on some timeseries problems, their approach can be on par or outperform other approaches.

Although I agree with the authors that complex models are not always needed, this claim is so general, that it does not teach me much. It would help me more if the paper would show what types of model bias or architecture is superfluous, and for what type of problems it may actually help. In the introduction you mention "we argue that complex network architectures or inductive biases designed to capture particular patterns may not be the key factors for improving the performance", but then it does not become clear what the key factors actually are.

Critical strengths

It is good that models are compare to sensible baselines. I agree that many proposed architectures/loss functions/biases may not fix/alleviate some identified problem.

Critical weaknesses

I don't see how this substantially adds to our scientific knowledge. We know that simple models often work well. This paper suggests a (not-so-simple) model and some loss function, applies it to some datasets and then shows that it outperforms all other models.

Questions for authors

You showed that your transformer often outperforms the other approaches. Is this because the considered datasets are relatively simple? It is mentioned somewhere that "Unlike image, which contain highly complex patterns such as edges, textures, and shadows, time series are usually more periodic and easier to characterize.". But if you would have considered video data (also some type of time series data) then this would be more complex, wouldn't it? So then the other appoaches may be more appropriate?

Another detail: what do I see in Table 1? The caption says AUC-ROC, R_AUC-ROC and VUS-ROC, but the table itself contains VUS-PR, F1 and R_F1.

Reviewer_cis9OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper claims that anomaly detection requires computing errors based on a global understanding, and proposes an autoencoder-style reconstruction-based anomaly detection method. The method uses RevIN, a normalization scheme commonly used in time series, extracts a CLS token for a global understanding of the sequence, and then computes anomaly scores in an autoencoder manner. The anomaly score is computed using both a simple error and a frequency-based error. Through this method, the paper shows good performance, raises doubts about scaling laws, and concludes that the method performs well in transfer experiments because it learns representations that are not biased toward the training dataset.

Critical strengths
  • The paper deals with a global-understanding-based time-series anomaly detection method, which mainly appears in classification and occasionally in time-series anomaly detection.

  • The paper conducts various experiments, such as a scaling law analysis and an analysis of the encoder’s transferability.

Critical weaknesses
  • Global representations are used in studies ranging from classification (TS2Vec) and those using the Fourier transform (FreTS, TSLANet) to anomaly detection (TSPulse, TSLANet). This is not a new research direction.

  • The novelty of using the Fourier transform for anomaly detection is limited.

  • The model size is too small to claim the failure of scaling laws.

  • I think it is difficult to claim, based only on anomaly detection performance, that the representations are not biased toward a specific dataset. In addition, anomaly detection should work well when the model understands the current data relatively better than general data, so the statement that high anomaly detection performance indicates less bias seems odd.

Questions for authors
  • How does the novelty of this work compare with existing Fourier transform studies such as TSLANet and FITS?

  • Is there any connection or novelty among the components that make up the method?

  • Regarding the property of not being biased toward a specific dataset, is any additional analysis possible beyond anomaly detection performance?

  • The number of datasets in the experiments is too small. It would be good to use the datasets provided by TSB-AD.

  • Simple baselines are lacking. In particular, it would be good to include methods such as PCA that handle global features.

Reviewer_kymfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper revisits reconstruction-based time-series anomaly detection from a representation-learning perspective. It proposes a lightweight Transformer that aggregates patch embeddings into a global CLS representation and reconstructs the input through a linear head. Detection combines point-wise and frequency-domain reconstruction errors.

Critical strengths
  • Clear motivation and empirical performance. The paper addresses a practically relevant question: whether increasingly complex architectures are necessary for reconstruction-based anomaly detection. The reported results support the narrower claim that a relatively simple architecture, combined with an appropriate scoring strategy, can achieve competitive performance.
Critical weaknesses
  • Limited methodological novelty. The approach primarily combines established components, including patch-based modeling, a standard Transformer encoder, CLS-token aggregation, and time- and frequency-domain anomaly scores. The paper should more clearly articulate its distinctive methodological contribution and the insights obtained beyond combining these components.
  • Insufficient ablation of the central design choices. Although the paper includes ablation experiments, they do not adequately distinguish the contribution of global representation learning from that of the combined anomaly score. For example, removing the frequency-domain score substantially reduces performance, raising the question of whether the gains primarily arise from the representation or the scoring strategy.
Questions for authors
  • Benchmark coverage. Could the authors extend the evaluation to more diverse settings, such as those provided by TSB-AD [1] or TAB [2]? Broader coverage would help establish the generality of the findings.
  • Does the five-epoch training schedule apply only to the proposed method or also to the baselines? Please clarify the training budgets and provide evidence of convergence, particularly for the model-scaling experiments.
  • The manuscript cites Dual-TF [3], which is directly relevant to the combined scoring strategy. Could the authors explain its omission from the experimental comparisons and include it where feasible?
  • The manuscript contains several unnecessary capitalizations (e.g., line 214, 225).

[1] Liu, Qinghua, and John Paparrizos. "The elephant in the room: Towards a reliable time-series anomaly detection benchmark." Advances in Neural Information Processing Systems 37 (2024): 108231-108261.

[2] Qiu, Xiangfei, et al. "Tab: Unified benchmarking of time series anomaly detection methods." Proc. VLDB Endow 2025.

[3] Nam, Youngeun, et al. "Breaking the time-frequency granularity discrepancy in time-series anomaly detection." Proceedings of the ACM Web Conference 2024. 2024.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.