acceptodds
Under review as a conference paper at ICLR 2027

Beyond Forecasting: Memory-Augmented Time Series Anomaly Prediction

Abstract

Time series anomaly prediction (TSAP) aims to anticipate future anomalous states from historical observations for early warning and proactive decision-making. Existing point-level methods typically infer anomaly states from forecasted futures, yet forecasting favors the most likely, normal-dominated evolution, while anomaly prediction depends on preserving rare but discriminative abnormal dynamics. This mismatch often leads to over-smoothed futures and weakened point-level anomaly signals. To bridge this gap, we propose LAMP, a lightweight memory-augmented framework that complements parametric prediction with non-parametric evidence from previously observed future evolutions. Given a historical window, LAMP retrieves futures associated with similar historical patterns from a multi-scale memory and adaptively integrates them with parametric predictions, allowing TSAP to exploit not only what is likely to happen, but also what actually happened after similar histories. Experiments on five multivariate time-series benchmarks against 16 baselines show that LAMP consistently improves anomaly prediction, achieving average gains of 6.71% in V-PR and 9.37% in V-ROC, while training up to 8.2× faster with 14.0× lower GPU memory usage than forecasting-based alternatives. Code is available at https://anonymous.4open.science/r/lamp/.

Primary area:
learning on time series and dynamical systems
Keywords:
multivariate time series, anomaly prediction, memory augmentation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Pw9hOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LAMP, a memory-augmented framework for point-level time series anomaly prediction. It addresses temporal over-smoothing in forecasting-based methods by retrieving future residual representations associated with similar historical patterns and integrating them with parametric predictions. Experiments on five datasets across six forecasting horizons demonstrate improvements over existing baselines, supported by ablation and efficiency analyses.

Critical strengths
  1. The paper addresses a meaningful problem in point-level anomaly prediction. The observation that forecasting may suppress rare abnormal dynamics provides a reasonable motivation for memory augmentation.

  2. The experiments cover multiple datasets, forecasting horizons, and baseline combinations. The ablation studies and efficiency analysis further support the effectiveness and practicality of the proposed framework.

Critical weaknesses
  1. The distinction from existing anomaly detection methods is insufficiently discussed. Forecasting-based TSAD and early anomaly detection already explore related prediction settings. The paper does not clearly establish its novelty relative to these approaches. Moreover, recent point-level TSAP methods such as RED-F and FATE are discussed but not included in the experiments.

  2. The memory selection mechanism lacks sufficient validation. The method prioritizes entries with large future reconstruction residuals, but these residuals do not necessarily correspond to informative abnormal patterns. The experiments do not establish whether the selected entries capture anomaly-related dynamics or simply high-variance observations.

Questions for authors
  1. Could the authors clarify the distinction between the proposed TSAP formulation and existing forecasting-based TSAD approaches, and provide comparisons with RED-F and FATE?

  2. Could the authors analyze the retained memory entries and compare the proposed importance-based selection with random or diversity-based selection to validate its effectiveness?

Reviewer_JcsfOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Time series anomaly prediction (TSAP) aims to anticipate anomalies before they occur by analysing historical patterns. Therefore, the authors propose LAMP, which leverages previously observed abnormal patterns to complement forecast-based predictions and preserve informative anomaly signals. LAMP improves detection accuracy while reducing GPU memory usage compared with alternative forecasting-based methods.

Critical strengths

• The authors present a methodology that achieves higher accuracy while maintaining a lightweight model architecture. The emphasis on model efficiency is supported by comparisons with baseline methods. • Unlike conventional models that focus on minimizing reconstruction errors, the proposed residual-to-residual training objective offers an interesting direction for future exploration.

Critical weaknesses

• The residual-to-residual training objective offers an interesting direction. However, the paper lacks an intuitive explanation of how the approach works and why it is effective. • It is unclear how the model achieves TSAP, specifically which component is designed to provide early warnings before an anomaly occurs. Also, there is no time series plots to illustrate this tasks. • In the abstract, the authors describe the proposed method as “a lightweight memory-augmented framework,” but how does the model achieve “lightweight” is not clearly elaborated.

Questions for authors

• In Figure 2, multi-scale contractive memory, it unclears what’s the history part. In section 3.4, it stated X_t^(h,c), is it the prediction of residual representation? If so, why the authors only do Multi temporal scale after residual representation? Or is it in this order: history input-> multi temporal scale->dual reconstruction? • L220– L223: “With limited capacity, storing every training pair can fill the bank with nearly flat future values, leaving fewer examples of pronounced deviations.” 227–L228: “When capacity is exceeded, we retain the top-M entries by ρj. This keeps stronger future continuations available when similar patterns recur. Magnitude is only a proxy and may reflect noise.” Referring to the two statements, the author stated nearly flat future values, 1) if so, selecting the top-M entries may not also provide more informative pattens. 2) Also why are larger mean absolute magnitudes in Eq (4) can considered as importance score? Larger values do not necessarily indicate more informative patterns. Further justification is needed for this selection criterion, through experimental results, theoretical analysis, or an intuition.

Reviewer_icuJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses temporal over-smoothing in point-level time series anomaly prediction (TSAP), where models trained on normal-dominated data tend to produce smooth predictions that blur anomaly signals. To mitigate this, the authors propose LAMP, a lightweight memory-augmented framework that retrieves historical-future representation pairs from a multi-scale memory bank and adaptively fuses them with an LSTM-based parametric prediction via a gated mechanism. Evaluated on five multivariate datasets, LAMP demonstrates improved V-PR/V-ROC performance and high computational efficiency compared to existing baselines.

Critical strengths
  1. Identifies a Practical and Valid Problem: The paper accurately diagnoses "temporal over-smoothing" as a critical bottleneck in point-level time series anomaly prediction. The motivation is well-illustrated (e.g., Figure 1), making the problem both theoretically grounded and practically relevant.
  2. High Computational Efficiency: The proposed framework is highly lightweight (0.4M parameters, 53G FLOPs). The demonstrated improvements in training speed and GPU memory reduction make the approach highly attractive for real-world, real-time deployment.
Critical weaknesses
  1. Lack of direct evidence for the core premise (over-smoothing mitigation): The paper is motivated by temporal over-smoothing but fails to empirically substantiate that LAMP actually mitigates it. The reported improvements in V-PR/V-ROC are indirect and conflate the specific de-smoothing mechanism with general capacity gains from the memory and fusion modules. The authors should explicitly include LAMP's anomaly score curve in Figure 1 (or similar qualitative plots) and provide a quantitative metric (e.g., variance of first-order differences) to demonstrate that LAMP generates distinctly sharper anomaly signals than the baselines.

  2. Inadequate literature review on over-smoothing in the time series anomaly domain: Despite over-smoothing being the central motivation, the related work lacks a dedicated discussion of how this specific problem has been addressed within the time series anomaly detection (TSAD) literature itself. The authors must contextualize LAMP among existing TSAD-specific over-smoothing studies. For instance, DifNet [1] explicitly identifies and mitigates "feature over-smoothing" (FOS) in reconstruction-based anomaly detection, arguing that over-smoothing blurs the boundary between normal and anomalous patterns. A clear explanation is required to distinguish why LAMP's "historical-future pairing" mechanism is fundamentally necessary and superior to these existing TSAD-oriented solutions.

[1] Wang H, Li J, Chen Y, et al. DifNet: Difference-based multi-resolution decomposition for time series anomaly detection: H. Wang et al[J]. Applied Intelligence, 2025, 55(7): 680.

  1. Questionable methodological novelty and unproven synergy: The framework appears to be an ad-hoc assembly of standard modules (residual decomposition, contrastive memory, MLP gating). Using a basic LSTM as the base predictor is highly questionable, as LSTMs are inherently prone to over-smoothing. The authors must conduct rigorous ablations to prove synergy: 1) replacing the LSTM with a stronger predictor (e.g., DLinear) to see if the memory gains persist; 2) comparing against pure non-parametric retrieval; and 3) justifying why this specific combination is fundamentally necessary.

  2. Inadequate baseline coverage and striking experimental anomalies: The paper utilizes only A2P as the over-smoothing-specific baseline, ignoring other paradigms. More critically, the experimental results present a severe anomaly: A2P (ICML 2025) performs significantly worse than the general foundation model MOMENT (ICML 2024) on the evaluated datasets. This suggests A2P suffers from domain adaptation failure, making LAMP's massive advantage appear potentially as a benchmark-specific coincidence rather than a universal solution to over-smoothing.

  3. High vulnerability to distribution shift: LAMP relies on a static, non-parametric memory bank built entirely from the training distribution, making it highly sensitive to distribution shifts (e.g., concept drift or changing operating regimes). Under such shifts, the forced softmax retrieval of irrelevant keys will inevitably lead to spurious matches and catastrophic negative transfer. The authors must evaluate LAMP under distribution shifts and analyze whether the gating mechanism (αt\alpha_t) can successfully reject spurious memories, or if LAMP degrades more severely than a pure parametric baseline.

Questions for authors

Q1 (Evidence for de-smoothing): Can you provide direct empirical evidence that LAMP generates less smooth anomaly signals than baselines? Specifically, could you add LAMP's anomaly score curve to Figure 1 and report a quantitative metric (e.g., variance of first-order differences) to verify the mitigation of over-smoothing?

Q2 (Literature on TSAD over-smoothing): Given that over-smoothing is the core motivation, why is there no discussion of existing TSAD-specific studies on this issue (e.g., DifNet [KDD 2025])? Please clarify how LAMP's "historical-future pairing" fundamentally differs from and improves upon these existing over-smoothing mitigation solutions.

Q3 (Architectural synergy and ablations): The framework appears to be an ad-hoc assembly of standard modules (residual decomposition, contrastive memory, MLP gating). Using a basic LSTM as the base predictor is highly questionable, as LSTMs are inherently prone to over-smoothing. The authors must conduct rigorous ablations to prove synergy: 1) replacing the LSTM with a stronger predictor (e.g., DLinear) to see if the memory gains persist; 2) comparing against pure non-parametric retrieval; and 3) justifying why this specific combination is fundamentally necessary.

Q4 (Baseline anomalies): Why does A2P (ICML 2025) perform significantly worse than the general foundation model MOMENT (ICML 2024) on the evaluated datasets? Please provide an explanation for this anomaly and supplement the baselines with other over-smoothing mitigation paradigms (e.g., HCAN, F2A) to validate the universality of LAMP's superiority.

Q5 (Robustness to distribution shift): Since LAMP relies on a static memory bank, how does it perform under real-world distribution shifts (e.g., concept drift)? Does the gating mechanism (αt\alpha_t) successfully learn to reject spurious memories when the test distribution deviates from the training distribution, or does it cause severe negative transfer?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.