acceptodds
Under review as a conference paper at ICLR 2027

Normality Constraint Learning: Adapting Foundation Models for Time Series Anomaly Detection

Abstract

Time Series Foundation Models (TSFMs) achieve strong generalization by learning to reconstruct or forecast broad temporal patterns from large-scale time series during pre-training. Yet this strength can become a weakness for anomaly detection: TSFMs may model rare anomalous patterns as effectively as normal ones, allowing anomalies to be accurately reconstructed or forecasted and thus diminishing their reconstruction/forecasting error-based anomaly scores. This paper proposes (), a lightweight plug-and-play framework that adapts pre-trained TSFMs for accurate anomaly detection without modifying their pre-trained parameters. Our key insight is to constrain the broad pattern space of TSFMs to the normal structure of a target time series, preventing their broad modeling capability from obscuring abnormal deviations. Specifically, NCL constructs a compact normality subspace from a few normal patch features and adaptively steers each patch feature toward normality within this subspace, guided by contrastive constraints that form compact and discriminative normality manifolds. The calibrated features are aggregated to reinforce normal components and fused with the original TSFM output, amplifying the discrepancy between normal and abnormal observations for the reconstruction/forecasting error-based anomaly scoring. Extensive experiments across diverse TSFM families and benchmarks show that NCL consistently improves anomaly detection performance, providing a generalizable framework for adapting TSFMs to anomaly detection.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Foundation Models, Anomaly Detection, Normality Constraint Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_MwGpOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposed to use a (variant of a) PCA anomaly detector in the representation learned by some foundation model. Because these models are large and flexible, they potentially are able to reconstruct all types of signals, including anomalies, and therefore some adaptations are needed to tune the anomaly detector to the new task at hand. This paper proposes to enhance the PCA by two additional contributions: a scaling of the PCA features (patch-adaptive normality transform) and an additional contrastive loss to adapt the PC components.

The paper is very extensive (including appendices 43 pages!), including numerous datasets and foundation models. I focused on the main text of the paper.

Critical strengths

Good idea to use a PCA-based anomaly detector in the representation of foundation models. (A reference to an anomaly detection approach using PCA in a normal feature space would be nice)

Critical weaknesses
  • A question that I have is that the initial features of the representation on which the PCA is applied, is not per se well scaled. PCA is sensitive to the scaling of features, and will focus on the large-variance directions. In step two of the approach, these PCA features may be rescaled, but in step one the PCA still depends on the relative scaling of the original representation. The contrastive learning often seem to solve it always (as I can see in Table 2): is this PCA initialisation really needed?

  • Another question that I have is the optimization of hyperparameters, like the number of PCA components, or N, or \lambda. If you optimize for the best reconstruction of normal data, then you probably end up with the original problem (that you make the model as complex as possible such that all data, including anomalies, can be perfectly reproduced). Or do you need supervised anomaly data in order to optimize the hyperparameters? This is not clear to me.

Questions for authors

These are the two questions I had in the 'Critical weaknesses':

  1. how essential is the PCA initialization?
  2. how are the hyperparameters optimized; do you need supervised information?
5 comments
AuthorsOpenReview ↗

Response to Reviewer MwGp (Part 1/3)

Dear Reviewer MwGp,

We sincerely thank you for the positive assessment and the insightful questions. Below we address each concern in detail.


W1/Q1: Is the scaling issue addressed properly? How essential is the PCA initialization?

Feature Scaling and PCA Sensitivity. We agree with you that PCA is sensitive to feature scaling and naturally focuses on high-variance directions. In our case, the patch features are extracted from frozen TSFMs that have been pre-trained on large-scale corpora. Two properties mitigate the scaling issue:

  1. Foundation model feature normalization: Vanilla TSFMs (MOMENT, UniTS, Aurora, etc.) typically apply layer normalization or batch normalization within their architectures, which standardizes feature magnitudes across dimensions to some degree. While not perfect, this substantially reduces the arbitrary scaling differences that would severely bias PCA in raw input space.
  2. The normality transformation is adaptive: Even if some dimensions have larger variance purely due to scaling artifacts rather than semantic importance, the patch-adaptive normality transformation (PatchNT) in Step two learns direction-specific target values α\boldsymbol{\alpha} and calibration strengths β\boldsymbol{\beta} for each patch feature via learnable linear heads. This effectively re-weights and re-centers the contributions from different directions based on the reconstruction/forecasting loss and the contrastive constraint, correcting for potential scaling biases.

Essentiality of PCA Initialization. The driving force behind the consistent improvement of our contrastive learning (as shown in Table 2) lies in its effectiveness in further enhancing the discriminability of features. However, this does not mean that PCA initialization is unimportant. They simultaneously enhance the anomaly detection capability of the entire module from different aspects. Below, we further extract the comparative data from Table 2 to illustrate the significance of PCA initialization. We conducted three sets of comparisons simultaneously, including Vanilla vs. NCLN\text{NCL}_{\text{N}}, NCLP\text{NCL}_{\text{P}} vs. NCLNP\text{NCL}_{\text{NP}}, and NCLPC\text{NCL}_{\text{PC}} vs. NCLNPC\text{NCL}_{\text{NPC}}. These variants mean:

  • Vanilla: Original TSFMs without PCA initialization.
  • NCLN\text{NCL}_{\text{N}}: Uses only NormSpace (PCA initialization) without PatchNT or ConNC.
  • NCLP\text{NCL}_{\text{P}}: Uses only PatchNT (learnable transformation) without NormSpace or ConNC.
  • NCLNP\text{NCL}_{\text{NP}}: Uses NormSpace (PCA initialization) + PatchNT, without ConNC.
  • NCLPC\text{NCL}_{\text{PC}}: Uses PatchNT + ConNC, without NormSpace (PCA initialization) .
  • NCLNPC\text{NCL}_{\text{NPC}}: Full method (NormSpace + PatchNT + ConNC).

Vanilla vs. NCLN\text{NCL}_{\text{N}}:

Variant\Dataset-ModelIOPS-ROSEIOPS-TMMIOPS-UniTSPSM-ROSEPSM-TMMPSM-UniTS
Vanilla0.13070.18770.15300.36720.36110.4203
NCLN\text{NCL}_{\text{N}}0.17280.28210.15450.38290.45490.4479

NCLP\text{NCL}_{\text{P}} vs. NCLNP\text{NCL}_{\text{NP}}:

Variant\Dataset-ModelIOPS-ROSEIOPS-TMMIOPS-UniTSPSM-ROSEPSM-TMMPSM-UniTS
NCLP\text{NCL}_{\text{P}}0.14290.23560.15890.37610.44090.4375
NCLNP\text{NCL}_{\text{NP}}0.18110.30920.16230.39120.46550.4507

NCLPC\text{NCL}_{\text{PC}} vs. NCLNPC\text{NCL}_{\text{NPC}}:

Variant\Dataset-ModelIOPS-ROSEIOPS-TMMIOPS-UniTSPSM-ROSEPSM-TMMPSM-UniTS
NCLPC\text{NCL}_{\text{PC}}0.18430.30590.16680.38350.46600.4512
NCLNPC\text{NCL}_{\text{NPC}}0.19360.33380.18610.41070.50390.4735

From the pairwise comparison results, we can clearly find that PCA initialization consistently improve anomaly detection performance. The PCA initialization is essential in the sense that it provides a sample-efficient starting point for identifying the normal subspace. Without it, the learnable transformation (as NCLP\text{NCL}_{\text{P}}) must discover these directions from scratch using only the reconstruction/forecasting loss, which is less efficient. The PCA initialization provides a "warm start" by concentrating the transformation on the directions that already capture the most normal variation. The contrastive loss then refines these directions into compact, discriminative manifolds.

AuthorsOpenReview ↗

Response to Reviewer MwGp (Part 2/3)

W2 / Q2: How are hyperparameters optimized? Do you need supervised information? Dose this optimization lead to overfitting to anomalies?

We first clarify that NCL has three main hyperparameters:

  1. Number of principal directions kk (for the PCA subspace in NormSpace)
  2. Number of training patches NN (few-shot normal patches)
  3. Fusion weight λ\boldsymbol{\lambda} (learnable per-channel weights in Equation (9))

Then we elaborate in detail on how the hyperparameters are optimized without the supervision information. All hyperparameters are set in a purely unsupervised manner, with no access to anomaly labels during training. Specifically:

  1. kk (number of PCA components): We fix k=16k=16 across all experiments. This choice is motivated by two considerations:
    • Theoretical guidance (Theorems 4, Appendix B.2.2): The top-kk approximation error is bounded by σk+1\sigma_{k+1}, and empirically the singular value spectrum of TSFM patch features decays rapidly (most datasets have an effective rank re≪dr_\mathrm{e} \ll d where dd is the feature dimension). Retaining k=16k=16 components typically captures main variance in the normal training patches.
    • Sensitivity analysis (Appendix P, "Sensitivity Analysis for Singular Vectors"): We evaluated k∈8,12,16,20,24k \in \\{8, 12, 16, 20, 24\\} across three TSFMs (ROSE, TMM, UniTS) on ten datasets. As reported by the results in Appendix P, the performance of our method is relatively stable in the range k∈[12,20]k \in [12, 20], with k=16k=16 delivering good performance while remaining computationally modest.
  2. NN (number of training patches): We obtain NN training patches from five normal subsequences across all experiments. This extreme few-shot setting is motivated by:
    • Theorem 2 (Appendix B.1): The subspace estimation error decreases as O(C/N)O(C/\sqrt{N}). With five samples and typical patch counts (16–32 patches per sample), the total number of patches is 80–160, which, as Theorem 2 shows, is sufficient for stable subspace estimation.
    • Empirical validation (Appendix O, "Utility of Few-Shot Normal Samples"): We evaluate sample sizes ranging from 1 to 20 on three TSFMs across ten datasets. Performance improves with sample size but exhibits diminishing returns beyond five samples. We choose only five samples as a practical default that balances performance and data efficiency.
  3. λ\boldsymbol{\lambda} (fusion weights): These are learnable parameters optimized via backpropagation to minimize the loss on the few-shot normal training samples. These are learnable parameters optimized via backpropagation to minimize the loss on the few-shot normal training samples. They are initialized to be 0.2 (equal weighting) and adapted during training through gradient descent. Specifically, at each training epoch, we compute the gradients ∂L∂λ\frac{\partial \mathcal{L}}{\partial \boldsymbol{\lambda}} where L\mathcal{L} is the total loss (Equation 11) evaluated on normal samples, and update λ\boldsymbol{\lambda} via: \begin{equation} \boldsymbol{\lambda} \leftarrow \boldsymbol{\lambda} - \eta \frac{\partial \mathcal{L}}{\partial \boldsymbol{\lambda}}, \end{equation} where η\eta is the learning rate. No anomaly data is involved in this optimization.

Another concern you raised is: "If you optimize for the best reconstruction of normal data, then you probably end up with the original problem (that you make the model as complex as possible such that all data, including anomalies, can be perfectly reproduced)." This concern is addressed by two mechanisms:

  1. The TSFM is frozen. We do not optimize the TSFM's parameters, which means we do not increase the model's capacity to fit arbitrary patterns. The TSFM's broad pattern-modeling capability (which causes it to reconstruct anomalies well) remains unchanged. NCL only learns a low-dimensional projection and re-weighting of the TSFM's frozen features.
  2. NCL constrains the feature space to a low-dimensional normal subspace. The PCA step explicitly restricts the model to operate within a kk-dimensional subspace (k=16≪d≈512–1024k = 16 \ll d \approx 512–1024 for typical TSFMs). This reduces the effective capacity relative to the full TSFM feature space. The learnable transformation (PatchNT) has only 2kd2kd parameters (the linear heads for α\boldsymbol{\alpha} and β\boldsymbol{\beta}), which is tiny compared to the frozen TSFM (millions of parameters). The contrastive constraint (ConNC) further regularizes this transformation to form compact normal manifolds.

In summary, optimizing for normality learning within a low-dimensional PCA subspace does not increase the model's ability to fit anomalies, instead, it constrains the model to the normal subspace, which is precisely the goal. The frozen TSFM provides the initial broad representations, and NCL "narrows" them to normality.

AuthorsOpenReview ↗

Response to Reviewer MwGp (Part 3/3)

Additional Clarifications

Reference to PCA-based anomaly detection: Thank you for this suggestion. In the Related Work section of the revised submission, we have added several references to classic PCA-based anomaly detection methods to provide better connections to prior related studies, and at the same time, we also positioned PCA-based anomaly detection in the context of TSFMs by clarifying how NCL is related to the PCA detector via NCLN\text{NCL}_{\text{N}}. NCLN\text{NCL}_{\text{N}} is a variant of our method NCL that is equivalent to using PCA on top of TSFM's representation space for anomaly detection. This variant retains the sample efficiency and interpretability of PCA while leveraging the powerful representational capacity of frozen TSFMs. The subsequent PatchNT and ConNC in NCL components further augment this simple variant with adaptive transformation and geometric regularization, enabling greater discriminative power for anomaly detection than vanilla TSFMs.

Reviewer_fhasOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NCL, a lightweight adaptation framework for frozen time-series foundation models. It estimates normal feature directions by SVD, learns patch-adaptive directional transformations with a contrastive objective, and fuses a learned prediction branch with the original model output. Inference additionally introduces a view-misalignment score. Experiments report improvements across multiple forecasting and reconstruction backbones and ten anomaly-detection datasets.

Critical strengths
  1. The problem is relevant: a broadly capable foundation model need not provide a discriminative anomaly score in a particular target domain. Adaptation using only a few normal subsequences is practically attractive.
  2. The empirical coverage is substantial. Table 1 provides paired comparisons across eight backbones and ten datasets; the appendices add further backbones, simple statistical baselines, specialized anomaly-detection foundation models, multiple metrics, and run variability. The component and hyperparameter studies are useful.
Critical weaknesses

1. The error-amplification proof does not analyze the implemented predictor. Equation (9) fuses the outputs of the original model and a separate NCL decoder. However, Appendix A.2.1, Eq. (32), identifies this output with the frozen model evaluated on a fused input. Neither the architecture nor the assumptions establish that identity. Moreover, Eq. (31) assumes both arguments lie outside the original support neighborhood, whereas the subsequent argument uses a reference point in the support. For the actual scalar fusion, setting e=Y−Yhat and d=Y_ncl−Yhat gives an error change of λ²‖d‖²−2λ〈e,d〉, whose sign requires conditions on residual alignment and magnitude. The current proof therefore does not establish Theorem 1 for NCL. Appendix A.2.2 also reverses the hinge-loss condition: violating the margin in Eq. (8) gives positive rather than zero loss, and even zero contrastive loss would not eliminate prediction-loss updates. These issues undermine the stated guarantees without demonstrating that the empirical gains are absent.

2. Several subspace guarantees are overstated or incorrect. Theorem 3 assumes decoder sensitivity follows singular-value order merely because the decoder is well trained. A counterexample is Z=diag(2,1), k=1, and w=(0,1)ᵀ, with target Zw: retaining the leading singular direction gives decoded squared error 1, while retaining the second gives error 0. Theorem 5 similarly transfers optimal approximation of noisy Z to optimal recovery of clean E; an estimator computed from Z is generally correlated with its noise, invalidating the discarded cross term in Eq. (73). Finally, the numerical errors following Eq. (57) are obtained from 1/√N despite the eigengap and other constants in Eq. (56), and the i.i.d. patch assumption is not connected to the contextual patches from five subsequences. The empirical shot-count sweep is valuable but does not validate those guarantees. The claims should be corrected or narrowed to what is actually established.

Questions for authors
  1. Error amplification: Can Theorem 1 be established for the actual output fusion in Eq. (9), without identifying it with input fusion? Please specify the conditions on residual alignment, correction magnitude, and fusion weights, and address the domain mismatch when applying Eq. (31).

  2. Contaminated training data: How can Appendix A.2.2 establish protection against anomaly fitting when margin violations yield positive hinge loss and prediction-loss updates remain possible even with zero contrastive loss?

  3. Subspace guarantees: Can you address the counterexample to Theorem 3 and justify discarding the noise-dependent cross term in Theorem 5? Please also clarify which finite-sample conclusions remain valid when retaining the constants in Eq. (56) and accounting for dependence among contextual patches.

11 comments
AuthorsOpenReview ↗

Response to Reviewer fhas (Part 1/7)

Dear Reviewer fhas,

We sincerely thank you for the thorough and meticulous reading of our paper. Your critical comments provide us with important guidance in refining the submission, particularly the proofs in the appendices. We address your concerns one-by-one below.


W1.1: Equation (9) fuses the outputs of the original model and a separate NCL decoder. However, Appendix A.2.1, Eq. (32), identifies this output with the frozen model evaluated on a fused input.

Thank you for identifying this conflation. Our initial intention was that due to the black-box nature of neural networks, we cannot directly characterize the effect of transformed features on the output in a closed form. Therefore, we assumed the Lipschitz continuity property of neural networks (this is a widely used assumption[1-4]), and further started from the mapping relationship between the input and output.

However, as you (and Reviewer BQgq) kindly pointed out, in our subsequent analysis, we conflated two independent decoding processes. The original text claimed "Let X~a\tilde{\mathbf{X}}_a denote the λ\lambda-fusion of the original input and the input reconstructed from the transformed patch features z~a(i)i=1k\\{{\tilde{\mathbf{z}}^{(i)}_a}\\}_{i=1}^k," which implicitly equated the pre-training model decoding process with the NCL decoding process. We instead should only analyze the impact brought by the transformation separately, rather than reversing it back to the fused input in advance. According to this logic, we have further modified the proof in Appendix A.2.1, and we present several key changes below.

Our revision now conducts theoretical analysis directly in output space. We explicitly formalize the NCL-augmented prediction as: \begin{equation} \mathcal{F}^{\prime}(\mathbf{X}_a) = \tilde{\mathbf{Y}}_a = \boldsymbol{\lambda} \odot \tilde{\mathbf{Y}}_a^{\mathrm{ncl}} + (1 - \boldsymbol{\lambda}) \odot \hat{\mathbf{Y}}_a = \boldsymbol{\lambda} \odot \mathcal{F}^{\mathrm{ncl}}(\mathbf{X}_a) + (1 - \boldsymbol{\lambda}) \odot \mathcal{F}^*(\mathbf{X}_a), \end{equation} where Fncl(Xa)=Fdencl(T(F∗_en(Xa)))\mathcal{F}^{\mathrm{ncl}}(\mathbf{X}_a) = \mathcal{F}^{\mathrm{ncl}}_{\mathrm{de}}(T(\mathcal{F}^*\_{\mathrm{en}}(\mathbf{X}_a))), F∗_en(⋅)\mathcal{F}^*\_{\mathrm{en}}(\cdot) is the encoder of the pre-trained model, T(⋅)T(\cdot) is the normality transformation, and Fdencl(⋅)\mathcal{F}^{\mathrm{ncl}}_{\mathrm{de}}(\cdot) is the NCL linear decoding head. We then regard the feature transformation as an equivalent input space shift, i.e., we let X~ancl\tilde{\mathbf{X}}^{\mathrm{ncl}}_a denote the transformed input whose extracted feature equals z~a=1k∑i=1kz~a(i)\tilde{\mathbf{z}}_a = \frac{1}{k}\sum_{i=1}^k \tilde{\mathbf{z}}_a^{(i)}, so that \begin{equation} \mathcal{F}^{\mathrm{ncl}}(\mathbf{X}_a) = \mathcal{F}^{\mathrm{ncl}}_{\mathrm{de}}(\mathcal{F}^*_{\mathrm{en}}(\tilde{\mathbf{X}}^{\mathrm{ncl}}_a)). \end{equation} Critically, for theoretical analysis, we decompose the NCL decoding process to independently characterize the normality transformation and output fusion, without conflating them with the pre-trained model's behavior. The displacement τ\tau induced by the normality transformation is applied to characterize how X~ancl\tilde{\mathbf{X}}^{\mathrm{ncl}}_a moves away from Sj\mathcal{S}_j, and the subsequent Lipschitz-based argument operates on Fdencl∘F∗_en\mathcal{F}^{\mathrm{ncl}}_{\mathrm{de}} \circ \mathcal{F}^*\_{\mathrm{en}} rather than on F∗\mathcal{F}^* directly. The updated proof in Appendix A.2.1 of the revised submission is now established for the actual output fusion in Equation (9), without identifying it with input fusion.

[1] Khromov, Grigory, and Sidak Pal Singh. "Some fundamental aspects about lipschitz continuity of neural networks." International conference on learning representations. Vol. 2024. 2024.

[2] Asadi, Kavosh, Dipendra Misra, and Michael Littman. "Lipschitz continuity in model-based reinforcement learning." International conference on machine learning. PMLR, 2018.

[3] Shang, Yuzhang, et al. "Lipschitz continuity retained binary neural network." European conference on computer vision. Cham: Springer Nature Switzerland, 2022.

[4] Kobayashi, Taisuke. "L2c2: Locally lipschitz continuous constraint towards stable and smooth reinforcement learning." 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2022.

AuthorsOpenReview ↗

Response to Reviewer fhas (Part 2/7)

W1.2: Domain mismatch in Assumption 1 (Eq. 31).

Thank you for pointing out the mistakes in the assumption. In fact, here we should define all X1∈Nρ+τ(Sj)∖Nρ(Sj)\mathbf{X}_1 \in \mathcal{N}_{\rho + \tau}(\mathcal{S}_j) \setminus \mathcal{N}_{\rho}(\mathcal{S}_j), X2∈Nρ(Sj)\mathbf{X}_2 \in \mathcal{N}_{\rho}(\mathcal{S}_j), so that we can further analyze the relationship between the internal and external regions of the ρ\rho domain of Sj\mathcal{S}_j.

In the revised version, the assumption is corrected to:

There exists a strictly increasing ω:[0,∞)→[0,∞)\omega:[0,\infty)\to[0,\infty) with ω(0)=0\omega(0)=0 such that for all X1∈Nρ+τ(Sj)∖Nρ(Sj),X2∈Nρ(Sj)\mathbf{X}_1\in\mathcal{N}_{\rho + \tau}(\mathcal{S}_j) \setminus \mathcal{N}_{\rho}(\mathcal{S}_j),\mathbf{X}_2\in\mathcal{N}_{\rho}(\mathcal{S}_j), \begin{equation} \omega\big(||\mathbf{X}_1-\mathbf{X}_2||_2\big) \le ||\mathcal{F}^*_{\mathrm{en}}(\mathbf{X}_1)-\mathcal{F}^*_{\mathrm{en}}(\mathbf{X}_2)||_2 \le L_{\mathcal{F}}||\mathbf{X}_1-\mathbf{X}_2||_2 . \end{equation}

With this corrected assumption, we further proceed with the proof by applying the reverse triangle inequality with X~∈Sj\widetilde{\mathbf{X}} \in \mathcal{S}_j (which is in Nρ(Sj)\mathcal{N}_{\rho}(\mathcal{S}_j)) and X~ancl\tilde{\mathbf{X}}^{\mathrm{ncl}}_a (which lies in Nρ+τ(Sj)∖Nρ(Sj)\mathcal{N}_{\rho+\tau}(\mathcal{S}_j)\setminus\mathcal{N}_{\rho}(\mathcal{S}_j)).


W1.3: Sign condition for the error change λ2∥∣Y~ancl−Y^a∥∣22−2λ<Ya−Y^a,Y~ancl−Y^a>\lambda^2\||\tilde{\mathbf{Y}}_a^{\mathrm{ncl}}-\hat{\mathbf{Y}}_a \||_2^2 - 2 \lambda <\mathbf{Y}_a-\hat{\mathbf{Y}}_a, \tilde{\mathbf{Y}}_a^{\mathrm{ncl}}-\hat{\mathbf{Y}}_a> under output fusion.

We respectfully argue that your subsequent understanding was based on the perspective of square decomposition for the proof. That is:

\begin{align} ||\mathbf{Y}_a-\tilde{\mathbf{Y}}_a||_2^2 - ||\mathbf{Y}_a-\hat{\mathbf{Y}}_a||_2^2 &= ||\mathbf{Y}_a-\lambda \tilde{\mathbf{Y}}_a^{\mathrm{ncl}} - (1-\lambda) \hat{\mathbf{Y}}_a ||_2^2 - ||\mathbf{Y}_a-\hat{\mathbf{Y}}_a||_2^2 \\ &= ||(\mathbf{Y}_a-\hat{\mathbf{Y}}_a) -\lambda (\tilde{\mathbf{Y}}_a^{\mathrm{ncl}}-\hat{\mathbf{Y}}_a) ||_2^2 - ||\mathbf{Y}_a-\hat{\mathbf{Y}}_a||_2^2 \\ &= \lambda^2||\tilde{\mathbf{Y}}_a^{\mathrm{ncl}}-\hat{\mathbf{Y}}_a ||_2^2 - 2 \lambda <\mathbf{Y}_a-\hat{\mathbf{Y}}_a, \tilde{\mathbf{Y}}_a^{\mathrm{ncl}}-\hat{\mathbf{Y}}_a>. \end{align}

If we continue from this perspective, a simple assumption could be that the directions of Y^a−Ya\hat{\mathbf{Y}}_a - \mathbf{Y}_a and Y~ancl−Y^a\tilde{\mathbf{Y}}_a^{\mathrm{ncl}} - \hat{\mathbf{Y}}_a are the same, meaning that the errors should be amplified in the same direction without being cancelled out in the opposite direction. This would lead to ∥Ya−Y~a∥22−∥Ya−Y^a∥22≥0\|\mathbf{Y}_a - \tilde{\mathbf{Y}}_a\|_2^2 - \|\mathbf{Y}_a - \hat{\mathbf{Y}}_a\|_2^2 \ge 0. However, this assumption is too strong. In the revised proof, we adopted another analytical approach. We avoid analyzing the squared norm difference directly and instead work with the norm difference via the triangle inequality:

\begin{align} ||\mathbf{Y}_a-\tilde{\mathbf{Y}}_a||_2 - ||\mathbf{Y}_a-\hat{\mathbf{Y}}_a||_2 &= ||\mathbf{Y}_a-\lambda \tilde{\mathbf{Y}}_a^{\mathrm{ncl}} - (1-\lambda) \hat{\mathbf{Y}}_a ||_2 - ||\mathbf{Y}_a-\hat{\mathbf{Y}}_a||_2 \\ &= ||\lambda(\mathbf{Y}_a-\tilde{\mathbf{Y}}_a^{\mathrm{ncl}}) + (1-\lambda) (\mathbf{Y}_a-\hat{\mathbf{Y}}_a) ||_2 - ||\mathbf{Y}_a-\hat{\mathbf{Y}}_a||_2 \\ &\ge \lambda||\mathbf{Y}_a-\tilde{\mathbf{Y}}_a^{\mathrm{ncl}}||_2 - (2-\lambda)||\mathbf{Y}_a-\hat{\mathbf{Y}}_a||_2. \end{align}

This decouples the analysis into two independent components, ∥∣Ya−Y~ancl∥∣2\||\mathbf{Y}_a - \tilde{\mathbf{Y}}_a^{\mathrm{ncl}}\||_2 and ∥Ya−Y^a∥∣2\|\mathbf{Y}_a - \hat{\mathbf{Y}}_a\||_2, avoiding overly complex analysis of the residual cross term ⟨Ya−Y^a,Y~ancl−Y^a⟩\langle\mathbf{Y}_a - \hat{\mathbf{Y}}_a, \tilde{\mathbf{Y}}_a^{\mathrm{ncl}} - \hat{\mathbf{Y}}_a\rangle. The lower bound on ∥∣Ya−Y~ancl∥∣2\||\mathbf{Y}_a - \tilde{\mathbf{Y}}_a^{\mathrm{ncl}}\||_2 then follows from the corrected Assumption (discussed in W1.2) together with the linear decoder bound, and the upper bound on ∥∣Ya−Y^a∥∣2\||\mathbf{Y}_a - \hat{\mathbf{Y}}_a\||_2 follows from the Lipschitz-based over-generalization result in Appendix A.1. The final inequality ∥∣Ya−Y~a∥∣22>∥∣Ya−Y^a∥∣22\||\mathbf{Y}_a - \tilde{\mathbf{Y}}_a\||_2^2 > \||\mathbf{Y}_a - \hat{\mathbf{Y}}_a\||_2^2 follows when κ\kappa satisfies the stated lower bound. The updated derivation is provided in Appendix A.2.1 of the revised submission.

AuthorsOpenReview ↗

Response to Reviewer fhas (Part 3/7)

W1 / Q1: The current proof therefore does not establish Theorem 1 for NCL. Can Theorem 1 be established for the actual output fusion in Eq. (9), without identifying it with input fusion?

Based on the three detailed issues (W1.1, W1.2, and W1.3) you raised, we have reorganized the proof of Theorem 1. Currently, we conduct the revised proof to establish Theorem 1 directly from the perspective of actual output fusion, and the detailed proof can be found in Appendix A.2.1 of the revised submission. We have made corrections in the revised proof in response to W1.1, W1.2, and W1.3:

  1. W1.1 (architecture mismatch): The revised proof explicitly decomposes F′(Xa)\mathcal{F}^{\prime}(\mathbf{X}_a) as an output fusion of two separate computation paths, the frozen TSFM and the NCL, and then further analyzes the role of the NCL output in the fusion process.
  2. W1.2 (domain mismatch): Assumption 1 is corrected so that we can further analyze the relationship between a point outside Nρ(Sj)\mathcal{N}_{\rho}(\mathcal{S}_j) and a point inside Nρ(Sj)\mathcal{N}_{\rho}(\mathcal{S}_j).
  3. W1.3 (residual cross term): We replace the squared-norm decomposition with an inequality on the norm, which avoids the residual cross term entirely and requires no assumption on the alignment of Y^a−Ya\hat{\mathbf{Y}}_a - \mathbf{Y}_a with Y~ancl−Y^a\tilde{\mathbf{Y}}_a^{\mathrm{ncl}} - \hat{\mathbf{Y}}_a.

Based on the revisions made in response to your suggestions, I believe that the revised proof accurately establishes Theorem 1 for NCL.

We are grateful to you for identifying these issues, which have further enhanced the completeness of our proof.

Reviewer_BQgqOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Normality Constraint Learning (NCL) to improve anomaly detection using Time Series Foundation Models (TSFMs). The main idea is to adjust features along the principal directions of normal data so that the adjustment is more effective for normal data than for anomalies. Experiments cover both reconstruction- and forecasting-based backbones on ten datasets, with reported improvements after adaptation using only five normal samples.

Critical strengths
  1. Using SVD to characterize normal data: Using SVD to identify the dominant directions of normal data is a key strength of this paper. Adjusting features along these directions may emphasize normality and make it easier to distinguish normal data from anomalies.
  2. Applicability to existing foundation models: As shown in Table 1, the proposed approach can be applied to both reconstruction- and forecasting-based TSFMs. The results are impressive compared with LoRA and CoRA.
Critical weaknesses
  1. Proof of Theorem 1: In Equation (32), the output Y~a\widetilde{Y}_a is replaced with F(X~a)F^(\widetilde{X}_a) without showing that the actual decoder-and-gate architecture has this form. On page 4, the authors write Y~t=F′(Xt−L)\widetilde{Y}t = F'(X{t-L}). The relationship between F′(X)F'(X) and F(X~)F^(\widetilde{X}) needs to be clarified, particularly whether F′(X)=F∗(X~)F'(X) = F^*(\widetilde{X}) holds for the proposed architecture.

  2. Meaning of a well-trained model: The meaning of well-trained is not clearly explained in the main text, although it is discussed in the Appendix. The referee is not convinced that being well-trained necessarily implies c1≥c2≥⋯≥crc_1 \ge c_2 \ge \cdots \ge c_r. The authors should provide theoretical justification or empirical evidence that well-trained decoders exhibit this ordering.

Questions for authors
  1. Can the authors prove Theorem 1 using the actual output fusion in Equation (9)?
  2. Can the authors provide theoretical or empirical evidence that well-trained decoders satisfy c1≥c2≥⋯≥crc_1 \ge c_2 \ge \cdots \ge c_r?
10 comments
AuthorsOpenReview ↗

Response to Reviewer BQgq (Part 1/2)

Dear Reviewer BQgq,

We sincerely thank you for the thoughtful evaluation and constructive feedback. We address each concern below with detailed responses and clarifications.


W1/Q1: In Equation (32), the output Y~a\tilde{\mathbf{Y}}_a is replaced with F∗(X~a)\mathcal{F}^*(\tilde{\mathbf{X}}_a) without showing that the actual decoder-and-gate architecture has this form. Can the authors prove Theorem 1 using the actual output fusion in Equation (9)?

Thank you for pointing out this issue. In Appendix A.2.1, Equation (32) in the original proof implicitly equated the NCL-augmented output Y~a\tilde{\mathbf{Y}}_a with the frozen TSFM evaluated on a fused input X~a\tilde{\mathbf{X}}_a. This identity does not hold for the actual architecture described in Equation (9), where two separate decoding paths (the frozen TSFM and the NCL) produce outputs that are then fused. Therefore, as you noted, F′(X)=F∗(X~)\mathcal{F}^{\prime}(\mathbf{X}) = \mathcal{F}^*(\tilde{\mathbf{X}}) is unjustified in our architecture. Actually, we should only analyze the impact brought by the transformation separately, rather than reversing it back to the fused input in advance. According to this logic, in Appendix A.2.1 of the revised submission, we have directly proved Theorem 1 using the actual output fusion in Equation (9).

Specifically, we explicitly decompose the NCL-augmented prediction as:

\begin{equation} \mathcal{F}^{\prime}(\mathbf{X}_a) = \tilde{\mathbf{Y}}_a = \boldsymbol{\lambda} \odot \tilde{\mathbf{Y}}_a^{\mathrm{ncl}} + (1 - \boldsymbol{\lambda}) \odot \hat{\mathbf{Y}}_a = \boldsymbol{\lambda} \odot \mathcal{F}^{\mathrm{ncl}}(\mathbf{X}_a) + (1 - \boldsymbol{\lambda}) \odot \mathcal{F}^*(\mathbf{X}_a), \end{equation} where Fncl(Xa)=Fdencl(T(F∗_en(Xa)))\mathcal{F}^{\mathrm{ncl}}(\mathbf{X}_a) = \mathcal{F}^{\mathrm{ncl}}_{\mathrm{de}}(T(\mathcal{F}^*\_{\mathrm{en}}(\mathbf{X}_a))), F∗_en(⋅)\mathcal{F}^*\_{\mathrm{en}}(\cdot) is the encoder of the pre-trained model, T(⋅)T(\cdot) is the normality transformation, and Fdencl(⋅)\mathcal{F}^{\mathrm{ncl}}_{\mathrm{de}}(\cdot) is the NCL linear decoding head. We then regard the feature transformation as an equivalent input space shift, i.e., we let X~ancl\tilde{\mathbf{X}}^{\mathrm{ncl}}_a denote the transformed input whose extracted feature equals z~a=1k∑i=1kz~a(i)\tilde{\mathbf{z}}_a = \frac{1}{k}\sum_{i=1}^k \tilde{\mathbf{z}}_a^{(i)}, so that \begin{equation} \mathcal{F}^{\mathrm{ncl}}(\mathbf{X}_a) = \mathcal{F}^{\mathrm{ncl}}_{\mathrm{de}}(\mathcal{F}^*_{\mathrm{en}}(\tilde{\mathbf{X}}^{\mathrm{ncl}}_a)). \end{equation} Critically, for theoretical analysis, we decompose the NCL decoding process to independently characterize the normality transformation and output fusion, without conflating them with the pre-trained model's behavior. The displacement τ\tau induced by the normality transformation is applied to characterize how X~ancl\tilde{\mathbf{X}}^{\mathrm{ncl}}_a moves away from Sj\mathcal{S}_j, and the subsequent Lipschitz-based argument operates on Fdencl∘F∗_en\mathcal{F}^{\mathrm{ncl}}_{\mathrm{de}} \circ \mathcal{F}^*\_{\mathrm{en}} rather than on F∗\mathcal{F}^* directly. The updated proof in Appendix A.2.1 of the revised submission is now established for the actual output fusion in Equation (9), without identifying it with input fusion.

AuthorsOpenReview ↗

Response to Reviewer BQgq (Part 2/2)

W2 / Q2: The referee is not convinced that being well-trained necessarily implies c1≥c2≥⋯≥crc_1 \geq c_2 \geq \cdots \geq c_r. Can the authors provide theoretical or empirical evidence that well-trained decoders satisfy c1≥c2≥⋯≥crc_1 \geq c_2 \geq \cdots \geq c_r?

Thank you for this important concern. When we assumed a well-trained decoder, we indeed made the assumption that the contribution strength of the right singular vector vi\mathbf{v}_i to the decoder's output increases monotonically with the corresponding singular value σi\sigma_i, that is, ci≜∑l=1s(vi⊤wl)2c_i \triangleq \sum_{l=1}^{s} (\mathbf{v}_i^{\top} \mathbf{w}_l)^2, where c1≥c2≥⋯≥cr≥0c_1 \geq c_2 \geq \cdots \geq c_r \geq 0.

This is due to our intuition that a well-trained decoder should prioritize the singular vectors corresponding to high singular values. However, we did not clearly define what "well-trained" means beforehand. The same issue is also raised by Reviewer fnas. Therefore, a solution to eliminate ambiguity is to construct a stronger assumption, that is, to directly assume that the decoder sensitivity follows the singular-value order. However, the theorem derivation under this strong assumption lacks practical guiding implications.

Thus, we re-examined whether we could directly analyze the information differences of the matrices Zˉ\bar{\mathbf{Z}} and Z^\hat{\mathbf{Z}}. We further abandoned the ambiguous decoder definition that was introduced earlier and directly analyzed the Frobenius-norm differences between the matrices Zˉ\bar{\mathbf{Z}} (or Z^\hat{\mathbf{Z}}) and the original matrix Z\mathbf{Z}, thereby proving that the top-kk truncated approximation retains more information than arbitrary kk approximation. The detailed proof of this theoretical argument has been updated in Appendix B.2.1.

To empirically verify this, We have also conducted a comparison between NCL (arbitrary kk approximation) and NCL (top-kk approximation) on IOPS and PSM datasets with three foundation models (ROSE, TMM, UniTS):

Variant\Dataset-ModelIOPS-ROSEIOPS-TMMIOPS-UniTSPSM-ROSEPSM-TMMPSM-UniTS
NCL (arbitrary kk approximation)0.19030.31740.17180.40340.48560.4690
NCL (top-kk approximation)0.19360.33380.18610.41070.50390.4735

From the comparison results, it can be observed that compared to arbitrary kk approximation, top-kk approximation can retain more normal information, thereby enhancing the anomaly detection performance.

In summary, we eliminate the argument associated with the so-called "well-trained decoder", and instead directly examine the Frobenius norm differences between the matrices Zˉ\bar{\mathbf{Z}} (or Z^\hat{\mathbf{Z}}) and the original matrix Z\mathbf{Z}. This helps establish an alternative argument that the top-kk truncated approximation retains more information than arbitrary kk approximation, with the revised proof presented in Appendix B.2.1 of the revised submission.

Thank you for the response. The proposed revision to Theorem 3 is reasonable, provided the theorem and its interpretation are limited to Frobenius approximation error.

For Theorem 1, the corrected decomposition alone does not establish error amplification after output fusion. Please clarify which assumptions and inequalities in the revised proof prevent cancellation between the two prediction errors. Please also justify the existence of an input corresponding to the transformed features.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear Reviewers and Area Chair,

We sincerely thank all three reviewers (BQgq, fhas, and MwGp) for the valuable time and effort spent reviewing our paper.

We are encouraged that the reviewers recognize the core contributions of our work: adjusting TSFM features along the principal directions to sharpen the normality, the broad applicability across diverse TSFMs, and the extensive empirical validation.

We appreciate your constructive feedback on our paper, particularly the detailed technical concerns regarding theoretical proofs. In response to the comments, we have carefully revised and enhanced the paper as follows:

  • Revision of Theorem 1 proof to establish error amplification for the actual output fusion without input-fusion conflation (Appendix A.2.1)
  • Elimination of "well-trained decoder" assumption and addressing the issue with a direct proof of the Frobenius norm differences between the matrices Zˉ\bar{\mathbf{Z}} (or Z^\hat{\mathbf{Z}}) and the original matrix Z\mathbf{Z} (Appendix B.2.1)
  • Correction of contrastive loss analysis to clarify that LConNC\mathcal{L}_\mathrm{ConNC} functions as an effective geometric regularizer against anomaly overfitting (Section 4.5, Appendix A.2.2)
  • Explicit justification of cross-term treatment in noisy SVD (Appendix B.2.3)
  • Retention of full constants and provision of numerical analysis examples. (Appendix B.1)
  • Replacement of i.i.d. patch assumption with ϕ\phi-mixing condition to handle temporal dependence (Appendix B.1)
  • Enhanced ablation analysis demonstrating PCA initialization is essential (extracted from Table 2, discussed in response to Reviewer MwGp)
  • Clarification of unsupervised hyperparameter selection for kk, NN, and λ\boldsymbol{\lambda} (Appendix D.3)
  • Addition of classical PCA-based anomaly detection references and explicit connection to NCLN\text{NCL}_{\text{N}} variant (Sections 2.1 and 2.2)

In the revised manuscript, these updates are temporarily highlighted in blue\textcolor{blue}{\text{blue}} for your convenience to check.


Update (Nov. 16th): we are extremely grateful to Reviewer BQgq for providing further guidance. We have added the justification about the existence of an input corresponding to the transformed features in Appendix A.2.1 and highlighted it in red\textcolor{red}{\text{red}} for easy identification.

Update (Nov. 18th): we are extremely grateful to all reviewers for the valuable insights. We have conducted the proof of Theorem 1 directly starting from the normality transformation in Appendix A.2.1, and have also adjusted the statement of Theorem 1 in Section 4.5. From the perspective of obtaining the upper bound of the Frobenius norm error, Theorem 5 has been adjusted in Appendix B.2 and proved in Appendix B.2.3. For easy identification, we have highlighted the above revised parts in red\textcolor{red}{\text{red}}.


We sincerely believe that these updates fully address the raised concerns and may help us better deliver the benefits of the proposed NCL to the ICLR community.

Thank you very much,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.