acceptodds
Under review as a conference paper at ICLR 2027

Same Magnitude, Different Direction: Signed-Orientation Response in Time-Series Anomaly Detection

Abstract

Modern time-series anomaly detectors respond very differently to signed cross-channel relations, and the corresponding score component can help or harm detection on natural data. We establish these differences using a partial channel sign flip that preserves every coordinatewise magnitude. Across nine released models on 65 multivariate views, responses range from near chance to strong separation; longer training changes them unevenly. An exact even–odd decomposition isolates the flip-sensitive component of the same trained score, linking the controlled comparison to a natural-detection ablation. At 50 epochs, removing this component changes magnitude-matched AUC by more than one percentage point in 19 of 54 model–dataset combinations and by as much as 14.37 points. Yet Anomaly Transformer and GDN, only 1.00 native-AUC point apart on a common panel, differ by 11.36 points in response strength. Similar native performance is therefore insufficient evidence of similar signed response: this behavior and its effect on detection require direct measurement.

Primary area:
learning on time series and dynamical systems
Keywords:
time series anomaly detection, multivariate time series, benchmark evaluation, model underspecification, cross-channel dependence, invariance testing

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_e3xYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  • This paper investigates signed-orientation response in trained time-series anomaly detection (TSAD) scores, particularly whether models with similar native AUC exhibit different responses to the same sign-flip transformation. The authors introduce a partial channel sign flip that preserves every coordinatewise magnitude while reversing signed products across the flipped/unflipped channel partition. Response strength is measured through rank separation between original and flipped normal windows.

  • An exact even–odd decomposition separates the same trained score into flip-invariant and flip-sensitive parts. The authors compare the full score with its invariant part on labeled natural data to examine whether the flip-sensitive component helps or harms detection, without retraining.

  • Nine released TSAD implementations are evaluated across 65 multivariate views. A separate common panel of 56 views from six datasets supports paired natural-detection and training-budget comparisons. At 50 epochs, removing the flip-sensitive component changes magnitude-matched AUC by more than one percentage point in 19 of 54 model–dataset combinations, with effects in both directions. The experiments also show that similar native AUCs can coexist with large differences in response strength.

Critical strengths

1. Careful empirical protocol.

  • Nine publicly released TSAD models are evaluated under a common preprocessing and evaluation protocol, with frozen code revisions and hierarchical-bootstrap confidence intervals.
  • The high-response subgroup and magnitude matching were specified before examining the relevant natural-detection results.

2. Connection to natural detection.

  • The exact even-odd decomposition isolates the flip-sensitive component of the same trained score without retraining. Removing it changes magnitude-matched AUC by more than one percentage point in 19 of 54 model-dataset pairs at 50 epochs. It shows that response strength and detection utility are distinct.
Critical weaknesses

1. Unclear task-level significance and added value

  • With flipped windows as the positive class, ATA_T measures rank separation between original and flipped normal windows, that is, how well a score detects one synthetic perturbation of normal data. That models with similar native AUC respond differently to this flip is not surprising: Anomaly Transformer and GDN differ by 1.00 native-AUC point but by 11.36 points in response strength (Abstract; Table 9). The paper does not show whether this response corresponds to natural anomalies in the evaluated datasets, or what it adds beyond the behavioral-testing and underspecification work it cites (§2; Ribeiro et al., 2020; D'Amour et al., 2022).

  • Larger responses tend to accompany larger natural changes (descriptive r≈0.89r \approx 0.89 across nine model averages at 50 epochs; Figure 3b; Table 11). However, this association concerns the size of natural detection changes, not whether the flip-sensitive component helps or hurts detection. The flip-sensitive component changes Anomaly Transformer's magnitude-matched AUC by +14.4 points on HAI and −3.1 on PSM at 50 epochs (Figure 3a). Without labels, the audit cannot determine whether a stronger response is desirable; with labels, DnatD_{\mathrm{nat}} is a more direct measure. The added evaluation or deployment value of CT\mathcal{C}_T therefore remains unclear.

2. Limitations of the response metric (CT\mathcal{C}_T) and incomplete attribution

  • CT\mathcal{C}_T summarizes rank separation on normal windows, whereas the odd score component gives the pointwise flip-sensitive contribution (§3.3). DCdetector has CT≈0.01\mathcal{C}_T \approx 0.01 at both budgets, yet removing its flip-sensitive component changes magnitude-matched AUC by about 1.9 points. This is comparable to TimesNet at 5 epochs (CT=0.13\mathcal{C}_T = 0.13, 1.8 points; Table 11). The USAD–DCdetector contrast in §6 therefore characterizes rank separation, not whether DCdetector's flip-sensitive component is negligible.

  • Appendix B gives no quantitative results for the signed-marginal audit or the relational-shuffle control, and its focused sanity checks use constructed scorers rather than the nine released models. DnatD_{\mathrm{nat}} is also reported only for the canonical group (§4; §5.2), whereas Appendix L examines channel-group sensitivity for CT\mathcal{C}_T alone.

Questions for authors

Q1. Task-level significance (Weakness 1)

  • Do the labeled anomalies in the evaluated datasets (e.g., HAI or SWaT attacks) exhibit changes in signed cross-channel relations that the sign-flip audit is intended to probe?
  • Since the sign of DnatD_{\mathrm{nat}} varies across datasets, how should CT\mathcal{C}_T be used when labels are unavailable, and what does it add beyond the behavioral-testing and underspecification work cited in §2?

Q2. Interpretation of low response strength (Weakness 2)

  • DCdetector has near-chance response strength (about 0.01), yet removing its flip-sensitive component changes magnitude-matched AUC by about 1.9 points (Table 11). How does this affect the USAD–DCdetector contrast in §6? Would the authors report a paired sensitivity measure, such as the mean absolute odd-score contribution normalized by the score spread on normal windows, alongside CT\mathcal{C}_T?

Q3. Attribution and robustness (Weakness 2)

  • Could the authors report the quantitative results of the signed-marginal audit and relational-shuffle control (Appendix B), ideally for the nine released models? Do the size and sign pattern of DnatD_{\mathrm{nat}} hold under the alternative channel groups examined in Appendix L?
3 comments
AuthorsOpenReview ↗

Practical value and paired sensitivity

Thank you for recognizing the empirical protocol and the connection to natural detection. Your distinction between distributional rank separation and pointwise sensitivity is important. We have added measurements addressing that distinction, practical use, and the missing attribution controls.

1. Natural relevance and the role of the audit (W1/Q1)

To examine natural anomalies, we measured cross-group signed relations relative to a training-normal reference on the original magnitude-matched windows. The resulting relation score has AUC 0.7102 on PSM, 0.3941 on HAI, and 0.5109 on SWaT. We retain the original direction of the statistic rather than reversing scores below 0.5.

As a temporal-alignment control, we circularly shifted the complementary channels within each window. Across the 49 nonzero shifts, median AUCs are 0.4893 for PSM, 0.5847 for HAI, and 0.4892 for SWaT. These results identify label-associated relation differences in some datasets, rather than a uniform pattern across all datasets.

Within the 11 HAI views where the statistic is defined, its correlation with signed DmatchedD_{\mathrm{matched}} is ρ=−0.845\rho=-0.845 for Anomaly Transformer and +0.736+0.736 for TimesNet. We will report these as descriptive associations, not as evidence that natural attacks reproduce the exact sign-flip intervention.

We agree that larger CTC_T does not establish that the response is desirable. Without anomaly labels, its role is to measure a particular behavioral difference between trained scores, including scores with similar native AUC. When labels are available, DD directly evaluates the effect of symmetrization; the two quantities answer different questions.

We also tested a concrete use of the decomposition: selecting between ss and sevens_{\mathrm{even}} using validation labels. For Anomaly Transformer, this improves held-out native AUC by 1.11 percentage points (95% CI [0.24, 2.17]) and matched AUC by 0.95 points ([0.05, 2.04]), relative to always using the original score. A broader model-then-variant selection analysis shows little additional gain, so we present this as a model-specific application.

Relative to the behavioral-testing and underspecification work cited in Section 2, our contribution is the specific magnitude-preserving intervention, its exact decomposition of the same trained score, and the empirical connection between this decomposition and natural detection. We will make this distinction explicit rather than present behavioral heterogeneity itself as the novelty.

2. Low response strength, paired sensitivity, and DCdetector (W2/Q2)

We agree that low CTC_T does not imply a negligible pointwise odd component. We now report the requested paired measure on normal windows:

PT=E∣sodd(X)∣SD⁡[s(X)].P_T=\frac{\mathbb{E}|s_{\mathrm{odd}}(X)|}{\operatorname{SD}[s(X)]}.

The following values are dataset-equal averages on the 56-view common panel at 50 epochs; Time-RCD is zero-shot.

ModelPTP_T
Anomaly Transformer3.572
GDN3.842
ModernTCN0.773
TimesNet0.881
USAD4.410
ScatterAD0.041
Time-RCD0.092

For CATCH, whose evaluation is stochastic, the same measure is 0.043 on the archived scoring pass. DCdetector is excluded for the readout issue described below.

Your DCdetector example prompted an implementation audit. Our submitted adapter averages a temporal softmax over the same axis, making its readout constant in exact arithmetic. The approximately 1.9-point mean absolute matched-AUC change comes from ranking floating-point differences, not meaningful score variation.

We will remove this implementation from behavioral comparisons and replace the USAD–DCdetector interpretation in Section 6. After exclusion, the main result is 19/48 rather than 19/54 model–dataset cells, with the maximum matched effect unchanged at 14.37 points. The model-level correlation between response strength and mean absolute detection change is 0.9349.

We will report paired sensitivity alongside rank separation and consistently distinguish low CTC_T from pointwise invariance.

AuthorsOpenReview ↗

Attribution controls and channel-group robustness

3. Quantitative attribution controls (W2/Q3)

We added independent-channel shuffling to the released-model evaluation. This preserves each channel's collection of normal-window trajectories while disrupting their cross-channel correspondence. A synchronous shuffle applies the same permutation to every channel as a control. Independent shuffling uses three fixed repetitions.

Results below use the 56-view common panel at 50 epochs, with Time-RCD evaluated zero-shot.

ModelOriginal CTC_TSynchronous shuffleIndependent shuffle
Anomaly Transformer0.35230.35230.2456
GDN0.26610.26610.2528
ModernTCN0.18370.18370.0065
TimesNet0.25470.25470.0071
USAD0.21820.21820.2520
ScatterAD0.02820.03140.0190
Time-RCD0.00320.00320.0056

Independent shuffling reduces response strength by 97.2% for TimesNet, 96.5% for ModernTCN, and 30.3% for Anomaly Transformer. GDN and USAD retain high separation, demonstrating that the effect of disrupting channel correspondence differs across models.

We complemented this test with four-state scoring of xx, TGxT_Gx, TGˉxT_{\bar G}x, and −x-x. We compare

\delta_2=s(-x)-s(T_{\bar G}x).$$ Opposite signs cannot occur for $s(x)=h(f(x_G)+g(x_{\bar G}))$ with strictly increasing $h$. Using numerical screening and requiring both effects to exceed 1% of the normal-score standard deviation, strict reversals occur in **84.74% of normal windows for Anomaly Transformer, 66.57% for GDN, 57.43% for ModernTCN, and 46.46% for USAD**. Thus, the models retaining high separation after shuffling can still exhibit cross-group interaction. For context-dependent models, including TimesNet, ScatterAD, and Time-RCD, we retain and identify the original scoring context rather than claim that all other windows were held fixed. These deterministic controls cover seven models: DCdetector is excluded for the readout issue described in Part 1, and CATCH because repeated evaluation is stochastic. We also quantified a training-fitted independent-channel ECDF scorer. Its maximum view-level mean absolute odd component is **$5.43\times10^{-13}$**. We will report score differences alongside rank-based measurements, since ranking tiny floating-point differences can otherwise give misleading separation values. **4. Does the natural-detection effect hold under alternative groups? (Q3)** We completed 1,064 model/view/seed tasks, evaluating ten channel-group configurations per task. These include top-1/2/4, the canonical group, three random groups, a low-correlation group, and groups containing 25% or 50% of the channels. The trained score, preprocessing, evaluation windows, and original matched sets are held fixed. Model response rankings correlate with the canonical ranking at **0.964–1.000** for the alternatives except the low-correlation group (**0.607**). The direction of $D$, however, is more group-dependent: | Alternative group | Cells with the same sign of $D$ | Opposite signs, with both magnitudes above 1 pp | | :--- | :---: | :---: | | Top-4 | 33/42 | 2 | | Top-25% | 35/42 | 1 | | Top-50% | 27/42 | 3 | For example, AT–HAI remains strongly positive under top-4 (**+13.75 points**, versus **+14.37** for the canonical group), whereas AT–PSM changes from **−3.15 to +1.20 points**. We will include these results explicitly: response rankings are broadly stable across these groups, but the natural-detection benefit of the odd component is not independent of the chosen group. The revision will add the paired-sensitivity measure, quantitative attribution controls, and group-wise natural-detection results, and correct the DCdetector comparisons. We would appreciate your assessment of whether these additions address the metric and attribution concerns, and whether the validation-based application clarifies the practical value.

I appreciate the authors' substantial additional experiments and clarifications, which address several of my concerns.

However, my central concern regarding task-level significance remains. The connection between the audited behavior and natural anomalies is still limited, and the proposed measure does not establish whether the observed sensitivity is beneficial for anomaly detection. The group-dependent detection effects further complicate its practical interpretation.

Therefore, I maintain my original rating.

Reviewer_ayGVOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper analyzes how strongly multivariate time-series anomaly detection (TSAD) models respond to signed relations between channels (signed orientation). The authors use a transformation TGT_G that flips the sign of a channel group selected from normal training data. This transformation preserves every coordinatewise magnitude while reversing the sign of the products between flipped and unflipped channels.

  • Response measurement: For a trained anomaly score ss, the authors measure the difference between the score distributions of original and sign-flipped normal windows as an AUC (ATA_T), and define the response strength as CT=∣AT−0.5∣C_T=|A_T-0.5|.
  • Score decomposition: The score ss is decomposed into a flip-invariant part sevens_{\text{even}} and a flip-sensitive part sodds_{\text{odd}}. The effect of the flip-sensitive part on detection is then measured by the AUC difference between ss and sevens_{\text{even}} (DnatD_{\text{nat}}).
  • Relation to native performance: The authors show that models with similar native AUC can differ substantially in response strength, and formalize this through the notion of "capability certification."

Experiments use nine released TSAD models (USAD, GDN, Anomaly Transformer, TimesNet, ModernTCN, DCdetector, CATCH, ScatterAD, Time-RCD) and the HAI, MSL, PSM, SMAP, SMD, and SWaT datasets, with shared preprocessing (VAR(1) residuals and rank-Gaussian calibration) applied to all models. The results show that:

  • response strength varies widely across models and changes unevenly with training epochs;
  • removing the flip-sensitive part improves detection on some datasets and degrades it on others;
  • native AUC alone does not determine response strength.
Critical strengths

S1 (Reproducibility): The experimental procedure is transparent and systematic.

  • The analyzed subgroup and the main analyses were pre-specified before inspecting the results.
  • The information needed for reproduction is provided in detail.

S2 (Quality): The paper uses a simple and precisely controlled probe.

  • The sign flip exactly preserves coordinatewise magnitudes, enabling a controlled comparison that changes only signed relations without altering magnitudes.
  • The authors verify that the ranking of model responses remains stable across different channel groups.
Critical weaknesses

W1 (Significance): The practical meaning of the measured quantity is unclear, and the results are largely predictable.

  • The paper does not show how sign flips arise in real anomalous situations, nor how this measurement could inform practical decisions.
  • It is self-evident that flipping signs changes cross-channel products. It is equally predictable that this change matters when it occurs in channels important for detection and matters little otherwise.
  • The flip is applied to VAR(1) residuals calibrated to a rank-Gaussian scale rather than to the raw signals, which makes it even less clear what a sign flip in this space corresponds to physically.
  • The sign-flipped windows are synthetic inputs outside the training distribution, so it is natural for models not designed to be sign-invariant to respond to them. Beyond measuring the response strength, no further insight or application is offered.

W2 (Soundness): The measured response is not shown to originate from cross-channel relations.

  • A sign flip changes not only cross-channel products but also the signs of the flipped channels themselves.
  • Even if rank-Gaussian calibration makes each channel's marginal distribution approximately symmetric, the learned model function itself may respond asymmetrically to the sign of individual channels, producing score differences unrelated to cross-channel relations. Score differences alone therefore cannot establish that a model uses cross-channel signed orientation.
  • The most direct control for separating the two is a global flip of all channels, but this control is not performed.
  • The controls in Appendix B (independent-marginal scorer and relational shuffle) are described only qualitatively, without quantitative results. Moreover, they are applied to separate simple scorers rather than to the nine released models.

W3 (Soundness): The interpretation of the natural-data analysis (DnatD_{\text{nat}}) has fundamental problems.

  • Averaging with out-of-distribution inputs:
    • Computing seven(x)=[s(x)+s(TGx)]/2s_{\text{even}}(x)=[s(x)+s(T_Gx)]/2 requires the score of the synthetic input TGxT_Gx, which does not occur in real data.
    • For strongly responding models, s(TGx)s(T_Gx) is inflated on normal windows, and this value enters sevens_{\text{even}} directly.
    • DnatD_{\text{nat}} therefore reflects the effect of averaging with scores on out-of-distribution inputs rather than the effect of removing a component the model actually uses.
  • Dependence on the score scale:
    • Detection AUC and ATA_T are rank-based and remain unchanged under any strictly increasing transformation of the score.
    • In contrast, sevens_{\text{even}} is an arithmetic mean of two score values, so the ordering between windows can change under such a transformation.
    • Consequently, DnatD_{\text{nat}} and the results in Section 5.2 and Table 2 can depend on the arbitrary scale of the score. This also affects comparisons across models, whose scores are on different scales.
  • Limitations of matching:
    • Magnitude matching relies on a single summary, the mean squared window magnitude, together with a caliper.
    • Windows with the same summary can still differ substantially in their channel-wise and time-wise absolute-value patterns.
    • Magnitude-matched AUC therefore cannot be interpreted as a comparison that removes all absolute-value differences.

W4 (Quality): The conditions under which the flip-sensitive part helps or harms detection are not analyzed.

  • The paper reports that the direction of the effect varies across datasets even within the same model.
  • However, no basis is provided for judging in advance which data or anomaly characteristics make it beneficial.
  • The results likely depend strongly on how each model computes its anomaly score (reconstruction, forecasting, association discrepancy, etc.). This dependence is not analyzed, and the explanations in the Discussion remain at the level of hypotheses.
  • The meaning of the reported changes is also not sufficiently explained. For instance, the mean absolute change in native AUC in the pre-specified subgroup is reported as a few points, but there is no interpretation of how meaningful a change of this size is for actual detection.

W5 (Quality): It is unclear whether the evaluation protocol represents the actual behavior of the released models.

  • All models receive VAR(1) residuals calibrated to a rank-Gaussian scale instead of the raw signals, and the number of training epochs is fixed.
  • Native AUCs on the common panel are generally low. Moreover, increasing the training epochs substantially changes the response strength of some models, which suggests that training may not have sufficiently converged.
  • It is therefore difficult to tell whether the measured responses are intrinsic properties of the models or artifacts of the modified inputs and limited training.

W6 (Quality): The evidence from held-out selection is weak.

  • The finding that selecting by validation response strength yields higher test response strength is nearly self-evident, since the selection criterion and the evaluation metric are the same. It merely restates the claim of Section 5.3 that native AUC and response strength are not perfectly correlated.
  • In the reduced candidate pool, which excludes scores whose sign behavior is built directly into their definitions, the confidence interval of the selection gain includes zero (Appendix F). It is therefore unclear whether this result is robust to the composition of the candidate pool.

W7 (Quality): The choice of channel group is insufficiently justified.

  • The channel group consists of the channels with the largest summed absolute training correlations, with its size capped at 8.
  • However, the number of channels varies widely across datasets, and no justification is given for fixing the group size at 8 or for using the top-correlation criterion.

W8 (Clarity): The overall clarity and readability of the paper are poor.

  • The paper contains many separate analysis panels and is written largely in terms of numbers, which makes it difficult to follow what each result means.
  • The figures are not intuitive, making it hard to grasp the main claims at a glance.
Questions for authors

Q1 (W1): In what practical situations does the sign-flip response matter?

  • Can you provide examples of real anomalous situations that correspond to the sign flip? What is the research significance of this?
  • What change in the raw signals does a sign flip in the VAR-residual space correspond to?
  • How can the measured response strength be used for practical decisions such as model selection or model design?

Q2 (W2): How can you confirm that the measured response originates from cross-channel relations?

  • Could you report the response of the nine released models to a global flip of all channels, which preserves all cross-channel products and changes only individual signs?
  • Could you provide quantitative results for the controls in Appendix B (independent-marginal scorer, relational shuffle)?

Q3 (W3): How much does DnatD_{\text{nat}} depend on out-of-distribution inputs and the score scale?

  • Given that sevens_{\text{even}} includes the score of the synthetic input TGxT_Gx, what justifies interpreting DnatD_{\text{nat}} as the effect of removing a component?
  • Are the magnitude and direction of DnatD_{\text{nat}} preserved when a monotone transformation (e.g., a log) is applied to the score, or when rank-based scores are used?
  • Can you quantify the channel-wise and time-wise absolute-value differences that remain after matching?

Q4 (W4): When does the flip-sensitive part help detection, and when does it harm it?

  • Can you identify data or anomaly characteristics (e.g., whether an anomaly alters cross-channel relations) that explain the direction of the effect?
  • Could you analyze how the results vary with the way each model computes its anomaly score?

Q5 (W5): Does the evaluation protocol reflect the intrinsic behavior of each model?

  • Are the same results observed with raw-signal inputs and each model's default training settings?
  • Is there evidence that the training of each model has sufficiently converged?

Q6 (W6): Are the selection results robust to the candidate pool?

  • Is the selection gain maintained in a pool consisting only of candidates whose sign behavior is not explicitly designed, or in a larger candidate pool?

Q7 (W7): What is the rationale for the channel-group configuration?

  • Why is the group size fixed at 8 regardless of the number of channels in each dataset?
  • What is the rationale for using the top-correlation criterion?
  • How do the results change when the group size is set proportionally to the number of channels?
5 comments

Thank you for the detailed review. We have completed additional experiments addressing attribution, score scale, practical use, and protocol sensitivity. Your questions also led us to identify a DCdetector readout error, which we explain in Part 4. We divide our response into four comments because of the comment-length limit.

1. Does the response reflect cross-group interaction? (W2/Q2)

We evaluated four inputs using the same trained model: xx, TGxT_Gx, TGˉxT_{\bar G}x, and −x-x. We compare the effect of flipping GG under two orientations of its complement:

\delta_2=s(-x)-s(T_{\bar G}x).$$ For any score $s(x)=h(f(x_G)+g(x_{\bar G}))$ with strictly increasing $h$, these differences cannot have opposite signs in exact arithmetic. Resolved reversals therefore rule out this block-additive explanation. The following results use the 56-view common panel at 50 epochs; Time-RCD is zero-shot. Strict reversals require both differences to exceed the numerical screening threshold of 128 float32 ULP and 1% of the normal-score standard deviation. | Model | Partial-flip $C_T$ | Global-flip $C_T$ | Strict reversal | | :--- | :---: | :---: | :---: | | Anomaly Transformer | 0.3523 | 0.1542 | 84.74% | | GDN | 0.2661 | 0.2448 | 66.57% | | ModernTCN | 0.1837 | 0.0321 | 57.43% | | TimesNet | 0.2547 | 0.0477 | 64.60% | | USAD | 0.2182 | 0.2074 | 46.46% | | ScatterAD | 0.0282 | 0.0182 | 62.69% | | Time-RCD | 0.0032 | 0.0237 | 30.06% | We also independently shuffled normal-window trajectories across channels, preserving each channel's collection of trajectories while disrupting their correspondence. A synchronous shuffle uses the same permutation across channels as a control. Independent shuffling reduces response strength by **97.2% for TimesNet, 96.5% for ModernTCN, and 30.3% for Anomaly Transformer**, while synchronous shuffling approximately preserves their responses. GDN and USAD retain high separation under independent shuffling—0.2528 and 0.2520—despite their substantial four-state reversal rates. Thus, dependence of rank separation on channel correspondence and interaction in the score function are distinct properties. For TimesNet, ScatterAD, and Time-RCD, scoring depends on batch or other input context. Their four-state results retain that original context-dependent pipeline; we will not describe them as interventions that hold all other windows fixed. CATCH is excluded from these deterministic controls because repeated scoring is stochastic. DCdetector is excluded for the readout issue detailed in Part 4. An additional training-fitted independent-channel ECDF scorer has a maximum view-level mean absolute odd component of $5.43\times10^{-13}$. We will add these quantitative controls and revise the attribution language to distinguish partial-flip response from the interaction evidence established by the controls.

2. Score scale, transformed inputs, and magnitude matching (W3/Q3)

We agree that averaging scores does not generally commute with a nonlinear monotone transformation. We tested two nonlinear signed power transformations,

\phi_2(s)=s|s|,$$ applying each to both $s(x)$ and $s(T_Gx)$ **before** reconstructing the even score and computing AUC. At 50 epochs, neither transformation changes the sign of the magnitude-matched effect in any of the **48 model–dataset cells** after excluding DCdetector. The maximum changes in $D_{\mathrm{matched}}$ are 2.06 and 1.72 percentage points, respectively. Across both training budgets, neither transformation produces opposite signs with effect magnitudes exceeding one percentage point on both scales. For native AUC at 50 epochs, the signed-square transformation changes the sign in one of 48 cells. We will report these results as empirical robustness to the tested transformations, not invariance to arbitrary monotone transformations. Regarding transformed inputs, the even–odd decomposition is exact for the specified score and transformation, but $s_{\mathrm{even}}$ does use $s(T_Gx)$. We will state its interpretation directly: **$D$ measures the change in natural-label AUC when the trained score is symmetrized.** It does not identify the effect of removing a unique physical mechanism. Magnitude matching controls the scalar $m(w)=L^{-1}\sum_t\|w_t\|_2^2$, rather than the complete absolute-value pattern. We quantified the remaining differences between matched attack and normal sets using feature-wise Wasserstein distances. Across the six datasets, averaged distances range from **0.023–0.214** for channel-wise mean absolute values and **0.0141–0.0594** for time-bin mean absolute values, in calibrated input units. We will add these diagnostics alongside the matching definition.

3. Natural relevance and practical decisions (W1/W4, Q1/Q4)

The intervention acts on the calibrated residual representation. Let rtr_t denote the one-step VAR residual and zt=F(rt)z_t=F(r_t) its channel-wise, training-fitted rank-Gaussian representation. We negate selected coordinates of ztz_t, preserving their magnitudes in that space. This is not an equal-magnitude reversal of raw prediction errors or a physical inversion of sensor readings. We will clarify this distinction.

To examine natural relevance, we measured cross-group signed relations relative to a training-normal reference on the original matched windows. This relation score has matched AUC 0.7102 on PSM and 0.3941 on HAI. Circularly shifting the complementary channels gives median AUCs of 0.4893 and 0.5847, respectively. These results show label-associated relation differences that depend on temporal alignment; they do not establish that natural anomalies reproduce the exact sign-flip intervention.

Within the 11 HAI views where the relation statistic is defined, its correlation with signed DmatchedD_{\mathrm{matched}} differs by model: Spearman ρ=−0.845\rho=-0.845 for Anomaly Transformer and +0.736+0.736 for TimesNet. We will report these as descriptive associations that help characterize the observed heterogeneity.

The effects also span different scoring approaches. At 50 epochs, Anomaly Transformer's association-discrepancy score has CT=0.3523C_T=0.3523 and a mean absolute matched-AUC change of 5.84 points. Reconstruction- and prediction-based scores from GDN, TimesNet, USAD, and ModernTCN have CT=0.1837C_T=0.1837–0.26610.2661 and mean absolute changes of 2.41–3.06 points. CATCH, ScatterAD, and zero-shot Time-RCD have lower response strengths and mean absolute changes of 0.34–0.54 points. We will add this comparison without attributing the differences to scoring architecture alone.

For a concrete practical use, we tested selecting between ss and sevens_{\mathrm{even}} using validation labels. For Anomaly Transformer, this improves held-out native AUC by 1.11 percentage points (95% CI [0.24, 2.17]) and matched AUC by 0.95 points ([0.05, 2.04]), relative to always using ss. A broader model-then-variant selection analysis shows little additional gain, so we present this as a model-specific benefit.

Without anomaly labels, CTC_T reveals a behavioral difference rather than whether that difference is beneficial. With validation labels, the full/even comparison can support score selection. We will make these uses explicit.

4. Preprocessing and training budget (W5/Q5)

We completed 448 evaluations on standardized raw signals, retaining the original channel groups, evaluation indices, and training budgets. The model response ranking correlates with the canonical pipeline at Spearman ρ=0.7143\rho=0.7143, although absolute responses change substantially. The input audit also identifies cases where selected channels are constant and their representation determines whether a sign flip changes the input.

We will therefore describe the empirical results as properties of trained scores under specified pipelines, rather than preprocessing-independent properties of model architectures.

The existing 5/50-epoch comparison measures training-budget dependence. An additional 15-view, three-seed analysis includes 20 epochs: ModernTCN's response strength is 0.0185, 0.1120, and 0.1337 at 5, 20, and 50 epochs. This shows slower growth after 20 epochs in that panel, but does not establish convergence for every model. We have not completed a separate full-panel evaluation using each model's official default training configuration.

Reviewer_ff8hOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes a testing protocol for time series anomaly detection (TSAD) detectors. It works as follows. First, signs are flipped for a subset of input channels. This flip preserves the magnitude (squared values) of each channel, but reverses the sign pairs containing the flipped channels. The paper next proposes a metric that captures how much trained score separates original from sign-flipped windows. Furthermore, the paper uses even/odd decomposition to identify the flip sensitive component. The paper studies nine TSAD models on six datasets. The key insight is that TSAD detectors can post identical AUC while responding to different features of an input.

Critical strengths
  • The paper studies a question that is of importance to those working in time series anomaly detection: two models that have the same AUC, do these models see the same thing? The paper concludes that more often than not that is not the case.
  • The paper presents a method that can allow a practitioner study if a TSAD model is sign-blind (if A_T = 0.5, the detector ranks the original and the flipped windows the same). Sometimes it is fine, where as at other times it is not fine.
    • If the real anomalies in a data depend only on magnitude then a sign-blind TSAD detector doesn't loose anything.
    • However, if the real anomalies are captured within relationship between channels then a sign-blind TSAD detector will be fail to perform well, no matter the size of the TSAD detector model.
  • The paper proposes a simple, easy to understand and to implement method for studying pre-trained TSAD detectors.

Many TSAD models claim that they model cross-channel structure. However, if these models are sign-blind then their claim is inaccurate. This suggests that the work presented here have broad implications for the field. Going forth new methods in TSAD should not only report AUC, but also report C_T as computed in this paper.

Critical weaknesses
  • The paper only studies sign flip. There are many other ways to probe a detector, scaling, time-reversal, etc., that is ignored in this work.
  • What happens when the input data is preprocessed? Specifically, the paper's whole claim is that detectors respond to signed orientation. But these detectors are responding to has already been linearly decorrelated and rank-normalized. Different preprocessing could easily shift the C_T numbers. The paper never runs the probe on raw or differently-preprocessed data. Then how do we tell how much of the finding is about the models and how much is about the pipeline.
Questions for authors
  1. For practitioners, what should the recommended default be? Should they report (A_T, C_T, native AUC) tuples for every TSAD benchmark submission? If so, what is the compute overhead for a typical benchmark run?
1 comment

Thank you for recognizing the value of testing whether similar native AUC conceals different model behavior. Your questions helped us clarify the scope of the probe and its use in practice.

1. Why focus on sign flips?

We chose the partial sign flip because it preserves every coordinatewise magnitude while reversing signed products across the selected channel partition. It therefore tests a signed distinction without changing magnitudes. It is also an involution, giving an exact even–odd decomposition of the same trained score without retraining.

Scaling and time reversal are useful complementary probes, but they test different properties. Scaling changes magnitudes, while time reversal changes temporal order. We will explain this choice more directly and clarify that the paper studies one controlled behavioral distinction rather than proposing a comprehensive audit of all time-series behavior.

2. How much depends on preprocessing?

We completed 448 evaluations on standardized raw signals, retaining the original channel groups, evaluation indices, and training budgets. Across the eight models excluding the DCdetector readout discussed below, the response ranking correlates with the canonical VAR-plus-rank-Gaussian pipeline at Spearman ρ=0.7143\rho=0.7143.

Absolute responses do change substantially. For example, in this seed-0 comparison, Anomaly Transformer's CTC_T changes from 0.3554 under the canonical pipeline to 0.0592 on standardized raw signals, and TimesNet's changes from 0.2536 to 0.0963. Thus, the numerical response is a property of the trained model together with its input pipeline; it should not be treated as an architecture-only constant.

We will add this comparison and recommend reporting the preprocessing and training budget alongside the audit results. The experiment addresses your concern directly: model differences remain observable on raw-signal inputs, but their magnitude and ordering are affected by preprocessing.

3. What should practitioners report, and what does it cost?

For a benchmark using this audit, we recommend reporting native AUC, directed response ATA_T, and response strength CTC_T, together with the channel-group rule and preprocessing. We also recommend a paired sensitivity measure such as

E∣sodd(X)∣SD⁡[s(X)],\frac{\mathbb{E}|s_{\mathrm{odd}}(X)|}{\operatorname{SD}[s(X)]},

computed on normal windows. This distinguishes distributional rank separation from the size of within-window score changes.

In particular, AT=0.5A_T=0.5 does not imply that every original–flipped pair receives the same score. We will make this distinction clearer and avoid describing low CTC_T alone as pointwise sign invariance. Likewise, this probe tests a particular signed distinction, not all possible forms of cross-channel dependence.

The basic audit requires no retraining. If original normal-window scores are already available, it requires one additional scoring pass on their flipped counterparts. The paired measure uses these same scores. Comparing full and even scores on labeled natural data requires flipped scores for those evaluation windows as well.

For the same set of windows, computing both original and flipped scores therefore uses approximately twice the scoring work of the original-score evaluation alone, not twice the total training-and-evaluation cost. The actual wall-clock overhead depends on the model and evaluation-set size. The additional four-state attribution diagnostic uses four scoring arms in total and is separate from the basic audit.

4. Corrections and presentation

Our follow-up audit found that the submitted DCdetector readout averages a temporal softmax over the same axis, producing a constant in exact arithmetic. Its apparent detection changes arise from floating-point rank differences. We will remove this implementation from behavioral comparisons and correct the affected discussion and figures.

The main 50-epoch result becomes 19/48 rather than 19/54 model–dataset cells, with the maximum matched-AUC effect unchanged at 14.37 percentage points.

We will also simplify the presentation around the probe, its measured response, and its effect on natural detection, and add a short practical reporting guide covering the quantities and costs above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.