acceptodds
Under review as a conference paper at ICLR 2027

OneShot: Label-Efficient Detector Weighting for Time Series Anomaly Detection

Abstract

Time series (TS) anomaly detection (AD) lacks a universal detector, with the best choice determined by the series rather than by the algorithm. Existing methods therefore either select one detector for each series or average a pool of them. However, both assume that no label ever arrives, while a deployed monitor does receive them, since an operator confirms the alarms it raises. In this paper, we propose ONESHOT, which uses a single confirmed anomaly to weight the pool rather than to choose one detector from it. Specifically, we score each detector on the confirmed anomaly, and those scores become the weights of a soft mixture over the whole pool. We show that a single confirmed anomaly is enough to come close to the best detector for the series. Across 152 univariate and 73 multivariate series, ONESHOT outperforms the strongest single detector, averaging the whole pool, and even picking the best detector on the confirmed event. Code is available at: https://anonymous.4open.science/r/one_shot-C344.

Primary area:
learning on time series and dynamical systems
Keywords:
time series anomaly detection, model selection, detector ensemble, label efficiency, one-shot supervision, softmax weighting, unsupervised model selection

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_9PwcOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes OneShot, a weighted ensemble approach for time-series anomaly detection. Given one confirmed anomaly event, OneShot evaluates each detector on that event together with sampled, presumed-normal points, then converts these performance estimates into softmax ensemble weights.

Critical strengths

The paper studies a practically motivated setting in which a deployed anomaly detection system receives occasional operator feedback.

Critical weaknesses
  • Generalization across anomaly types remains unclear. The method assumes that detector performance on one confirmed event is informative about other events in the time series. A more fine-grained analysis should distinguish hidden anomalies that resemble the confirmed event from those with different characteristics. Transfer across related time series would also help establish the usefulness of a confirmed event beyond the series on which it was observed.
  • The concern regarding practicability of the proposed approach. For example, OneShot is not applicable on almost half of the benchmark datasets which contains only one labeled anomaly.
Questions for authors
  • Does baseline ensemble share the same type of normalization? Does score normalization use the entire time series, including future/hidden points?
  • Could you reconcile the exact-label results in Tables 2 and 9? “Best on labeled window” obtains 0.672 VUS-PR in Table 2 but 0.568 in Table 9, while OneShot obtains 0.710 and 0.699, respectively. Table 9 should also include ensemble baselines so that its robustness gains can be attributed specifically to the proposed weighting strategy.
  • Could you explain why confirming the earliest events sometimes performs worse than selecting events uniformly at random, given that the former better reflects practical deployment?
4 comments

Please let me know if I can provide any further clarification!

If my responses have sufficiently addressed your concerns, I would be grateful if you would consider revisiting your score.

I thank the authors for the detailed response! The proposed approach relies on the transferability of a confirmed anomaly event to hidden events. Therefore, further analysis of transferability is needed for Table R2. Also, why do the numbers in Table R5 differ from one another?

We sincerely thank the reviewer for engaging in the discussion and for the thoughtful follow-up questions!

Both questions helped us strengthen the paper, and we have posted a second updated PDF (changes in brown).

 

Changes from the first updated PDF:

  • [Appendix P.5] New analysis of when a confirmation transfers, with Table 41 (Q1)
  • [Table 9] New column "Correct" (share of the confirmed window that is truly anomalous) and grouped rows (Q2).
    • Thanks to the reviewer's careful reading, the "Shifted by half" row now matches Table R5 (Top-3 average 0.665, OneShot 0.662)
  • [Section 5.4] The discussion of Table 9 follows the new column

 


Q1: When does a confirmation transfer to a related series?

We thank the reviewer for this insightful suggestion, and we agree that the transfer relies on the confirmed event being informative for the target. Since OneShot transfers which detectors to trust, the natural measure is whether the two series rank the detectors alike. For the trials of Table R2, we computed the Spearman correlation rr between the per-detector accuracies of the source and the target series, and split the trials into three equal groups by rr:

Table R8 (Appendix P.5, Table 41). Transfer by detector agreement

  • Trials of Table R2, univariate, VUS-PR on the whole target series
  • Agreement: Spearman rr between source and target detector accuracies
AgreementMean rrScore averageBest on windowOneShot
High0.820.6660.7170.788
Mid0.530.5740.5790.677
Low-0.010.5290.3450.468
  • The gain grows with the agreement: +18% over the score average in both the high and mid groups (p<10−12p < 10^{-12}), and the per-trial gain correlates with rr (Spearman 0.40). This confirms that the transfer works through the mechanism of OneShot, i.e., the confirmation identifies the detectors that also suit the target.
  • Series of the same dataset mostly agree: r>0r > 0 in 84% of the trials (median 0.54), which is why the transfer helps on average in Table R2.
  • When the two series rank the detectors independently (r≈0r \approx 0), the confirmation carries little information about the target, and OneShot stays well above the window winner (0.468 vs. 0.345, p<10−9p < 10^{-9}), since its soft weight does not bet on a single detector.
  • The multivariate collection shows the same trend (high group: 0.409 vs. 0.282 for the score average).

 

In deployment, this suggests confirming the event on a series of the same source (e.g., the same machine or sensor type), which is how related series arise in practice.

 


Q2: Why do the numbers in Table R5 differ across rows?

We thank the reviewer for the careful check.

The rows differ in which series can be scored, not in the hidden part:

  • The score average uses no label, so it changes only with the set of series. On the 146 series of the Exact, Shifted, and Normal rows, it is identical (0.603). The shorter windows of the Half and Quarter rows cannot be scored on some series, which leaves subsets of 133 and 114 series.
  • Within each row, every method uses the same series, confirmed window, and hidden part, so each row is a paired comparison.
  • The Exact row (0.699) differs from Table 2 (0.710) because the corruption study draws its own trials (confirmed events and normal points).

 

To make the rows easier to read, we added the column "Correct" to Table 9:

Table R5 (revised Table 9). Imperfect labels

  • Series and pool of Table 2, VUS-PR
Confirmed eventCorrectSeriesScore averageTop-3 averageBest on windowOneShot
Window within the event
Exact100%1460.6030.6840.6210.699
Half of it100%1330.6140.6840.6090.696
Quarter of it100%1140.6490.6790.6110.698
Window (partly) off the event
Shifted by half53%1460.6030.6650.5870.662
Shifted by its length7%1460.6030.4990.4200.503
Normal window0%1460.6030.3370.2370.339
  • When the confirmation is CORRECT, OneShot is the best of all methods, including the Top-3 average (p≤0.0004p \le 0.0004), even when only a quarter of the event is confirmed.
  • When the window is (partly) OFF the event, the drop of OneShot is itself evidence that it truly uses the confirmation: its accuracy follows the share of correct points (0.699, 0.662, 0.503, 0.339 at 100%, 53%, 7%, 0%), which shows that its gain comes from the confirmed event rather than from averaging the pool. Even so, it stays tied with the best ensemble at 53% (p=0.25p = 0.25), leads it on both larger pools (Table R6), and stays above the window winner in every row.

 

We hope that these results address the remaining questions and can be reflected in the final assessment!

 

If any concern remains unresolved, please let us know and we will gladly address it.

Dear Reviewer 9Pwc,

As the discussion period is coming to a close in a few hours, we would like to thank you again for your time and consideration. We would be happy to address any remaining questions or comments before the discussion closes.

Best regards,

The Authors

Reviewer_RxohOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces OneShot, a simple label-efficient method for combining a fixed pool of anomaly detectors using a single confirmed anomaly event. Instead of choosing one detector or averaging all detectors, OneShot scores every detector on the confirmed event, converts these accuracies into softmax weights, and produces a weighted combination of score vectors.

Critical strengths

None.

Critical weaknesses
  1. The core idea, softmax over accuracies on a confirmed event, is too simple.
  2. The most relevant work to this paper should be model selection, but this paper does not even discuss or compare such works at all.
  3. This method requires running through all the detectors in the detector pool. This is acceptable with some reservations for simple statistical methods, but when the models are all deep models or foundation models, the running cost becomes a bit too high.
Questions for authors
  1. How does OneShot perform when comapred to other model selection methods?
  2. How does OneShot perform when the pool contains mostly weak detectors? Does it gracefully fall back to the score average, or does it overfit to the confirmed window?
5 comments

Please let me know if I can provide any further clarification!

If my responses have sufficiently addressed your concerns, I would be grateful if you would consider revisiting your score.

Thank you for your rebuttal. A further comparison with the model selection methods is appealing, but I still haven't received more information that highlights your method. What I would prefer to see is that you can elaborate in the abstract or introduction section exactly where the innovation of your method lies compared to previous model selection methods. As you stated in the abstract, "Existing methods either select one detector for each series or average a pool of them. However, both assume that no label ever arrives, while a deployed monitor does receive them, since an operator confirms the alarms it raises." I believe that "Existing methods" here does not fully encompass the existing more extensive works, at least model selection and few-shot methods are not included. In my opinion, the motivation might need to be reorganized, and the current structure does not fully reflect where the uniqueness of your method lies.

We sincerely thank the reviewer for engaging in the discussion!

Following the reviewer's suggestion, we have posted a fourth updated PDF (changes in brown), and we kindly ask the reviewer to download it.

 

Changes from the third updated PDF:

  • [Abstract] Names model selection and few-shot detectors, and states how OneShot differs from both
  • [Section 1] The second paragraph positions OneShot against model selection and few-shot detectors, with a pointer to Table 1
  • [Section 2] The paragraph on semi-supervised AD is now titled "Semi-supervised and few-shot AD"

 


1. Revised Text (Abstract and Introduction)

We agree that the abstract and the introduction should name these lines of work explicitly.

The revised text reads as follows (changes in bold):

[Abstract]

  • Before: "Existing methods therefore either select one detector for each series or average a pool of them. However, both assume that no label ever arrives, while a deployed monitor does receive them, since an operator confirms the alarms it raises. In this paper, we propose OneShot, which uses a single confirmed anomaly to weight the pool rather than to choose one detector from it."
  • After: "Existing methods therefore either select one detector per series or average a pool of them, and few-shot detectors train one with labels. However, a deployed monitor receives labels as an operator confirms its alarms, while its detectors stay fixed. In this paper, we propose OneShot, which uses a single confirmed anomaly to weight a fixed pool rather than to choose or train one detector."

[Section 1, second paragraph]

  • Before: "However, existing solutions assume that no labels ever arrive. That leaves them ranking detectors by the scores or by results on other datasets."
  • After: "However, model selection ranks detectors without labels and returns one, and few-shot detectors spend labels on training one. None turns a confirmed event into a weight over a fixed pool (Table 1), and OneShot fills this gap."

[Section 2]

  • Before: "Semi-supervised AD."
  • After: "Semi-supervised and few-shot AD."

The introduction now also points to Table 1, which places every family (model selection, ensembles, and detectors trained with labels) against OneShot.

 


2. Where the novelty lies

OneShot differs from prior work in

  1. What it reads,
  2. What it returns, and
  3. How it uses the label

 

  1. vs. Model selection (What it reads):
  • Existing model selection decides without any label of the series, from an internal measure of the scores or from evaluations on other datasets. OneShot instead reads the label that a deployed monitor already receives, i.e., the operator's confirmation, which needs no corpus at all. This is why it outperforms every selection family that decides without it (0.497 to 0.572 vs. 0.710, Table R1).
  1. vs. Selection with that label (What it returns):
  • Even when the confirmed event is used, picking its winner bets on one detector. OneShot weights the whole pool instead, which is why it also beats the window winner (0.710 vs. 0.672, Table R1).
  1. vs. Few-shot and semi-supervised detectors (How it uses the label):
  • These spend the labels on training a detector, which needs training data and a training loop. OneShot keeps every detector frozen and only re-weights their scores, which adds 45 ms (Table 10) and fits a running pipeline as it is.

 

In short, OneShot is the only family in Table 1 that is frozen, needs no corpus, uses a label of the series, and returns a mixture over the whole pool.

 


We hope that the revised positioning makes the contribution clearer.

If the reviewer's concerns have been addressed, we would sincerely appreciate it if the reviewer could kindly reconsider the evaluation and the score of our paper. If any concern remains unresolved, please let us know and we will gladly address it.

Reviewer_PKh3OpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors highlight that no best Time Series Anomaly Dectection (TSAD) method exist in general as the performance of each detector is highly task-dependent. Their answer to this problem is detector-ensembling with weights relying on the performances of the method on a confirmed anomaly, mimicking real-life scenarios where an operator confirms an alarm raised by a detector. Their method is named OneShot and ensembles the weights with a softmax-based strategy. An extension named k-Shot extends the method to k confirmed anomalies.

They rigorously evaluate their ensembling method on a fraction of TSB-AD-U (152 series) and TSB-AD-M (73 series) with 4 threshold-independant metrics. In particular they show that the OneShot method effectively beats naive usupervised average score ensembling and other ensembling strategies under one confirmed events. Furthermore they attest the robustness of their method by testing OneShot under label scarcity scenarios, different pools and imperfect labels. An additional informative ablation study is rigorously lead.

Critical strengths

The main identified strength of the paper is the rigorous experimental study combined with a proper ablation study of the method. In particular, it highlights that limited supervision combined with classical machine learning methods (softmax) can be effective.

Critical weaknesses

If my understanding of the method is correct, when the operator's confirmation is a false positive (the two last raw of Table 9 covers 0% of the event) unsupervised average score ensembling outperforms OneShot (0.554 or 0.468 VUS-PR for OneShot in Table9 versus 0.596 VUS-PR for average ensembling scores in Table3). This observation balance the claim of efficiency in a real-life scenario as not choosing could be more robust in this case.

Questions for authors

Do you believe the false positive scenario I mentioned in the weaknesses section is of high importance in the deployment of OneShot in practice ? In this setting, could k-Shot adress this issue ? Convincing me this issue can be tackled either by underlining performances of OneShot I might have missed/missunderstood or by highlighting k-Shot evaluation results that adress this point would raise my score.

5 comments

Please let me know if I can provide any further clarification!

If my responses have sufficiently addressed your concerns, I would be grateful if you would consider revisiting your score.

I thank the authors for the detailed rebuttal and the additional experiments.

My last concern is about how false confirmations are simulated. In practice, an alarm is raised by the monitor on a window where some detectors score high. If the corruption is made by choosing a random false positive, this would be to the best of my understanding the easiest false positive to detect.

Could the authors detail how they simulated the false positive windows ? Could the authors evaluate OneShot, the fallback and k-Shot on “hard” false positives ? The highest-scoring non-anomalous window of the window winner would be a candidate.

We sincerely thank the reviewer for engaging in the discussion.

We have posted a third updated PDF (changes in brown), and we kindly ask the reviewer to download it.

Its only addition to the second updated PDF is Appendix P.10 (Hard False Confirmations), which holds the new results below.

 


How the false confirmations were simulated

In Table 9 (Normal window) and in the kk-Shot study of Appendix P.3, the false confirmation is a window of the event's length (for kk-Shot, the median length of the true events), placed uniformly at random among the positions where the window and one window length on either side contain no anomaly. We agree with the reviewer that such a window is not where a monitor raises its alarms.

 


Hard false positives (new)

Following the suggestion, we replaced the random window with the highest-scoring anomaly-free window (same length and the same exclusion rule) of two monitors:

  • (A) The window winner (the reviewer's candidate): the detector that is best on the true event
  • (B) The score average: a natural monitor when no confirmation is available yet

 

Everything else follows Table 9 and Appendix P.3, with the same true events. For k=1k = 1, we keep the hidden parts of Table 9 on the 145 series where every window can be placed, and draw the normal points of each confirmed window anew, so the values differ slightly from Table 9.

Table R5 (Appendix P.10). One false confirmation (k=1k = 1)

  • VUS-PR, univariate, 145 series
False confirmationScore averageBest on windowOneShotOneShot + fallback
None (exact label)0.5970.6160.6990.697
Random normal window0.5970.2420.3540.530
Hard (A): window winner0.5970.4170.4920.574
Hard (B): score average0.5970.2720.3520.468
  • Hard false positives do NOT hurt OneShot more than random ones: Under (A), OneShot reaches 0.492, clearly above 0.354 under a random window (p<10−10p < 10^{-10}), and under (B) it reaches 0.352, with no significant difference (p=0.33p = 0.33). The random window of our study is therefore not an optimistic choice.
  • OneShot outperforms the window winner under every false confirmation (p<10−10p < 10^{-10}), since its soft weight does not bet everything on the detector that fits the false window.
  • The fallback raises OneShot under both hard windows (0.492 to 0.574 and 0.352 to 0.468).

 

Table R6 (Appendix P.10). kk-Shot with one hard false confirmation

  • VUS-PR, k−1k-1 true events and one false one, same trials as Appendix P.3
kkFalse confirmationScore averageBest on windowkk-Shotkk-Shot w/o the false one
2Random0.6010.5620.6480.695
2Hard (A)0.6120.5980.6570.705
2Hard (B)0.6240.5470.6220.703
3Random0.5930.6200.6870.700
3Hard (A)0.6030.6400.6940.710
3Hard (B)0.6100.6020.6760.708

A second confirmation dilutes a hard false positive as well: kk-Shot outperforms the window winner in every row (p<10−9p < 10^{-9}), and with two true events next to the false one (k=3k = 3), it keeps 95-98% of its accuracy without the false one and leads all three rows (p<10−5p < 10^{-5} vs. the score average).

Since some detector separates a hard window by construction, the fallback triggers less often there. In deployment, however, an operator who confirms several alarms over time is exactly the kk-Shot setting, which is why we regard kk-Shot as the main remedy against any false confirmation, random or hard.

 

If any concern remains unresolved, please let us know and we will gladly address it.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Comments

Summary of Our Response

Note: An updated PDF has been posted.

Text that we added or changed is colored brown for easy reference, and we encourage the reviewers to consult this version.

 

We thank all reviewers for their time and helpful comments.

We appreciate that the reviewers recognized "the rigorous experimental study combined with a proper ablation study" and that "limited supervision combined with classical machine learning methods (softmax) can be effective" (PKh3), and "a practically motivated setting in which a deployed anomaly detection system receives occasional operator feedback" (9Pwc).

 


Main points raised and what we added

[A] Comparison with model selection (Rxoh)

It is important to note that we ALREADY

  • (1) discuss model selection in Section 2,
  • (2) place each family against OneShot in Table 1,
  • (3) and explain in Appendix H why corpus-based selectors cannot run in the one-shot setting.

 

Nonetheless, we ADDITIONALLY compared against them directly:

  • OneShot outperforms every model selection family by up to +43% (p<0.001p < 0.001), including the ones that use a corpus of other series (Appendix P.1).
  • On the larger pools with deep and window-length variants, OneShot is the most accurate in all 24 combinations of collection, pool, and measure (Appendix P.8).

 


[B] Imperfect and false confirmations (PKh3, 9Pwc)

Table 9 now uses the series and pool of Tables 2 and 3 and includes the ensemble baselines. OneShot outperforms the window winner in every row and leads every ensemble whenever the confirmed window lies inside the event, and realistic errors (a late, partial or misaligned confirmation) keep over 94% of its exact-label accuracy.

For a false confirmation, we added two remedies (Appendix P.3, Section 5.4):

  • (1) kk-Shot dilutes a false confirmation: One further true confirmation is enough to outperform the score average, and with two it keeps 98% of its error-free accuracy
  • (2) A fallback to the score average when no detector fits the confirmed window recovers most of the loss, at a cost of only 0.002 on exact labels

Both remedies also hold on the larger pools with deep detectors (Appendix P.9).

 


[C] Generalization of a confirmed event (9Pwc)

  • (1) OneShot leads on hidden events that do not resemble the confirmed one, with its largest margin over the window winner on events of a different length (+7.8%, Appendix P.4)
  • (2) A confirmation transfers across related series (+13.9% over the score average on single-event series), which makes OneShot applicable to series with a single anomaly as well (Appendix P.5)

Both results also hold on the larger pools with deep detectors (Appendix P.9).

 


[D] Weak pools and simplicity (Rxoh)

OneShot trains NOTHING, tunes NOTHING per series, and needs NO corpus.

Its softmax is the unique entropy-regularized trade-off between trusting the window winner and averaging the pool, and it is stable to small errors in the window accuracies (Propositions 1, 2 in Appendix N, Table 11).

 

Nonetheless, we additionally built three weak pools, and OneShot leads in every one of them (p<0.001p < 0.001). Adding twice as many pure-noise detectors costs OneShot only 1%, versus 11% for the score average (Appendix P.2).

 


[E] Protocol and cost (9Pwc, Rxoh)

Every method ALREADY reads the same standardized scores (Appendix K).

 

Nonetheless, we additionally standardized each detector from the past only, which changes OneShot by 0.001 (Appendix P.6), and the order of confirmed events has no significant effect at any kk (Appendix P.7). OneShot itself adds 45 ms (Table 10) and reaches 0.712 on the statistical pool alone, without deep detectors (Table 4). The pool itself is shared by every ensemble and selector that reads detector outputs.

 


If any concern remains unresolved, please let us know and we will gladly address it.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.