acceptodds
Under review as a conference paper at ICLR 2027

Coarse Temporal Feedback for Test-Time Adaptation in Streaming Time-Series Anomaly Detection

Abstract

Test-time adaptation (TTA) can help time-series anomaly detectors cope with evolving normal behavior during deployment. Yet label-free TTA must decide which test observations are safe for adaptation using model-derived signals that may themselves become unreliable under distribution shift. External feedback offers an additional source of adaptation evidence, but in deployment it may be available only periodically and at a coarse temporal granularity. We therefore study a setting in which each completed interval is summarized by one binary response that can influence only future predictions: a negative response certifies the interval as normal, whereas a positive response reveals only that an anomaly occurred somewhere within it. We introduce Feedback-Interval Test-Time Adaptation (FITTA), a minimal adaptation method for this strictly chronological setting. FITTA reprocesses intervals confirmed to be normal for adaptation, penalizes reductions in the interval maximum anomaly score relative to the frozen source detector after positive feedback, and uses the latest response to set the resolution of the next feedback interval. With interval feedback simulated from benchmark labels and the seven benchmark families weighted equally, FITTA raises AUPRC from 0.327 to 0.551 and VUS-PR from 0.248 to 0.506 relative to the frozen detector. These results show that coarse existential feedback can provide substantial adaptation utility for subsequent detection without anomaly localization. A matched factorial analysis further shows that most of this utility comes from adaptation on intervals certified as normal, while positive-evidence preservation and adaptive resolution provide a smaller, non-additive refinement rather than independent additive gains.

Primary area:
learning on time series and dynamical systems
Keywords:
time series anomaly detection, test-time adaptation, streaming anomaly detection, coarse temporal feedback

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_2FYbOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies test-time adaptation (TTA) for streaming time-series anomaly detection under a deliberately coarse feedback protocol, after a contiguous interval of detection units has been fully scored, an external provider returns one binary bit. A negative bit certifies every timestamp in the interval as normal; a positive bit asserts only that at least one anomaly occurred somewhere in the interval, with no localization. Committed scores are immutable, and feedback can only affect future detection units.

Critical strengths
  1. The related works have been conducted in a thorough manner.
  2. The details of the experiment are also quite clear.
Critical weaknesses
  1. The method itself is a bit too simplistic and only takes up less than one page.
  2. M2N2 and CANDI receive no external feedback, and BALAD uses point-specific endpoint feedback under a different protocol. The authors acknowledge this ("Comparisons are therefore not uniformly matched"), but the main table still juxtaposes these methods as if FITTA's gains are comparable. A fairer comparison is needed.
  3. The visualization in this paper is seriously lacking.
  4. Continuous feedback throughout every period of time is also unrealistic. Could there be some moments when no feedback is needed? For instance, a long-term stable state.
Questions for authors

Please see Weaknesses.

2 comments

We thank the reviewer for the constructive comments and have revised the manuscript accordingly.

W1. Method simplicity. Our central question is whether coarse existential feedback can support future-only adaptation without revealing anomaly locations. FITTA is a minimal implementation of this information constraint, with contributions in the chronological feedback setting and its empirical characterization (Secs. 3–5).

W2. Baseline fairness. We agree that cross-protocol results should not imply equal-supervision algorithmic superiority. Revised Table 1 focuses on Frozen versus FITTA, while M2N2, CANDI, and BALAD are retained as contextual comparisons in Appendix B (Table 7). Controlled evidence is provided by the component ablations (Table 2), localization oracles (Table 4), and matched-response scheduling comparisons (Table 5).

W3. Visualization. We added a seven-family TPR/FPR analysis (Figure 2, Sec. 5.3) and a temporal case study (Figure 3, Appendix G).

W4. Continuous feedback. Our protocol requests one response per completed interval, not continuous pointwise labels. FITTA adjusts the next interval's resolution; deciding when to skip feedback is a separate acquisition problem. We clarify this distinction in Sec. 6.

The response did not address my concerns, and I notice that other reviewers have also raised many difficult questions that couldn't be easily resolved. Therefore, I will keep my rating.

Reviewer_dWk7OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies streaming time series anomaly detection with coarse interval feedback. FITTA adapts on intervals confirmed as normal, preserves source-relative maximum anomaly scores after positive feedback, and adjusts subsequent interval lengths based on the latest response.

Critical strengths
  1. The problem setting is clearly defined and follows a strict score-before-feedback protocol.
  2. The method reasonably exploits the asymmetric information provided by positive and negative feedback.
  3. Ablations identify adaptation on confirmed-normal intervals as the main source of improvement.
Critical weaknesses
  1. Conventional adaptation on confirmed-normal intervals explains approximately 95.5% of the main AUPRC gain, while the additional components provide smaller and less consistent improvements.
  2. Maximum-score preservation lacks sufficient justification. The two models may attain their maxima at different timestamps, so preserving the maximum does not necessarily preserve genuine anomaly evidence or anomaly discrimination.
  3. The interval lengths and switching rule are heuristic and lack a clear selection principle.
  4. Comparisons with strong baselines under identical feedback conditions are insufficient. Practical annotation costs and additional feedback delays remain unexamined.
Questions for authors
  1. How much annotation effort does coarse feedback save compared with precise localization, and how can an annotator reliably confirm that an entire interval is normal?
  2. Why is the interval maximum an appropriate preservation target? Could this objective amplify false positives when the maximum occurs at a normal timestamp?
  3. How should the coarse and fine interval lengths be selected without test labels? Does the strategy remain effective across different anomaly durations?
  4. Under the same feedback budget, are the additional gains over adaptation on confirmed-normal intervals statistically significant?
  5. Could the authors provide comparisons under identical feedback conditions and evaluate alternative evidence-preservation objectives?
1 comment

We thank the reviewer for the detailed feedback. We have revised the manuscript and added targeted analyses.

W1 / Q4 — Utility and significance. Confirmed-normal adaptation accounts for most of the observed gain (0.5411 AUPRC versus 0.3267 Frozen; FITTA reaches 0.5511). Crucially, this adaptation uses externally certified normal intervals rather than model-derived pseudo-labels. The previously reported 95.5% is a descriptive cross-configuration ratio, not a causal decomposition. Under identical adaptive feedback, preservation adds 0.0164 AUPRC (5/7 families; exploratory Holm-adjusted p = 0.1719; Sec. 5.3, Table 8).

W2 / Q2 — Preservation objective. MAX is an interval-level surrogate consistent with existential feedback, without assuming that the source and adapted maxima identify the same anomaly. Figure 2 quantifies the detection–false-alarm trade-off (+2.52 pp TPR, +0.64 pp FPR), and Appendix G provides an incorrect-witness example.

W3 / Q3 — Resolution and duration. The H20/H10 configuration is heuristic; Sec. 5.1 clarifies its development-time selection and feedback-only deployment rule. Table 5 evaluates matched-response schedules. Appendix H additionally reports duration-stratified preservation effects of −0.0048, −0.0042, and +0.0166 AUPRC across the three tested bins.

W4 / Q1 / Q5 — Comparisons and feedback practicality. Tables 2 and 8 provide same-feedback comparisons, including alternative LSE and source-witness objectives (0.5578 and 0.5642 AUPRC versus 0.5511 for MAX). Table 5 provides matched-response scheduling controls. We distinguish interaction frequency from human annotation effort (Sec. 6). An additional feedback-interval delay reduces AUPRC from 0.5511 to 0.5082 (Table 8).

Reviewer_dEV7OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes FITTA, an adaptive time series anomaly detection (TSAD) mechanism based on test-time adaptation. It uses positive and negative feedback based on the presence of an anomaly event in the previous subsequence of time steps and adopts the detector. The authors find that the majority of the gains come from adapting the sub-sequences detected as normal by the detectors.

Critical strengths
  1. The paper addresses an important problem- drift in normal data distribution and how it causes streaming data models, like a time series anomaly detector.
  2. They come up with an asymmetric update mechanism based on external feedback, which helps improve the TSAD performance reflected by the VUS-PR scores.
Critical weaknesses
  1. The paper is based on a feedback mechanism from an external feedback agent, which in itself has been discussed in several prior works before [1,2], questioning what the paper brings in on top of the existing knowledge on adaptive anomaly detection.
  2. The method's integral component is an external feedback agent which provides feedback regarding the normalcy of the previously time sub-sequence. However, the external agent itself was not studied neither in the form of a human-in-the-loop setting, where a human being provides feedback regarding whether the time series is anomalous or not, not attempting to make a feedback agent separately. Instead, they used the ground truth anomaly labels themselves to simulate the setting. Using labels to simulate adaptation and the fact that adapting normal data yielded the largest gains, separating normal data drift from anomaly is the biggest bottleneck in the paper. Using ground truth to simulate anomaly feedback entirely skips this, making it a circular problem.
  3. Using noise-based simulation can not mimic true human feedback or an external feedback agent.
  4. Without an effort in building a feedback agent for human feedback, the time lag and feedback accuracy can not be judged, which are critical in streaming data applications.
  5. The paper seems unnecessarily terminology-based, like an AI-written paper, and made the reading difficult to follow.

[1] Siddiqui, Md Amran, et al. "Feedback-guided anomaly discovery via online optimization." Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining.

[2] Ruff, Lukas, et al. "Deep semi-supervised anomaly detection." ICLR 2020.

Questions for authors
  1. Use the TSB-AD corpus [3] as a benchmark, which is a cleaned benchmark for TSAD, given the alarming issues found with previous TSAD datasets [4].

[3] Liu, Qinghua, and John Paparrizos. "The elephant in the room: Towards a reliable time-series anomaly detection benchmark." Advances in Neural Information Processing Systems 37 (2024): 108231-108261.

[4] Wu, Renjie, and Eamonn J. Keogh. "Current time series anomaly detection benchmarks are flawed and are creating the illusion of progress.

2 comments

We thank the reviewer for the feedback. We have clarified the contribution, experimental protocol, and practical assumptions.

W1 — Novelty. Our contribution is not external feedback itself, but strictly future-only TTA using existential interval feedback without anomaly localization. Unlike instance-specific feedback, one response summarizes multiple previously scored detection units (Secs. 2–4).

W2 — Feedback simulation and circularity. Ground-truth-based feedback simulation is established in feedback-driven anomaly detection, including Siddiqui et al. (KDD 2018, the reviewer's Ref. [1]). Our experiment evaluates adaptation given this feedback interface, not feedback acquisition. Standard FITTA receives only one existential bit after each completed interval; point labels are never supplied to its updater, and committed scores cannot be revised (Secs. 3 and 5.1). Thus, ground-truth simulation does not introduce circularity into the conditional adaptation evaluation.

W3–W4 — Feedback realism and latency. Table 6 evaluates sensitivity to directional feedback errors. We additionally evaluate a one-interval feedback delay, under which AUPRC decreases from 0.5511 to 0.5082 (Table 8). Sec. 6 distinguishes these controlled evaluations from real feedback-provider accuracy and annotation effort.

W5 — Presentation. We have clarified the terminology, feedback chronology, and experimental comparisons throughout the revised manuscript.

Q1 — TSB-AD. We appreciate the suggestion. Our evaluation uses the original versions of seven established dataset families under a consistent chronological protocol (Sec. 5.1; Appendix B).

Reviewer_dEV7OpenReview ↗

Acknowledgement

Thank you for the response. In light of unsatisfactory novelty and inadequate experiments- for example, not using the TSB-AD benchmark for evaluation, which is a cleaned version of the erroneous original datasets- I will keep my rating unchanged.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.