acceptodds
Under review as a conference paper at ICLR 2027

Can Language Models Program Better Time-Series Anomaly Detectors?

Abstract

Language-model agents for time-series anomaly detection typically inspect series directly or orchestrate existing detectors to identify and explain anomalies. We introduce AgentAD (Agent As Developer), in which the agent writes a standalone Python detector rather than serving as the detector itself. The resulting program runs without any language-model calls. We design an evidence-driven loop to help the agent develop detector programs that generalize to unseen series. The agent explores development series using code and submits candidate detector programs for evaluation. The framework evaluates each candidate on validation data, returning aggregate metrics and selected diagnostic cases through which the agent can inspect validation signals and labels. This bounded feedback is designed to help the agent diagnose failures, explore alternatives, and refine its code while reducing the risk of overfitting to the validation data. The framework retains the best-scoring detector program, while holdout series are reserved for final evaluation. We further train the agent with reinforcement learning in the same development environment, improving its ability to explore data, construct detector programs, and use diagnostic feedback to guide subsequent code revisions. Across multiple datasets and evaluation settings, detectors developed by AgentAD outperform established time-series anomaly detection methods. Under matched development budgets, reinforcement learning enables the trained agent to match or even surpass larger general-purpose language models at detector development. The agent framework, training pipeline, and model checkpoints are available at https://anonymous.4open.science/status/AgentAD_anonymous-9D20.

Primary area:
learning on time series and dynamical systems
Keywords:
time-series anomaly detection, LLM agents, program synthesis, reinforcement learning, code generation, AI for time series

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_mfTqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper uses an LLM agent as a detector developer. For each class of related series, the agent explores labeled training windows in a sandbox and writes a standalone Python detector, revising it over several submissions based on validation metrics and labeled validation examples. The best candidate is then run on held-out series from the same class with no LLM calls. On TSB-AD, detectors written by all seven developer models beat the strongest established method in VUS-PR. The paper also proposes C-DAPO, a variant of DAPO that gives each submission extra credit for later improvement, and uses it to train Qwen3.5-35B-A3B as a developer.

Critical strengths

The strongest contribution is the developer paradigm. The LLM is used only to build the detector. The final Python program is inspectable and runs without LLM calls. Complete held-out series remain outside development, which matters because the agent receives repeated validation feedback. The ablations show that diagnostic feedback helps and that unrestricted validation access leads to overfitting. The study also tests seven developer models and releases the code, costs, and checkpoints.

Critical weaknesses
  1. The agent uses labels at every stage. It sees training labels and labeled validation examples. Thresholds are set with validation labels, and the final program is chosen with label-based scores. The paper does not show that the baselines get the same labels or tuning. So the comparison does not show how much the LLM-written code itself helps. In this setup, each new class also needs labeled anomalies and a new agent run.

  2. The paper does not compare with Argos, the most similar earlier method. Like AgentAD, Argos uses labeled data and LLM feedback to build detectors that run without an LLM. So it is unclear whether AgentAD does better than Argos under the same conditions. The paper also does not compare with recent LLM- or VLM-based detectors, such as LLMAD and VLM4TS, which it discusses.

  3. Each detector is built and tested on series from the same class, which share the same dataset, domain, and channel layout. So the results show that detectors work on similar series, not on new datasets. TSB-AD is also public, so the developer models may already know it.

  4. Each class uses one data split and one agent run. Some gains over the best baseline are small. The RL comparison uses only seven classes, with no repeated seeds and no confidence intervals. The Segment Holdout results are not a matched comparison, but they are bolded with the main results.

  5. C-DAPO uses about 84% more tokens and 48% more submissions than DAPO, even though both have the same limits. The paper does not show how much of the gain comes from the method and how much from this extra compute. The claim that it matches larger models is also limited. It beats DeepSeek-V4.1-Flash, is about equal to Qwen3.8-27B, and is worse than both GLM models.

Questions for authors
  1. How do the baselines perform with per-class model selection and tuning on the same data?
  2. How does AgentAD compare with Argos, and with recent LLM- or VLM-based detectors?
  3. Do the detectors work on classes or datasets they were not built for?
  4. How much do the results change across data splits and repeated runs?
  5. How does C-DAPO compare with DAPO at the same token budget?
1 comment

You were right about Segment Holdout. The baselines were scored on the final portion specified by the benchmark, while AgentAD was scored on its own internal-test windows. We recomputed Table 1 with the same train, validation and test windows for every method. All seven language-model developers still exceed the strongest established method on mean VUS-PR. Revisions are marked in blue.

W1/Q1. Supervision and tuning. “Uses labels at every stage” conflates what the evaluator knows with what the agent sees. Labels mark individual timestamps. The agent sees training labels and selected labeled validation examples. Under Series Holdout it gets only aggregate internal-test scores from development recordings. Entirely separate recordings are held out. Under Segment Holdout the internal-test windows are the final holdout, so even their aggregate scores stay hidden during development. The delivered detector receives signals and scores each timestamp. Only the evaluator uses holdout labels.

A class groups recordings from the same dataset and domain with the same channel layout. Most neural baselines are trained or fine-tuned on normal segments identified from that class’s development labels. DADA and Time-RCD use released checkpoints. TSPulse is reported both zero-shot and fine-tuned. Classical methods fit each series without labels. All methods calibrate thresholds on validation data. Threshold choice cannot improve VUS-PR (Section 2.3, Table R6). AgentAD can also revise code after inspecting failures. We used published baseline settings and did no additional per-class search over methods or hyperparameters, so Table 1 cannot answer what that search would achieve.

Each new class requires labeled development data and one run. Among developers with public API prices, the highest average cost was under two dollars per class. The detector then runs on a CPU without model calls (Section 4.5).

W3/Q3. Generalization and pretraining exposure. Series Holdout tests a detector on new recordings from its class. A program built around ECG beat intervals has little use on server traffic. The developer can examine traffic data and write a suitable program. We tested that ability on seven classes from five datasets and five domains, excluded throughout supervised fine-tuning and RL. C-DAPO improved on DAPO on average (Section 5.3). Transfer of a frozen detector across datasets remains untested.

Even knowing that a TSB-AD recording contains an anomaly leaves its start and end unknown. In Series Holdout the developer never sees the holdout recording. It gets a short domain description and inspects development arrays through Python under standardized filenames. We supply neither dataset nor original file names. We saw no indication of recalled labels in the trajectories we inspected. We cannot audit pretraining corpora, but a public benchmark alone is no evidence that a developer recovered held-out annotations.

W2/Q2. Argos and LLM/VLM baselines. Argos also dispenses with the LLM after development. Its main evaluation combines binary rules with a trained base detector and uses point-adjusted F1. AgentAD writes a complete program that returns continuous scores. Their published results cannot settle which system wins under common conditions. We would be willing to add a camera-ready comparison with matched data access, budgets and evaluation. LLMAD calls a model for anomaly decisions and VLM4TS for interval refinement on new inputs. A comparison would need to count those ongoing inference costs. Section 2.2 explains our focus on standalone detectors.

W4/Q4. Robustness. One split and one run per developer per class cannot estimate seed or split variation. All seven developers beat the strongest baseline on mean Series Holdout VUS-PR. Comparing the best developer and best baseline within each class gives 26 wins in 33 classes (Appendix B.5). Small margins may vary. We will add repeated splits and runs in the camera-ready version, including for the seven-class RL comparison.

W5/Q5. Compute and model size. C-DAPO used more tokens under the same limits. Table R3 reconstructs the retained detector and cumulative cost after each attempt from saved runs, carrying both forward for runs that stopped early. At equal attempt count, DAPO led on holdout VUS-PR at the fifth and C-DAPO at the seventh as it continued revising. At similar mean token use, C-DAPO after five attempts was slightly ahead of DAPO’s final result. An identical token cutoff remains untested.

On Series Holdout VUS-PR the Qwen3.5-35B-A3B developer trained with C-DAPO beats DeepSeek-V4.1-Flash, is close to Qwen3.8-27B, and trails both GLM models (Table 3). We narrowed the claim accordingly.

Reviewer_EsE7OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates the use of language-model agents for automatically developing reusable time series anomaly detectors rather than directly performing anomaly detection. The authors propose AgentAD, an agent-based framework in which an LLM explores development data, generates executable Python detector programs, and iteratively refines them using evaluation metrics and diagnostic feedback. The resulting detectors can operate on unseen time series without further language-model inference. To improve the agent's iterative development ability, the paper further introduces Commit-level DAPO (C-DAPO), which incorporates subsequent detector-quality improvements into reinforcement learning. Experiments on TSB-AD demonstrate strong anomaly detection performance and improvements over standard supervised fine-tuning and trajectory-level DAPO.

Critical strengths
  1. AgentAD integrates data exploration, program generation, execution feedback, iterative refinement, and candidate selection into a coherent development workflow.
  2. The proposed framework achieves improvements over the reported conventional anomaly detection baselines across diverse TSB-AD datasets.
Critical weaknesses
  1. The paper does not adequately explain why its subsequent-gain formulation is preferable to simpler alternatives, such as immediate improvement rewards, discounted future improvements, or conventional process reward shaping.
  2. Although DAPO and C-DAPO share the same maximum development budgets, their actual computation consumption differs substantially. The reported performance advantage may partially result from additional search and refinement opportunities rather than a more effective credit assignment mechanism.
  3. The reported performance advantage may partially result from additional search and refinement opportunities rather than a more effective credit assignment mechanism.
  4. The paper argues that bounded diagnostic feedback and asymmetric data access improve detector development while limiting overfitting. However, several framework ablations change multiple factors simultaneously.
Questions for authors
  1. Why is the subsequent-gain formulation used in C-DAPO a more effective credit assignment mechanism than simpler progress-based reward designs?
  2. C-DAPO consumes approximately 84% more development tokens and performs 48% more commits than DAPO. Does C-DAPO still outperform DAPO when both methods receive the same computational resources rather than only the same maximum budget limits?
  3. Can the authors better isolate the contributions of diagnostic feedback, repeated candidate generation, and the candidate-selection mechanism?
  4. How well does C-DAPO generalize to unseen multivariate and substantially different anomaly detection domains?
2 comments

Thank you for asking whether the agent writes better candidates or simply has more chances to find one. We went back to the saved submissions to see where those explanations fit. The revisions are marked in blue.

Q1/W1. Why credit a submission for a later improvement?

Appendix R.1 follows a real development run, including the feedback the agent received each time it submitted a detector for evaluation. We call each such submission a commit. The first detector looked promising on a visible sensor anomaly, but its score fell on internal-test windows the agent could not inspect directly. That discrepancy prompted the agent to change how it combined evidence across sensors. The improvement came with the next submission. A reward based only on immediate improvement would miss the first submission’s contribution to the fix.

We use the gain in the best development score from just before a commit to the end of the run. A better detector is equally useful to us whether the agent finds it early or late in its short development budget, so we leave that gain undiscounted. This is a form of process reward shaping. We compare gains across runs on the same task at the same commit position and use them to make a small adjustment to DAPO’s update. The outcome of the whole run still determines whether its actions are encouraged or discouraged. The correction only changes the strength of the update for each submission. We call this adaptation C-DAPO and compare it with DAPO. Whether an immediate or discounted reward would do as well remains open.

Q2/W2. What happens at comparable development cost?

Both policies had the same development limits, and C-DAPO used more tokens and made more submissions before stopping. In Section 5.4 we use the saved runs to reconstruct the detector each policy retained and its cumulative token use after each attempt. Runs that stopped early keep their final detector and cost in the comparison (Table R3).

Comparing equal numbers of attempts, DAPO had higher mean holdout VUS-PR at the fifth and C-DAPO at the seventh, by which point more C-DAPO runs were still revising. At similar average token use, the comparison is between different stages of development. At its fifth attempt C-DAPO had spent roughly what DAPO spent over its entire run, on average, and scored slightly above DAPO’s final detector. These later revisions contributed to C-DAPO’s final advantage. Testing an exact token cutoff would require new runs.

Q3/W3. What can the ablations tell us about feedback, revision, and selection?

Keeping only the first submission from each full run shows how much later attempts added. To test the value of diagnostic feedback, we kept the same submission limit and aggregate scores but withheld the labeled validation excerpts showing where a detector succeeded or failed. Even the best candidate found during those runs scored substantially lower on holdout. Seeing individual cases helped the agent write better candidates (Sections 4.3–4.4).

Figure 3 also asks whether the agent wrote a good candidate that the framework then failed to select. We compared the detector retained using development scores with the best candidate we could identify on holdout after the run. Their mean holdout scores were close in the full framework.

For the access ablations, we first removed the internal-test score from both the feedback given to the agent and the rule for choosing the detector. Selection then relied on validation alone (No Test Feedback). This step changes two things, so it cannot tell us how much each contributed. Starting from that setting, we changed just what the agent could inspect, giving it the complete validation windows and labels in place of selected examples (Full Validation Access). It still produced strong candidates, but a wider gap opened between their holdout performance and that of the detectors selected on validation.

Q4. How far does the trained developer generalize?

We group recordings from the same dataset and domain with the same channel layout into a class. We reserved seven such classes for evaluation, excluding every recording in them from both supervised fine-tuning and reinforcement learning. At evaluation the trained agent developed a fresh detector for each class and improved average holdout performance over DAPO. These classes span five domains, including traffic and medicine, and the split was made at the class level. Transfer to a substantially different domain would need a separate test. Each detector is intended for new series within its own class (Section 5.3).

Training included multivariate tasks. All seven evaluation classes were univariate, so transfer to an unseen multivariate class is still an open question. We would be willing to add that evaluation in the camera-ready version.

Thank you for your reply and clarification. However, there are still several issues that remain unresolved, and the paper still has room for further improvement. Therefore, I have decided to maintain the original score.

Reviewer_REoBOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AgentAD (Agent as Developer), a framework that uses LLMs to develop standalone Python programs for time-series anomaly detection (TSAD). Given class-specific development data, the agent iteratively generates and revises detector programs using performance metrics and diagnostic feedback from an evaluation harness. The selected detector is then applied to unseen time series without requiring further LLM inference.

To improve the agent's development capability, the authors first perform supervised fine-tuning (SFT) using teacher-generated development trajectories, followed by reinforcement learning (RL). They further propose Commit-level DAPO (C-DAPO), which incorporates subsequent quality gains across commits into the advantage estimation. Experiments on TSB-AD show improved detection performance over established TSAD methods, while C-DAPO further improves the agent's detector development capability.

Critical strengths

1. Flexible program generation for task-specific TSAD. The flexibility of program generation is well suited to the heterogeneity of time-series data and the task-specific nature of anomaly detection, enabling detector designs to adapt to different detection scenarios.

2. Separation of detector development and deployment. AgentAD produces standalone Python detectors that operate without further LLM inference, separating LLM-assisted development from deployment and reducing inference-time overhead.

Critical weaknesses

1. Insufficient isolation of the benefits of full-program generation. While full-program generation offers greater flexibility than rule-based approaches such as Argos, the paper does not sufficiently motivate or experimentally isolate the benefits of this broader design space. Consequently, it remains unclear how much of AgentAD's effectiveness can be attributed specifically to full-program generation rather than iterative feedback-driven optimization.

2. Insufficient clarity and control of baseline training conditions. Although Series Holdout provides a common evaluation setting, AgentAD explicitly leverages historical anomaly labels and repeated validation feedback for detector development. The fitting, fine-tuning, and supervision protocols of the traditional baselines are not described with comparable clarity. Without a label-matched non-agentic baseline, it is difficult to disentangle the benefits of LLM-based program generation from those of task-specific supervision and optimization.

3. C-DAPO's improvements are not fully disentangled from increased development computation. Although DAPO and C-DAPO share the same maximum development budget, C-DAPO uses substantially more actual commits and tokens (Table 5). Therefore, it remains unclear whether its performance gains primarily reflect more effective commit-level credit assignment or simply greater exploration effort.

Minor concern The Method section, particularly the description of the AgentAD framework, is overly concise, with important details deferred to the appendix. The Introduction also lacks a clear concluding summary of the paper's main contributions. Together, these issues make the overall methodology and narrative somewhat difficult to follow.

Questions for authors
  1. Could the authors further clarify the motivation for full-program generation over more constrained rule-generation approaches, such as Argos, and provide a controlled comparison under matched data access and development budgets?
  2. How are traditional baselines fitted or fine-tuned using the class-specific development data? Do they have access to comparable supervision and optimization opportunities? A label-matched detector selection or hyperparameter optimization baseline would help clarify the source of AgentAD's improvements.
  3. Does C-DAPO retain its advantage over DAPO when actual token consumption or commit counts are matched? Additionally, how does the proposed subsequent-gain advantage compare with simpler alternatives, such as immediate commit-level improvement?
2 comments

Thank you for asking where the gains come from. We have clarified how each method uses development labels and marked the revisions in blue.

W2/Q2. How did the baselines use the labels?

A class groups recordings from the same dataset and domain with the same channel layout. All methods share the splits within each class, and holdout labels stay hidden during development. We trained or fine-tuned most neural baselines on segments identified as normal from the development labels. DADA and Time-RCD provide checkpoints without a fine-tuning procedure. We evaluated both zero-shot and fine-tuned TSPulse. Classical methods fit each series without labels. All methods calibrate thresholds on validation data. This affects Range-F1, while VUS-PR measures ranking across thresholds (Section 2.3 and Table R6 in Appendix R.2).

AgentAD can inspect a labeled validation failure and revise its code. For the baselines we used published configurations. Your suggested per-class search over detectors and hyperparameters would test a further use of those labels that our comparison leaves open.

W1/Q1. What did writing a full program add?

Appendix R.1 traces a case in which several sensors went quiet during a validation anomaly. The agent checked the training recordings and found even quieter periods of ordinary rest. Its first detector therefore learned which values were rare for each sensor and combined rarity with stillness. We later found that this design already beat the strongest established method on holdout. What the agent saw during development was a gain over a simple histogram detector on validation but a drop on internal-test windows it could not inspect. It changed the scoring code to give a few unusual sensor readings more weight. That edit accounted for most of the subsequent holdout gain (Section 4.6).

Keeping the first submission from each run instead of the detector retained after revision lowers mean holdout VUS-PR by about 0.19. Giving the agent all development data and labels to explore in Python, but only one final submission and no intermediate evaluation, gave a result close to the first submission. Removing just the labeled diagnostic examples also hurt when scores and further attempts remained available (Section 4.3).

How much could a rule-writing agent gain from the same feedback? Argos revises Python rules and combines them with a trained base detector to produce binary decisions evaluated by point-adjusted F1. AgentAD writes the fitting and scoring code and returns continuous scores. Appendix D.3 gives four complete programs that run on a CPU without another model. We would be willing to add a camera-ready Argos comparison under matched data access, budgets and evaluation.

W3/Q3. How much does development cost explain?

Under the same limits, C-DAPO kept revising longer and spent more tokens. We reconstructed Table R3 from the saved runs, recording the retained detector and cumulative cost at each attempt. Runs that stopped early keep their final detector and cost. At equal attempt count, DAPO led on mean holdout VUS-PR at the fifth and C-DAPO at the seventh. At comparable mean token use, C-DAPO’s fifth attempt was already slightly ahead of DAPO’s final result. An identical token cutoff remains untested.

A submission can expose a failure that the next revision fixes. An immediate-improvement reward would miss that contribution. We kept DAPO’s final reward and added a small correction for each detector submitted for evaluation (a commit). The outcome of the run determines the direction of the update, while later progress adjusts its strength for each commit. We tested this adaptation against DAPO. Its performance against immediate or other progress rewards remains open.

Presentation. Section 3 now explains the train, validation and internal-test windows and the feedback after each submission. The Introduction closes with the contributions. Section 4 brings the AgentAD experiments together, including development cost (Section 4.5). At the public API prices we used, even the most expensive developer with a reported price averaged under two dollars per class. The finished detector scores new series without further model calls.

Reviewer_REoBOpenReview ↗

Response after having read the rebuttal

Dear Authors,

Thank you for the detailed responses. While these clarify a number of aspects, I'm still not sufficiently convinced. Also the presentation of the paper could still be further improved. Therefore, I will retain my original score.

Best regards

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.