acceptodds
Under review as a conference paper at ICLR 2027

SERAM: Self-Evolving Runtime Anomaly Detection for LLM Agents

Abstract

LLM agents execute interdependent steps, allowing local errors to propagate through subsequent actions. Runtime detectors must identify such errors from the observed trajectory prefix. Changing tasks, tools, and policies can introduce new failure patterns, while feedback on detection errors arrives only later and cannot be used by a static detector. We study adaptation from delayed feedback under three deployment requirements: uninterrupted detection, validated updates, and budgeted LLM inference. We propose SERAM, a self-evolving runtime detector that keeps its LLM frozen and couples an editable behavior graph with lightweight retrieval and scoring models. The graph encodes agent behaviors, transitions, and anomaly rules; the lightweight models handle most steps and escalate uncertain cases to LLM analysis according to a shared routing rule. Given delayed labels, SERAM groups errors by retrieved behavior nodes, revises their rules or adds nodes, and retrains the lightweight models to incorporate the revised knowledge. Updates are prepared asynchronously and deployed only after passing predefined validation checks on held-out feedback. We evaluate under a streaming protocol that releases labels in batches so that updates affect only subsequent predictions, on StreamAD, a new benchmark with step-level labels and measured step timings. Across the full test stream, self-evolution improves average precision (AP) by 10.25% over the frozen counterpart. Self-evolution also reduces mean detection latency by 33% relative to the frozen counterpart. Overall, SERAM improves AP by 62% over the strongest baseline and is the only evaluated LLM-based detector whose mean latency is below the agents' mean step time.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Streaming Agent Anomaly Detection, Budget-Aware Detection, Asynchronous Evolution

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_iTZ2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper introduces a framework to detect and correct anomalies in multi-agent systems through an automated feedback loop. SERAM treats anomaly detection rules as evolving hypotheses that evolve over time.

The approach operates by diagnosing anomalies against retrieved behavior nodes, proposing rule revisions, retraining lightweight models (such as text encoders, GRU, MLP), and validating the resulting candidate against a frozen baseline before deployment.

The authors report substantial gains across diverse domains.

Critical strengths

I like the idea of safe evolution via correction, which entails that a new candidate should both improve PR-AUC and correct more decisions than it regresses. This implicitly targets the notion of harmful vs. useful updates in agentic systems, often overlooked

Critical weaknesses
  • Results are reported for four unseen domains, but no transferability is demonstrated to substantially different system types. Coverage couldbe broader.

  • The paper does not quantify the empirical complexity of maintaining a frozen baseline and validation on each candidate batch. I am concerned about settings with tight latency budgets

  • I could not find a link to the code implementation

  • One potential drawback is the limited background infromation about for readers unfamiliar with agentic system architectures. But I understand that authors had to deal with space limitations.

Questions for authors

To better understand SERAM's scalability, how does the system maintain its frozen baseline over extended horizons, and what metrics trigger its periodic retraining cycles? What are the latency bottlenecks, if any, and can they be mitigated? Furthermore, have the authors considered the chance of non-stationary, adversarially shifting threat distributions where anomaly definitions drift? On a different note, one concern is whether the domains adopted are large enough in scope to justify the generability of the approach?

3 comments
AuthorsOpenReview ↗

Response to Reviewer iTZ2 (Part 1)

Thank you for your constructive feedback and suggestions. We respond to each point below.

W1: Results are reported for four unseen domains, but no transferability is demonstrated to substantially different system types. Coverage could be broader.

Thank you for your suggestion regarding broader coverage. We currently cover airline, banking, retail, and telecom, which differ in their knowledge requirements, business rules, and interactions. Telecom agents guide users through device operations and troubleshoot based on user feedback; banking agents handle sensitive account information under authentication, authorization, and confidentiality requirements; retail agents query and modify orders and process returns and exchanges; and airline agents combine itinerary information with ticketing policies to change reservations. These monitored agent systems differ in their task states, tool operations, business constraints, and user interactions, providing broad coverage for examining our method's applicability across business systems. We have added descriptions of these environments and representative tasks in Appendix B.1 (Table 4) of the revised manuscript.

W2: The paper does not quantify the empirical complexity of maintaining a frozen baseline and validation on each candidate batch. I am concerned about settings with tight latency budgets.

Thank you for raising the concern about update overhead and latency budgets. Regarding update and validation overhead, a complete update cycle takes approximately 13–15 minutes on a Xeon Silver 4214 and one NVIDIA RTX 6000, including diagnosis, proposal, retraining, and validation. The system maintains only the current and candidate versions, with each checkpoint occupying approximately 368 MB, including optimizer state. During validation, the estimator-router modules of both versions rescore the validation data without additional LLM calls. Updates run in the background while the current version continues online detection.

For online detection, SERAM controls LLM usage by adjusting the target Screen/Review/Audit shares to accommodate different cost and latency budgets. The configurations are shown below.

Routing budgetAPMean detection latencyp95p99Detection LLM cost (USD)
60/30/10 (default)0.4202.36 s11.3 s21.3 s$14.02
80/15/50.3950.91 s7.77 s15.4 s$5.62
90/8/20.3940.52 s5.01 s11.3 s$3.19
100/0/00.3470.002 s0.002 s0.002 s$0.00

When the latency budget is particularly tight, we can disable LLM inference to achieve the lowest latency and cost.

W3: I could not find a link to the code implementation.

Thank you for flagging this. Our anonymous code repository is available at SERAM.

W4: One potential drawback is the limited background information for readers unfamiliar with agentic system architectures. But I understand that authors had to deal with space limitations.

Thank you for your understanding. An agent typically comprises a large language model, tool interfaces, and an execution environment: the model selects actions based on the task and interaction history, calls tools, receives results, and then uses those results to decide its next step. Errors in an earlier step can therefore affect subsequent decisions. This motivates the use of SERAM to detect anomalies during execution using the current step and preceding history, helping identify errors that may propagate to later steps. We have described the monitored agent's execution flow and where SERAM performs detection in Appendix A.1 of the revised manuscript, and added descriptions of the benchmark environments and representative tool operations in Appendix B.1.

Q1: how does the system maintain its frozen baseline over extended horizons, and what metrics trigger its periodic retraining cycles?

During each update, the current detector remains fixed, continues online detection, and serves as the reference for candidate validation. In our experimental setup, labels are released after each batch of trajectories has been scored, triggering a background update. A candidate that passes validation replaces the current version, so only the current and candidate versions need to be maintained.

Q2: What are the latency bottlenecks, if any, and can they be mitigated?

The main bottleneck is LLM analysis in Review and Audit, which take 7.3 s and 14.1 s on average, respectively, while Screen takes 0.002 s. Increasing the target Screen share reduces LLM calls and latency. See W2 for the specific budgets and detection performance.

AuthorsOpenReview ↗

Response to Reviewer iTZ2 (Part 2)

Q3: have the authors considered the chance of non-stationary, adversarially shifting threat distributions where anomaly definitions drift?

We use feedback to revise anomaly rules and train lightweight models, allowing the detector to adapt to changes in the monitored agent's behavior; candidates are deployed only after validation. In the unseen-target-domain experiment, feedback-driven updates improve performance in all four domains, with an average relative gain of 9.9% (Figure 4(d)). This supports adaptation to unseen domains.

Changes in task domains themselves introduce distribution shifts, which our unseen-target-domain experiment evaluates. However, the current dataset does not adequately characterize or quantify natural drift in the monitored agent during prolonged operation, so we cannot yet accurately assess our method's performance under such drift. The lack of evaluation of long-term natural drift is acknowledged as a limitation in Appendix A.5.

Q4: whether the domains adopted are large enough in scope to justify the generability of the approach?

The four domains described in W1 have different knowledge requirements, tool operations, and business constraints. We also use a task-family split to evaluate unseen tasks and a leave-one-domain-out experiment to assess adaptation after transfer to an unseen business domain. Together, these experiments support generalization across the evaluated settings.

Thank you for the helpful clarifications, the expanded experiments, and the release of the code implementation. I am still not fully convinced about the transferability across domains, but I remain positive about the paper. I will increase my score in the Soundness area

Reviewer_pZSQOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

SERAM studies how to detect abnormal LLM-agent actions during execution. It assumes that labeled feedback arrives later and can help improve future detection. The method uses a behavior graph, a lightweight detector, and selective LLM calls. When feedback arrives, it updates the rules and retrains the detector. New versions are checked before use. The authors build STREAMAD with injected faults across four customer-service domains. It contains 2,045 trajectories and 15 anomaly types. Experiments compare SERAM with eight baselines and test its main components. SERAM reports a 62% relative gain in average precision over the strongest baselines and a 10.25% gain over its frozen version. The ablation results suggest that model retraining helps more than graph updates.

Critical strengths

Detecting abnormal LLM-agent actions during execution is important, especially when errors can affect later steps. The focus on delayed feedback and runtime monitoring is useful.

It uses a behavior graph, lightweight detection models, and selective LLM calls to balance detection accuracy and cost. The framework also checks updates before deployment.

SERAM improves average precision across all four tested domains. The ablation studies and frozen-model comparison also help show which parts of the framework contribute to the results.

Critical weaknesses
  1. Runtime anomaly detection and feedback-based adaptation are already well-studied problems. SERAM mainly combines existing methods, including retrieval, supervised retraining, rule revision, and uncertainty-based routing. Its fundamental methodological contribution remains unclear.

  2. The paper reports substantial gains over existing detectors but lacks a strong continual-learning baseline that uses the same delayed labels, update schedule, and inference budget. Therefore, it is difficult to determine whether the improvements come from the proposed self-evolution mechanism or simply from additional supervised training.

  3. STREAMAD is newly constructed using injected faults and LLM-generated step-level annotations. The paper provides limited independent validation of the fault realism and annotation quality. This weakens confidence in the reliability and generalizability of the reported performance.

  4. The modeling assumptions behind self-evolution are not clearly established. Although the paper motivates the method using changing agent behaviors, the evaluation shuffles trajectories into feedback batches rather than introducing explicit temporal distribution shifts. It is therefore unclear whether SERAM can handle gradual drift, sudden policy changes, or emerging anomaly patterns. The reported improvements may mainly reflect learning from additional labeled data under a relatively stable distribution.

Questions for authors
  1. What is the main technical novelty beyond combining existing methods?
  2. How does SERAM compare with standard continual learning using the same delayed feedback? 3.Does the shuffled evaluation setting reflect real temporal distribution shifts?
  3. How were the injected faults and LLM-generated labels validated?
4 comments
AuthorsOpenReview ↗

Response to Reviewer pZSQ (Part 1)

Thank you for your careful review and for recognizing the value of runtime detection, delayed feedback, and validation before deployment. The following are our responses.

W1: Runtime anomaly detection and feedback-based adaptation are already well-studied problems.

Thank you for the opportunity to clarify our contribution. To our knowledge, existing runtime detectors for LLM agents remain fixed after deployment, and existing benchmarks do not evaluate detectors on streams with delayed label release. We would be glad to compare with any additional methods we may have overlooked.

Our methodological contribution is a design for this setting rather than any single component. At runtime, the LLM can analyze only a minority of steps, so both detection and adaptation have to work largely outside the LLM. SERAM therefore makes one editable behavior graph the shared knowledge source for the whole system: retrieval reads it, the lightweight estimator-router scores and routes steps using it, the LLM cites it on escalated steps, feedback revises it, and the lightweight models are retrained over the revised graph. The results below show what this design provides.

  1. Detection without the LLM. With no LLM calls (100/0/0), SERAM reaches 0.347 AP at about 2 ms per step, above TRAIL and ADR (0.259 each), which take 13.7 s and 9.87 s per step on average. The structure matters: replacing the behavior graph and trained retrieval with plain RAG lowers AP from 0.420 to 0.382, and removing retrieval lowers it to 0.353 (Table 2).

  2. LLM analysis within a runtime budget. Under the default budget, SERAM reaches 0.420 AP, 62% above the two baselines, at a mean latency of 2.36 s. The probability of completing detection within the inter-step interval is 80.6%, compared with 15.55% for TRAIL and 30.46% for ADR.

  3. Feedback improves accuracy and reduces cost together. Relative to the same detector without self-evolution, AP rises by 10.25% (0.381 to 0.420), mean latency falls by 33% (3.53 s to 2.36 s), and detection LLM cost falls from $21.40 to $14.02 ($15.52 including evolution). In unseen domains, target-domain feedback improves AP by 9.9% on average and by up to 17.5% (see W4). See W2 for a comparison with estimator-router training alone, including accuracy, detection cost, and latency.

  4. Updates are validated before deployment. Of four candidates in the main run, two are rejected, including one that raises validation PR-AUC but regresses more decisions than it corrects. The candidate-validation analysis in the revised appendix reports these results.

AuthorsOpenReview ↗

Response to Reviewer pZSQ (Part 2)

W2: The paper reports substantial gains over existing detectors but lacks a strong continual-learning baseline that uses the same delayed labels, update schedule, and inference budget.

Thank you for raising this concern. We distinguish the advantage of the underlying detection framework from the gains obtained through feedback-driven updates, and report inference resource usage alongside detection performance.

First, with self-evolution disabled, SERAM achieves 0.381 AP without using test-stream feedback for additional training, compared with 0.259 for both TRAIL and ADR. Thus, an advantage of 0.122 AP is already present before feedback-driven adaptation.

MethodAPPR-AUC
TRAIL0.2590.261
ADR0.2590.248
SERAM without self-evolution0.3810.382

Second, to examine whether other methods with additional supervised training can achieve comparable results, we draw on the continual-learning method in the original OCAR paper [1] and adapt it to anomaly detection at the current step. It receives the same current step and preceding trajectory history as SERAM and uses the same frozen text embedding encoder. Plain RAG retrieves relevant historical examples by embedding similarity, and an MLP detects anomalies using the current input and retrieved information. OCAR continually updates the detection model using new feedback. We use the same delayed-label release mechanism and four-round feedback schedule as SERAM. To make inference resource differences explicit, we report both default SERAM and its configuration without inference-time LLM calls. Without inference-time LLM calls, SERAM and OCAR both have millisecond-scale detection latency, while SERAM achieves better detection performance. This shows that its detection advantage over this continual-learning baseline does not depend on additional inference-time LLM calls.

MethodAPPR-AUCF1Steps using an LLMMean detection latency
OCAR0.2580.2580.2740%0.004 s
SERAM (100/0/0)0.3470.3570.3170%0.002 s
SERAM0.4200.4210.38426.6%2.36 s

To further examine the contribution of feedback-driven updates, we compare detection performance, detection LLM cost, and mean detection latency across three configurations: SERAM without self-evolution, estimator-router training alone with the behavior graph and retrieval models fixed, and full SERAM.

ConfigurationAPDetection LLM cost (USD)Mean detection latency
SERAM without self-evolution0.381$21.403.53 s
Estimator-router training only0.411$19.043.20 s
Full SERAM0.420$14.022.36 s

All three configurations use the same Screen/Review/Audit target shares (60/30/10). The realized fractions of steps sent to the LLM are 41.4%, 36.4%, and 26.6% for SERAM without self-evolution, estimator-router training only, and full SERAM, respectively. By updating the behavior graph and retrieval models, knowledge evolution provides more suitable evidence for the estimator-router. Compared with estimator-router training alone, full SERAM reduces mean uncertainty from 0.195 to 0.159, allowing more steps to be handled directly by Screen and thereby reducing the realized fraction of steps sent to the LLM. Correspondingly, detection cost decreases from $19.04 to $14.02, and mean detection latency decreases from 3.20 s to 2.36 s. These results show that knowledge evolution provides further improvements in detection performance beyond estimator-router training, while reducing detection latency and LLM cost.

[1] Edoardo Urettini and Antonio Carta. Online Curvature-Aware Replay: Leveraging 2nd Order Information for Online Continual Learning. ICML, 2025.

AuthorsOpenReview ↗

Response to Reviewer pZSQ (Part 3)

W3: STREAMAD is newly constructed using injected faults and LLM-generated step-level annotations.

Thank you for raising the concern about dataset construction and label reliability. We provide additional evidence from detection on naturally occurring anomalies and independent label verification.

First, we select autonomous trajectories generated entirely without fault injection from the StreamAD test set and compare SERAM, TRAIL, and ADR. This subset contains 226 trajectories and 3,544 steps, including 458 anomalous steps (12.9%). All three methods follow the main experiment's prefix-only constraint, using only the current step and preceding history for detection, and are evaluated using step-level AP. SERAM achieves an AP of 0.275 on this subset, compared with 0.182 for TRAIL and 0.180 for ADR, showing that its detection advantage also holds for naturally occurring anomalies.

MethodAP
TRAIL0.182
ADR0.180
SERAM0.275

StreamAD uses environments from the τ\tau-bench family to simulate practical task scenarios, including multi-turn interactions between agents and users, as well as tool calls to retrieve information and query or modify stateful databases. The trajectories include both autonomous runs without intervention and runs that continue autonomously after fault injection. We annotate steps by examining the specific behavior in each trajectory together with task-success or task-failure feedback from the environment. This feedback serves as supporting evidence, allowing us to obtain step-level annotations for naturally generated trajectories as well.

Second, we independently audited 228 of the 2,230 originally anomalous test steps (10.2%) and 228 originally normal steps as controls. The human annotator confirmed 89.5% of the original anomalous labels and 92.5% of the original normal labels, agreeing with the original labels on 415 of the 456 audited steps (91.0%). Claude Opus 5.5 independently annotated the same sample and also agreed with the original labels on 415 steps (91.0%).

We randomly sampled steps within each domain, with proportional coverage across the four domains and a 1:1 balance between originally anomalous and normal steps. The human annotator was blinded to the original labels and reviewed each sampled step in trajectory order, using only the current step and preceding history. The results are shown below.

Original labelHuman: normalHuman: anomalousClaude: normalClaude: anomalous
Normal (228)211 (92.54%)17 (7.46%)219 (96.05%)9 (3.95%)
Anomalous (228)24 (10.53%)204 (89.47%)32 (14.04%)196 (85.96%)

To assess the sensitivity of method comparisons to annotation differences, we recomputed AP for SERAM, TRAIL, and ADR on the same human-audited test subset using the human labels and Claude labels, respectively. SERAM outperforms TRAIL and ADR under both sets of labels.

MethodHuman-label APClaude-label AP
TRAIL0.7240.697
ADR0.6720.646
SERAM0.7960.776
AuthorsOpenReview ↗

Response to Reviewer pZSQ (Part 4)

W4: The modeling assumptions behind self-evolution are not clearly established.

Thank you for drawing attention to changing operating conditions and the assumptions behind self-evolution. We focus on whether the detector can use feedback to adapt to new behaviors when the monitored agent's tasks change. Our setting assumes that labels for earlier steps arrive with a delay and that the initialization data do not cover all behaviors encountered after deployment. Label feedback triggers updates, and a candidate is deployed only after passing validation; this process does not require a particular form of temporal drift to be specified in advance.

Our experiment examines adaptation from the initialization domains to an unseen target domain. Specifically, we initialize the detector using three domains, then use the same initial detector in two experiments, with self-evolution enabled and disabled, and compare their results on the fourth, unseen target domain. Both process the same trajectories; the former updates using target-domain feedback, while the latter remains fixed. The results are shown below.

Target domainSERAM APFrozen APRelative gain
Airline0.4490.4109.5%
Banking0.3040.2953.1%
Retail0.3660.33210.2%
Telecom0.3550.30217.5%
Average0.3680.3359.9%

These results show that the detector can use target-domain feedback to adapt to the monitored agent's behavior in a new business domain.

We also acknowledge that the main test stream shuffles trajectories within each domain and does not explicitly introduce temporal drift across feedback batches. The main experiment examines adaptation to task families unseen at initialization, while the leave-one-domain-out experiment further examines domain shifts between initialization and deployment. Natural drift in the monitored agent during prolonged operation has not been fully characterized or evaluated in the current dataset, and we acknowledge this limitation in Appendix A.5 of the revised manuscript.

Q1: What is the main technical novelty beyond combining existing methods?

The novelty is a design for runtime detection with delayed feedback under an LLM budget: one editable behavior graph shared by retrieval, the lightweight estimator-router, LLM analysis, and feedback-driven revision, with updates prepared asynchronously and validated before deployment. This design gives 0.347 AP without any LLM calls (against 0.259 for the LLM-based baselines) and 0.420 AP at 2.36 s mean latency under the default budget. Feedback-driven updates improve AP by 10.25% and reduce mean latency by 33% relative to SERAM without self-evolution. See W1 for details.

Q2: How does SERAM compare with standard continual learning using the same delayed feedback?

We have added a comparison with the continual-learning method OCAR using the same delayed-label release mechanism and four-round update schedule. With neither method making inference-time LLM calls and both operating at millisecond-scale detection latency, SERAM achieves 0.347 AP, compared with 0.258 for OCAR; their mean detection latencies are 0.002 s and 0.004 s, respectively. See W2 for the setup and results.

Q3: Does the shuffled evaluation setting reflect real temporal distribution shifts?

Our evaluation focuses on adaptation to tasks and domains not covered at initialization: the main experiment uses test task families disjoint from the initialization data, and the leave-one-domain-out experiment further examines transfer to an unseen target domain (see W4). In the main test stream, trajectories are shuffled within each domain, while steps within each trajectory retain their actual execution order; temporal drift is not explicitly introduced across feedback batches. Natural drift in the monitored agent during prolonged operation has not been fully characterized or evaluated in the current dataset, and we acknowledge this limitation in Appendix A.5 of the revised manuscript.

Q4: How were the injected faults and LLM-generated labels validated?

The trajectories come from task environments in the τ\tau-bench family and include both fault-injected runs and autonomous runs without intervention. Both types are annotated using the specific behavior in the trajectories together with environment feedback. We independently audited 228 originally anomalous steps, representing 10.2% of the originally anomalous test steps, and 228 originally normal steps as controls. The human annotator was blinded to the original labels, and Claude Opus 5.5 independently annotated the same sample; each agreed with the original labels on 91.0% of the audited steps. We also recomputed detection AP on the audited subset using the human and Claude labels, and SERAM outperformed TRAIL and ADR under both sets of labels. See W3 for the setup and results.

Reviewer_Xs3jOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

SERAM detects anomalous LLM-agent steps online. It combines a revisable behavior graph and trained retrieval with a low-cost estimator/router, escalating selected steps to an LLM under an inference budget; delayed feedback updates the graph and lightweight models after held-out validation (Figure 2; Sections 3.2–3.4). STREAMAD contains 2,045 trajectories and 31,545 labeled steps across airline, banking, retail, and telecom tasks, with 15 injected anomaly types (Section 4.1; Appendix B). The paper reports higher detection AP than adapted baselines in all four domains and an evolving-stream AP of 0.420 versus 0.381 for its frozen counterpart, while reducing average detector latency (Tables 1–2; Figures 3–4).The results support the promise of the complete system but warrant narrower claims until these issues are resolved.

Critical strengths
  1. A consequential online problem with a realistic information boundary.
  2. A principled accuracy–latency design.
  3. Strong results and meaningful internal baselines. Table 1 reports the best AP in all four domains (0.505, 0.382, 0.442, 0.405), while Table 2 compares retrieval removal, plain RAG, other LLMs, knowledge-evolution removal, and model-training removal. The frozen SERAM comparison in Figure 4 is especially relevant to the self-evolution claim.
  4. Feedback validation and cost accounting are considered. The update procedure validates candidate revisions before deployment; two of four candidates are accepted in the main run (Section 4.4; Appendix). Figure 4(c) accounts for LLM calls used in evolution, not only detection, and reports a net token-priced saving over the frozen version.
Critical weaknesses
  1. The benchmark may not represent naturally occurring agent failures.
  2. Average latency does not establish runtime deadlines.
  3. Most adaptation benefit appears to come from supervised estimator/router training.
  4. Baseline adaptation may affect comparative fairness. TRAIL and ADR are adapted from post-hoc trajectory analysis to prefix-only step prediction, while SERAM is designed for this protocol (Section 4.1; Appendix B.3). The common prefix constraint is appropriate, yet this adaptation could materially weaken those methods. Show exactly which information and prompts each baseline receives, compare token/latency-matched configurations, and treat the frozen SERAM comparison as the cleaner evidence for online evolution. 5.Repeated validation and schedule sensitivity need closer analysis. The same stream is studied under 2, 4, 8, and 12 feedback cycles; two cycles finish at AP 0.425, four at 0.420, and twelve at 0.405 (Figure 4(a); Section 4.4). Candidate updates use a held-out feedback pool repeatedly (Appendix), creating possible selection dependence over time. This is a fixable statistical concern. Report uncertainty across streams/seeds, clarify whether validation samples are reused across decisions, and evaluate with a strict time-ordered validation buffer not consulted repeatedly. 6.Cost claims omit some material resources. Figure 4(c) prices LLM calls but explicitly excludes local lightweight-model training; the paper also relies on retrieval, graph maintenance, and model-specific token prices (Section 4.4). Present wall-clock and hardware cost, embedding/graph storage, and sensitivity to different LLM price and routing assumptions before claiming lower total operating cost.
Questions for authors
  1. What fraction of test anomalies has been independently human-verified, and how are injection or judge artifacts prevented from entering the detector input?
  2. What are the p95/p99 detector latencies and the fraction of steps whose detection finishes before that step’s actual next-agent-action deadline?
  3. If only the estimator/router is fine-tuned on released labels, with no graph revision, how does its AP and cost compare with full SERAM under the same feedback budget?
  4. Are the same validation examples reused to accept multiple update candidates, and what safeguards prevent overfitting to this pool?
4 comments
AuthorsOpenReview ↗

Response to Reviewer Xs3j (Part 1)

Thank you for your careful review and for recognizing the importance of online detection, the prefix-only information constraint, and the accuracy–latency trade-off. The following are our responses.

W1: The benchmark may not represent naturally occurring agent failures.

Thank you for this suggestion. StreamAD includes two types of trajectories: autonomous runs without intervention, and runs that continue autonomously after faults are injected at selected steps. To address this concern, we added an evaluation on autonomous trajectories generated entirely without fault injection. Specifically, we select these trajectories from the StreamAD test set to form a separate evaluation subset and compare SERAM, TRAIL, and ADR. This subset contains 226 trajectories and 3,544 steps, including 458 anomalous steps (12.9%). All three methods follow the main experiment's prefix-only constraint, using only the current step and preceding history for detection, and are evaluated using step-level AP. As shown below, SERAM achieves an AP of 0.275 on this subset, compared with 0.182 for TRAIL and 0.180 for ADR, showing that its detection advantage also holds for naturally occurring anomalies.

MethodAP
TRAIL0.182
ADR0.180
SERAM0.275

Regarding benchmark construction, we use environments from the τ\tau-bench family to simulate practical task scenarios, including multi-turn interactions between agents and users, as well as tool calls to retrieve information and query or modify stateful databases. These environments ground anomaly detection in concrete task execution and state changes, while also providing task-success or task-failure feedback. To annotate naturally occurring anomalies in autonomous runs, we examine the specific behavior in each trajectory together with environment feedback. The feedback serves as supporting evidence for identifying anomalies, their types, and the steps at which they occur. This allows us to obtain step-level annotations for naturally generated trajectories as well.

W2: Average latency does not establish runtime deadlines.

Thank you for this suggestion. We agree that mean latency alone is insufficient. To assess whether detection can finish in time, we use the interval between consecutive step-end timestamps of the monitored agent as the deadline: meeting this deadline means that detection finishes before the next step arrives. We combine this interval with the latency distribution of the detection level assigned to that step to calculate the probability of completing detection within the inter-step interval. Using the same set of action intervals, we compare TRAIL, ADR, and different SERAM configurations, reporting AP and detection latency below.

Method or routing budgetAPMean detection latencyp95p99Completion within the inter-step interval
TRAIL0.25913.7 s20.5 s33.8 s15.5%
ADR0.2599.87 s18.5 s28.32 s30.4%
SERAM (60/30/10, default)0.4202.36 s11.3 s21.3 s80.6%
SERAM (80/15/5)0.3950.91 s7.77 s15.4 s92.4%
SERAM (90/8/2)0.3940.52 s5.01 s11.3 s95.7%
SERAM (100/0/0)0.3470.002 s0.002 s0.002 s100%

The results show that default SERAM completes detection within the inter-step interval with a probability of 80.6%. Adjusting the target Screen/Review/Audit shares to 80/15/5 and 90/8/2 increases the on-time completion probability to 92.4% and 95.7%, respectively, and reduces mean latency to 0.91 s and 0.52 s; p95 and p99 also decrease. Thus, depending on the monitored agent's execution speed, we can adjust the budget to trade off detection accuracy and response time and better meet the required deadlines. We have added p95/p99 detection latencies and the probability of completing detection within the inter-step interval to the revised appendix, providing a fuller assessment of timeliness beyond mean latency.

AuthorsOpenReview ↗

Response to Reviewer Xs3j (Part 2)

W3: Most adaptation benefit appears to come from supervised estimator/router training.

Thank you for considering the contributions of the individual components. We agree that estimator-router training provides an important foundation for improved detection accuracy, while the additional AP gain from knowledge evolution is relatively modest. However, knowledge evolution also provides more suitable information for detection, thereby reducing detection cost and mean latency. To illustrate this contribution, the following table reports detection performance, detection cost, and mean latency together.

ConfigurationAPDetection LLM cost (USD)Mean detection latency
SERAM0.420$14.022.36 s
SERAM without knowledge evolution0.411$19.043.20 s
SERAM without self-evolution0.381$21.403.53 s

All three configurations use the same Screen/Review/Audit target shares (60/30/10). The realized fractions of steps sent to the LLM are 41.4%, 36.4%, and 26.6% for SERAM without self-evolution, SERAM with estimator-router training only, and full SERAM, respectively. By updating the behavior graph and retrieval models, knowledge evolution provides more suitable evidence for the estimator-router. Compared with estimator-router training alone, full SERAM reduces mean uncertainty from 0.195 to 0.159, allowing more steps to be handled directly by Screen and thereby reducing the realized fraction of steps sent to the LLM. Correspondingly, detection cost decreases from $19.04 to $14.02, and mean detection latency decreases from 3.20 s to 2.36 s. The component analysis in the main text identifies estimator-router training as an important contributor. Appendix A.5 further acknowledges that the additional AP gain from knowledge evolution is relatively modest, while its benefits are more pronounced in detection cost and mean latency.

W4: Baseline adaptation may affect comparative fairness.

Thank you for raising the concern about comparative fairness. We understand that restricting methods to a trajectory prefix may affect their detection performance. Our adaptations of TRAIL and ADR preserve their core analysis structures. ADR was originally designed for runtime detection, and we retain its two-tier detection structure; TRAIL is adapted from complete-trajectory analysis to step-by-step prefix-only detection. We explicitly state in the prompts that only the current step and preceding history are available and ask the methods to determine whether the current step is anomalous. Both methods use the same LLM and anomaly-type definitions as SERAM and are evaluated on the same test stream. We have added the adaptation details to Appendix B.3 and clarified that these comparisons concern the TRAIL and ADR implementations evaluated under the shared prefix-only protocol.

We also emphasize that SERAM's detection advantage does not depend on a larger LLM-call budget. With no LLM calls (100/0/0), SERAM achieves 0.347 AP at a mean detection latency of approximately 2 ms per step, outperforming TRAIL and ADR, which both achieve 0.259 AP with mean detection latencies of 13.7 s and 9.87 s, respectively (see W2). This shows that SERAM's underlying detection framework remains competitive even without LLM analysis.

Furthermore, SERAM's detection advantage does not arise solely from self-evolution and subsequently released labels. With self-evolution disabled, SERAM does not use subsequent feedback from the test stream to update itself, yet its AP remains 0.381, higher than 0.259 for both TRAIL and ADR. This further demonstrates that SERAM's underlying detection framework remains competitive even without updates using subsequent feedback.

AuthorsOpenReview ↗

Response to Reviewer Xs3j (Part 3)

W5: Repeated validation and schedule sensitivity need closer analysis.

Thank you for this valuable suggestion. In our experiments, feedback is used in the order “detect →\rightarrow release labels →\rightarrow update →\rightarrow detect the next batch,” with each batch's predictions recorded before its labels are released. Each label release provides new evolution samples and expands the cumulative validation pool. To further assess the stability of our method, we conducted runs with two additional random seeds to vary the data order. The results show consistent accuracy and latency across the three seeds and their corresponding data orders.

SeedAP without self-evolutionAP with self-evolutionAP gainMean detection latency
Original seed (42)0.3810.4200.0392.36 s
Additional seed 1 (43)0.3850.4230.0382.14 s
Additional seed 2 (44)0.3830.4190.0362.52 s
Mean ±\pm SD0.3830±0.00200.3830 \pm 0.00200.4207±0.00210.4207 \pm 0.00210.0377±0.00150.0377 \pm 0.00152.34±0.192.34 \pm 0.19 s

To assess the effect of validation-sample reuse, we added a four-cycle experiment with non-cumulative validation. The experiment retains the main experiment's initialization, test-stream order, four-cycle update schedule, and candidate acceptance criteria, using only the current batch's new validation set DvalD_{\mathrm{val}} in each cycle without reusing validation examples from previous batches. The results are shown below. Using only the current batch's validation set yields an AP of 0.416, slightly lower than 0.420 with the cumulative validation pool but still higher than 0.381 with self-evolution disabled. This shows that self-evolution can still improve detection performance without reusing validation examples from previous batches.

MethodAPF1Accepted updates
SERAM (cumulative validation pool)0.4200.3842
SERAM (self-evolution disabled)0.3810.3410
SERAM (current-batch validation set only)0.4160.3791

W6: Cost claims omit some material resources.

Thank you for raising the question of resource overhead. Our cost-saving claims concern reductions in LLM charges, and we report detection and evolution charges separately. Local computation and storage requirements are reported separately in the revised appendix and are not included in these dollar amounts.

Local resourceConfiguration or usage
HardwareXeon Silver 4214 and one NVIDIA RTX 6000
Runtime per update cycleAbout 13–15 min, including diagnosis, proposal, retraining, and validation
GPU time for the full test streamAbout 1 GPU-hour
Behavior graph1,560 nodes, about 1.7 MB for structure and text
Node-text vectorsAbout 9.58 MB
Model checkpointAbout 368 MB, including model parameters and optimizer state for resuming training

The following table reports LLM costs under different SERAM configurations.

Configuration or cost itemLLM cost (USD)
SERAM without self-evolution, detection$21.40
SERAM, detection$14.02
SERAM, evolution$0.33–0.44 per update cycle
SERAM (60/30/10), detection and evolution$15.52
SERAM (80/15/5), detection$5.62
SERAM (90/8/2), detection$3.19

The corresponding AP, latency, and on-time completion probabilities are given in W2. LLM charges with evolution are lower than those without evolution, and different budget configurations allow further reductions in LLM charges. These results further demonstrate the effectiveness of evolution and the adaptability of our method to different budgets.

AuthorsOpenReview ↗

Response to Reviewer Xs3j (Part 4)

Q1: What fraction of test anomalies has been independently human-verified, and how are injection or judge artifacts prevented from entering the detector input?

Thank you for this question. We independently audited 228 of the 2,230 originally anomalous test steps (10.2%) and 228 originally normal steps as controls. The human annotator confirmed 89.5% of the original anomalous labels and 92.5% of the original normal labels, agreeing with the original labels on 415 of the 456 audited steps (91.0%). Claude Opus 5.5 independently annotated the same sample and also agreed with the original labels on 415 steps (91.0%).

We randomly sampled steps within each domain, with proportional coverage across domains and a 1:1 balance between originally anomalous and normal steps. The human annotator was blinded to the original labels and reviewed each sampled step in trajectory order, using only the current step and preceding history. The annotation results are shown below.

Original labelHuman: normalHuman: anomalousClaude: normalClaude: anomalous
Normal (228)211 (92.54%)17 (7.46%)219 (96.05%)9 (3.95%)
Anomalous (228)24 (10.53%)204 (89.47%)32 (14.04%)196 (85.96%)

In designing the StreamAD annotation procedure, we store generation records and annotations separately from detector inputs to prevent generation and annotation information from leaking into detection. We also remove template-specific metadata, such as execution timestamps and network-environment fields. The detector receives only the current step and preceding history, excluding generation records, timestamps, and other such injection-related metadata. Nevertheless, we acknowledge that implicit injection cues may remain in step descriptions, and we have added this point to the limitations.

To assess the sensitivity of method comparisons to annotation differences, we recomputed AP for SERAM, TRAIL, and ADR on the same human-audited test subset using the human labels and Claude labels, respectively. The results are shown below, indicating that SERAM still performs better than the other baselines under the human-corrected labels.

MethodHuman-label APClaude-label AP
TRAIL0.7240.697
ADR0.6720.646
SERAM0.7960.776

Q2: What are the p95/p99 detector latencies and the fraction of steps whose detection finishes before that step's actual next-agent-action deadline?

Default SERAM has p95 and p99 detection latencies of 11.3 s and 21.3 s, respectively, and an 80.6% probability of completing detection within the monitored agent's inter-step interval. This probability is calculated using the interval between consecutive step-end timestamps as the deadline and the latency distribution of the corresponding detection level. Adjusting the routing budget to 80/15/5 or 90/8/2 increases the on-time completion probability to 92.4% and 95.7%, respectively. See W2 for the full comparison across configurations and baselines.

Q3: If only the estimator/router is fine-tuned on released labels, with no graph revision, how does its AP and cost compare with full SERAM under the same feedback budget?

In the supervised-training control, we keep the behavior graph and retrieval models fixed and train only the estimator-router, achieving 0.411 AP at a detection LLM cost of USD 19.04. Full SERAM achieves 0.420 AP at a detection LLM cost of USD 14.02; adding USD 1.50 in evolution LLM charges brings the total to USD 15.52, still USD 3.52 below the former's detection cost. Thus, knowledge evolution and estimator-router training work together to further improve AP while reducing LLM costs. See W3 and W6 for the component ablations and local resource requirements, respectively.

Q4: Are the same validation examples reused to accept multiple update candidates, and what safeguards prevent overfitting to this pool?

Yes. The validation pool accumulates across feedback batches, and individual examples are used in multiple candidate decisions. However, additional validation examples are added each time to guard against overfitting to particular examples. We also impose a comprehensive set of candidate-update criteria covering detection performance, the balance of corrected and regressed decisions, retrieval overlap, target-node rank, and score shifts. These five criteria jointly constrain detection and retrieval behavior, rather than accepting updates solely on the basis of improvement in a single metric. We have added a control experiment using only the current batch's validation set, achieving an AP of 0.416, slightly lower than 0.420 with the cumulative validation pool but still higher than 0.381 with self-evolution disabled. See W5 for the experimental setup and full results.

Comments on the paper

1 comment
Area_Chair_ZHQSOpenReview ↗

Please engage in the discussion

Reviewers: if you have not responded to the authors' reply, please read it and engage in the discussion. Thanks!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.