acceptodds
Under review as a conference paper at ICLR 2027

AgentXRay: Grounding Agent Safety in System Evidence

Abstract

Computer-use agents translate natural-language instructions into consequential operating-system actions, including file access, command execution, permission changes, and network communication. Existing safety evaluations primarily rely on model responses, UI trajectories, screenshots, or tool calls. Although these traces reveal task context and apparent intent, they do not reliably establish whether attempted actions succeeded or what effects occurred in the underlying system. We introduce AgentXRay, a joint-evidence framework that pairs agent trajectories with independently collected Linux audit log. To our knowledge, AgentXRay is the first framework to pair synchronized agent trajectories with kernel-observed OS provenance and human annotations. Our evaluation uses 53 controlled executions spanning deliberate user misuse, prompt injection, model misbehavior, and benign activity with human annotation. We normalize each paired trace into an Agent-System Evidence Graph (ASEG), which represents agent actions, audit events, processes, affected objects, and temporal-semantic associations across the two evidence layers. An evidence-grounded LLM judge uses these representations to assess safety and task completion. Our evaluation shows that this alignment strategy is effective for agent safety detection, with particularly clear gains on prompt-injection executions. More broadly, the results suggest that explicitly structuring and relating agent actions to observed system effects is more effective than simply concatenating the two evidence sources.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Agent Security, System Provenance

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_giRsOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a benchmark and cross-evidence framework that combines agent traces with OS-level provenance for computer-use-agent safety evaluation. It complements the existing method that mainly relies on the observation in the agent-level with system-level audit records such as process lineage, processes, files, and sockets. Evaluation results across 53 executions show that AgentXRay improves safety detection in the primary GPT-4.1 evaluation, with the largest improvement on prompt injection.

Critical strengths
  • The paper identifies an important limitation in existing AI-agent safety evaluations that agent trajectories, tool calls, and model responses do not necessarily reflect what actually happens in the underlying operating system. And the proposed framework addresses this discrepancy by combining agent-level and system-level evidence.
  • The paper presents a new evaluation benchmark consisting of 53 controlled executions across deliberate user misuse, prompt injection, model misbehavior, and benigh activities, each of which contains agent trajectories, OS audit traces, and independently produced human annotations.
Critical weaknesses
  • The main technical contribution of AgentXRay is to associate agent-level actions wth OS-level events through the proposed Agent-System Evidence Graph. However, both system-provenance graph construction and the integration of heterogeneous execution logs across agent-level and system-level have been investigated in prior literature. In particular, AgentSight (2025) [1] proposed to address the semantic gap between agent intentions and their system effects by using eBPF to associate kernel events with agent activities, which can be deployed for security monitoring, including prompt injection detection.
  • The effectiveness of the proposed ASEG representation appears largely confined to prompt-injection scenarios, where aligning agent trajectories with underlying system effects is particularly well suited to identifying deviations from legitimate user instructions. However, the method shows little to no improvement in other safety categories, such as deliberate user misuse and model misbehavior. This narrow scope of empirical gains raises concerns about the broader applicability and significance of the proposed approach. Without demonstrating benefits across a wider range of safety scenarios, ASEG appears more like a task-specific engineering solution for problems like prompt-injection detection than a broadly applicable methodological contribution to agent safety research.

[1] Zheng, Yusheng, et al. "Agentsight: System-level observability for ai agents using ebpf." Proceedings of the 4th Workshop on Practical Adoption Challenges of ML for Systems. 2025.

Questions for authors
  • Why does it have 30 unsafe and 23 safe cases among 53 executions (line 368)? The authors mention early in the paper that the benchmark contains 14 deliberate-user-misuse, 14 prompt-injection, 15 model-misbehavior, and only 10 benign executions.
  • The Threat Model section has repeated text (line 181-189).
1 comment

Thanks for your response, which resolves part of my concerns. I'm inclined to maintain my original score.

Reviewer_FWirOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper argues that computer-use agent safety should be judged from both agent trajectories and the OS-level effects those actions actually produce. It introduces AgentXRay, which pairs OSWorld agent trajectories with Linux audit logs and normalizes both into an Agent-System Evidence Graph (ASEG). The graph links agent actions to processes, audit events and affected objects through temporal windows and lexical association, and an LLM judge assesses it (Eq. 4–6). The benchmark contains 53 human-annotated executions by o4-mini and Claude Haiku 4.5, derived from OS-Harm tasks, with labels for overall safety, behavioral safety, system-effect safety and task completion. Validation compares agent-only, system-only, raw-concatenation and ASEG inputs with GPT-4.1 and Claude Sonnet 4.6 as judges, reporting precision, recall, F1, FPR and accuracy overall and per risk category.

Critical strengths
  1. The motivation is important: agent traces show intent, not realized OS effects. Pairing OSWorld trajectories with kernel audit logs is a sensible, uncommon combination (Sec. 3.2).
  2. ASEG is a reasonable, human-inspectable design: typed nodes, temporal-lexical association and relevance-ranked serialization (Sec. 3.3, Eq. 4–6, Fig. 3).
  3. Separate behavioral and system-effect labels (Eq. 3) support a useful finding: LLM judges track attempted behavior more than realized effects (Table 3).
Critical weaknesses
  1. The main claim rests on two executions. GPT-4.1 recall rises from 13/30 to 15/30 (Table 1), and A.3 confirms exactly 2 corrected cases. No significance tests or intervals are reported.
  2. The result reverses with another judge. Under Claude Sonnet 4.6, agent-only beats AgentXRay on overall F1 (81.48 vs. 80.00) and recall (Table 4).
  3. The key example does not support the thesis. One of the two corrections has no unsafe system outcome (A.3), so the trace alone could support it. AgentXRay also has the highest system-effect FPR, 21.05% (Table 3).
  4. The benchmark is too small. It has 53 executions, about 14 per category. Model-misbehavior F1 is identical (28.57%) for every representation (Table 2), and release is deferred.
  5. ASEG hyperparameters are missing and its components unablated. δ, c_base, λ, K, c_max and the token cap are unspecified. There is no non-LLM provenance baseline.
Questions for authors
  1. How many unsafe executions does each category contain? Do any comparison in Tables 1, 2 or 4 survive a paired significance test?
  2. For the 15 unsafe executions AgentXRay misses under GPT-4.1, is the relevant system evidence present in the serialized ASEG? If so, why is it not used?
  3. How does a deterministic provenance-rule baseline (sensitive reads, permission changes, outbound connections) perform on the system-effect labels?
Reviewer_zanaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a method to ground safety of computer-use agents in system evidence. The idea is to combine agent-level actions with system-level actions in a unified graph-based representation in order to help LLM judges accurately classify safe vs. unsafe behaviors. Three kinds of unsafe behaviors are studied: deliberate user misuse, prompt injection attacks, and model misbehavior. The accuracy of judges is shown to improve on a benchmark comprising 53 agent traces, especially for prompt injection attacks.

Critical strengths

The paper addresses the important problem of agent safety in the setting of computer use.

The proposed idea is simple and novel to my knowledge: instrumenting system-level actions and overlaying them temporally with agent-level actions in a unified graph-based representation.

Critical weaknesses

The empirical results are quite weak. The performance of LLM judges using the proposed representation does not seem to improve significantly over simpler representations (e.g. agent-level actions only). In particular, much of the improvement is limited to the category of prompt injection attacks.

A large-scale study using real-world agent traces would go a long way in convincing the reader of the method’s effectiveness.

There is ample headroom for the proposed method even for the current controlled study. The paper would be significantly strengthened by improving the method so that its performance comes closer to ground-truth, and providing a convincing analysis of any remaining failures.

It is also not clear how feasible it would be to deploy the method since it relies on OS-specific details for observing system-level actions. It would be helpful to discuss what assumptions would one have to make (the threat model was certainly useful in this regard), and how the method would be able to detect and prevent attacks in real-time, despite the additional overhead.

Questions for authors

N/A unless I am misunderstanding the significance of the empirical results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.