acceptodds
Under review as a conference paper at ICLR 2027

On the Effectiveness of Kernel-Level Evidence for Agent Security

Abstract

LLM agents are deployed into infrastructure that grants them broad host authority, yet existing agent-security benchmarks and defenses operate almost exclusively at the application telemetry layer: the served tool manifest, the user prompt, and the model’s messages. Some threats, however, smuggle malicious instructions and actions past the application boundary, leaving them invisible to that layer. In this work, we bridge that gap by pairing application-level agent telemetry with kernel-level syscall traces to present the first paired-evidence characterization of kernel-level versus application-layer signal for agent security. To quantify the value of the enhanced telemetry, we introduce Agent Cross-Layer Evidence (ACE), a paired-session corpus of 4,047 sessions and 17 threat models spanning six delivery-vector families and 14 of the 25 OWASP LLM and agentic threat categories, organized into 12 attack mechanics with per-mechanic characterization of where the most discriminative evidence lies. Across four distinct detector families, we find that kernel evidence is discriminative on its own and that composing it with application-layer evidence generally outperforms either single-layer view, revealing complementary signals that single-layer analyses can miss. We further demonstrate generalization to unseen attack families and transfer to an alternate agent runtime. Together, these findings establish the value of cross-layer evidence for agent security.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Agents, AI Agent Security, Security Benchmark, Kernel Tracing

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_aw4XOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

ACE systematically investigates whether LLM agent security detection should extend from application-level intent to OS-level execution, using paired data and cross-model experiments to characterize the discriminative power and complementarity of different evidence layers.

Critical strengths

The paper addresses an important and underexplored problem in agent security. Its paired-evidence benchmark and systematic evaluation across attack mechanics and detector families provide valuable empirical insights into the complementary roles of application- and kernel-level evidence.

Critical weaknesses

Although ACE covers diverse attack mechanisms, its evaluation primarily relies on hand-crafted attacks in controlled environments. Further validation in real-world deployments could strengthen the generalizability of its findings. Compared with prior execution-level security studies such as AgentSight, the contribution lies mainly in systematic empirical characterization, and its distinct value could be more clearly emphasized.

Questions for authors

Could the authors clarify how the findings generalize from hand-crafted attacks to naturally occurring attacks?

3 comments
AuthorsOpenReview ↗

Response to Reviewer aw4X for [C1, Q1]

[C1] "Although ACE covers diverse attack mechanisms, its evaluation primarily relies on hand-crafted attacks in controlled environments. Further validation in real-world deployments could strengthen the generalizability of its findings."

[Q1] "Could the authors clarify how the findings generalize from hand-crafted attacks to naturally occurring attacks?"

[Response]

We agree that evaluation in real-world deployments would be valuable. In ACE, we use controlled attacks executed through real agent and tool runtimes to compare application and kernel evidence across attack mechanics. Section 2.5 describes 31 real third-party MCP servers alongside built-in tools, spanning six delivery-vector families. This gives us paired observations from the same sessions, explicit execution checks, and benign workloads that exercise the same tool surfaces (Appendices E.5 and E.7), which support consistent comparisons of the evidence layers.

We test generalization in two ways. First, the seven-fold evaluation excludes each tested attack family from training and LLM demonstrations (Section 3.3). Cross-View exceeds App-View for all ten detectors, with every pointwise paired 95% interval above zero (Appendix H.4, Table 12). Second, ACE-XA replaces both the agent model and execution scaffold while conditioning detectors only on ACE. Three of four tested detector-family representatives retain mean AUROC above 0.75 (Appendix M, Table 27). These results support the usefulness of execution evidence across held-out attack families and the tested alternate runtime.

For naturally occurring incidents, a useful evaluation needs paired execution records, or a faithful reconstruction of the affected environment, together with comparable benign activity. Replaying an incident in a different harness can change tool behavior, filesystem access, and network interactions, which are the signals we measure. We would therefore need to verify that the reconstruction preserves the relevant execution behavior as well as the malicious objective.

Case studies of disclosed incidents would provide a useful complement. However, assembling a sufficiently diverse collection is challenging: each incident must be documented in enough detail for faithful reconstruction, and the collection must cover distinct attack mechanics to support broader conclusions. Selecting only the incidents that are readily available and reproducible could disproportionately represent certain attack types. Such case studies would establish performance on those specific incidents, while assessing generalization would require a dedicated effort to collect, reconstruct, and validate a broader set of attacks.

Our focus in this study is controlled coverage and explicit transfer tests. Evaluation on naturally occurring incidents, using paired telemetry or reconstructions with validated behavioral fidelity and comparable benign workloads, remains a valuable direction for future work. We will clarify the scope of our generalization evidence and this future direction in the final paper.

AuthorsOpenReview ↗

Response to Reviewer aw4X for [C2]

[C2] "Compared with prior execution-level security studies such as AgentSight, the contribution lies mainly in systematic empirical characterization, and its distinct value could be more clearly emphasized."

[Response]

We agree this should be better clarified and address this overlapping concern in our response to Reviewer TnAv’s Comment 1. Please refer to that response here.

Thank you for the detailed clarification. I appreciate the authors pointing out the specific empirical evidence for generalization. I also understand the practical difficulties of evaluating naturally occurring incidents. I am satisfied with this explanation.

Reviewer_RyySOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks whether records of computer activity reveal attacks that the agent’s conversation and tool descriptions miss. Without these records, apparently harmless messages may conceal harmful activity by the programs behind them. The authors collect different records from 4,047 runs and compare ten detection methods. They test attack types excluded from benchmark training, as well as a different agent setup performing familiar attacks. Combining the records gives the best separation score for all ten methods under familiar conditions and seven when attack types are withheld.

Critical strengths

The paper compares different records from matching runs, as described in Appendix H.4 and reported in Table 12. This makes the comparison more convincing: differences in the underlying examples are less likely to explain the benefit of adding computer-activity records.

Combining the records also helps a method that does not receive the actual addresses or file paths. Appendix E.4 describes this restriction, and the XGBoost row in Table 8 shows its score rising from 0.703 to 0.850 on withheld attack types. This measures separation between attacks and other runs, rather than the percentage classified correctly. The authors also examine alternative labels (Table 5), removed identifying details (Table 21) and uncertainty among related runs.

Critical weaknesses

Removing address and file-path details and training Llama again lowers its score from 0.922 to 0.759 (Table 21). However, this also removes useful information about destinations and paths, so the decrease alone does not prove memorization of benchmark clues. The transfer test retains parts of the original setup (Appendix M.1), and its Qwen evidence has empty user-request blocks for all 592 records (Appendix M.2). These details leave uncertainty about how broadly the results transfer.

A run receives an attack label when predefined checks find the expected evidence. Some runs involving attack variants are counted as negative; others are excluded from scoring (Appendix E.3). How these decisions agree with an independent assessment of the actions and permissions matters to the interpretation of the results.

Questions for authors

Given the shared setup described in Appendix M.1, what evidence shows that the conclusions hold when those details change while actions and permissions remain comparable?

4 comments
AuthorsOpenReview ↗

Response to Reviewer RyyS for [C1]

[C1] "Removing address and file-path details and training Llama again lowers its score from 0.922 to 0.759 (Table 21). However, this also removes useful information about destinations and paths, so the decrease alone does not prove memorization of benchmark clues. ... These details leave uncertainty about how broadly the results transfer."

[Response]

We agree with the interpretation here. Removing this information reduces performance, but useful detection remains after the model learns from the redacted representation. The decrease alone cannot tell us whether the removed information helped through meaningful security context or memorized identifiers. However, Llama retains useful detection performance when literal IP addresses and file paths are redacted from both its training and evaluation summaries. Across seven held-out attack families, it achieves 0.759 AUROC with a 95% cluster-bootstrap interval of [0.686, 0.836], well above chance (0.5). This shows that the remaining evidence supports distinguishing attacks from benign activity even for attack families excluded from training. Detection therefore does not depend entirely on recognizing specific addresses or pathnames.

XGBoost provides complementary evidence that structural execution information supports detection without access to literal IP addresses or pathnames. Its kernel representation contains numerical features such as syscall counts, rates, and path/IP cardinalities (Appendix E.4). On held-out attack families, XGBoost achieves AUROCs of 0.703 with App-View and 0.750 with Kernel-View. Combining both evidence sources (Cross-View) further improves AUROC to 0.850. Because the literal addresses and pathnames are absent from its inputs, these results cannot arise from directly recognizing those identifiers. They demonstrate that structural execution features carry useful detection signal beyond identifier identity.

Together with the redacted Llama result, this supports the continued usefulness of kernel evidence when identifier information is restricted. We will clarify this interpretation in the final paper.

AuthorsOpenReview ↗

Response to Reviewer RyyS for [C2, Q1]

[C2] "The transfer test retains parts of the original setup (Appendix M.1)... These details leave uncertainty about how broadly the results transfer."

[Q1] "Given the shared setup described in Appendix M.1, what evidence shows that the conclusions hold when those details change while actions and permissions remain comparable?"

[Response]

We appreciate the reviewer's question regarding the scope of our cross-runtime generalization evaluation. ACE-XA tests whether useful detection extends beyond the agent implementation used to construct ACE. We keep the attack objectives and capture protocol comparable so that the detection target and evidence definitions remain consistent across runtimes. Under this design, detectors conditioned only on ACE must recognize the same types of attacks when a different agent and scaffold execute them.

This change produces observable differences in execution behavior. ACE-XA replaces Claude Code/Haiku with Devstral 2 in a Python ReAct scaffold and a separate MCP client (Appendix M.1). Tool dispatch and intermediate-result handling also change. All ACE sessions touch temporary paths, whereas 57% of ACE-XA sessions do not because the alternate scaffold processes intermediate results in memory (Appendix M.4). The transfer evaluation therefore changes the execution evidence on which detection depends.

Useful detection persists across this change without fitting or selecting detectors on ACE-XA. All training, fine-tuning, reference selection, and checkpoint selection use ACE alone (Appendix M.2). XGBoost Cross-View achieves mean AUROC 0.842 and Llama Kernel-View achieves 0.849 (Table 27). Our updated Qwen Cross-View evaluation, with user requests included, achieves 0.969, as detailed in our response to C3. Three of the four tested detector-family representatives thus retain mean AUROC above 0.75 in the alternate runtime.

The attack-family and identifier controls provide complementary evidence. Cross-View exceeds App-View for all ten detectors when entire attack families are excluded from training and demonstrations, with positive pointwise paired 95% intervals (Appendix H.4). Numerical detectors achieve useful discrimination without literal IP addresses or pathnames (Appendix E.4), and Llama retains 0.759 AUROC under the identifier-redaction protocol (Appendix K.5). These findings address dependence on familiar attack families and identifier information separately from the runtime comparison.

The shared capture protocol makes the runtime comparison interpretable, but also defines its scope: ACE-XA establishes transfer across the tested agent implementations while retaining the sinkhole and sentinel conventions. It does not independently test changes to those conventions. Our findings therefore support the usefulness of kernel evidence beyond the original agent and across held-out attack families, while broader transfer across capture environments remains future work. We will make this relationship between the experimental design, supported conclusions, and remaining transfer question explicit in the revised paper.

AuthorsOpenReview ↗

Response to Reviewer RyyS for [C3]

[C3] "... its Qwen evidence has empty user-request blocks for all 592 records (Appendix M.2). These details leave uncertainty about how broadly the results transfer."

[Response]

In response to this concern, we revisited the original ACE-XA capture records, recovered the initial user requests from the capture metadata, and reconstructed the detector inputs to include them. We then conducted a matched evaluation of Qwen3-Next-80B-A3B with and without those requests on the same 577 eligible sessions from the 592-session corpus, excluding the 15 signal-bearing latent sessions under the original scoring policy (Appendix M.1; Table 27), holding the reference examples, detector instructions, labels, folds, and decoding settings fixed.

With user requests included, mean AUROC is 0.969, compared with 0.996 without them. Per-fold AUROC with restored requests ranges from 0.904 to 1.000. Although including the requests reduces the mean AUROC, Qwen retains strong discrimination across all five evaluated attack families in the alternate agent runtime.

We will update Table 27 and the corresponding main-text results to report the evaluation with the user requests included, yielding mean AUROC of 0.969. We will also retain the matched comparison without requests in Appendix M as a sensitivity analysis and document the updated inference configuration.

AuthorsOpenReview ↗

Response to Reviewer RyyS for [C4]

[C4] "A run receives an attack label when predefined checks find the expected evidence. Some runs involving attack variants are counted as negative; others are excluded from scoring (Appendix E.3). How these decisions agree with an independent assessment of the actions and permissions matters to the interpretation of the results."

[Response]

We appreciate the reviewer raising this important distinction. ACE's labels represent specified attack effects under the threat models in Appendix E.1. A positive label requires recorded evidence satisfying the corresponding attack-specific criterion, such as a credential-access operation or an attacker-directed file-write operation. An unmatched predicate does not establish benignness: an attack may stop before the specified effect, execute partially, or produce behavior outside that criterion. Our protocol therefore distinguishes these outcomes explicitly.

The corpus-wide counts reported in Appendix E.3 are summarized below:

 

Session categoryCountTreatment
Malicious variant with a matched fire predicate1,358Positive
Unmodified benign run1,225Negative
Dormant: malicious variant present, target tool unexercised1,002Negative
Signal-bearing latent: target tool exercised, predicate unmatched396Excluded from scoring
Signature-free latent controls66Negative controls outside A–G OOD scoring

 

The 396 signal-bearing latent sessions are intentionally excluded from the primary evaluation to avoid assigning an unsupported positive or negative label. Dormant negatives reflect the absence of target-tool exercise. The signature-free controls represent the sensor-scope cases described in Appendix B and are separate from unmodified benign sessions.

To characterize the ambiguous cases, we examined their retained transcripts. We observed cases in which agents encountered WebFetch errors without invoking Bash, stopped after raising a security concern, ran into dependency-setup errors before the specified write, or summarized injected instructions without performing the requested credential access. These observations illustrate why engagement with a malicious artifact need not produce the specified execution effect, while still allowing influence on the agent's response. They do not establish that every predicate miss is benign.

Appendix E.4, Table 5 already tests the consequences of including these ambiguous cases as attacks. On the A–G OOD folds, the broader FL policy treats all 356 signal-bearing latent sessions within those folds as positives, keeping the trained detectors and their predictions fixed. Across 21 detector/view combinations, AUROC changes by −0.045-0.045 to +0.010+0.010, and Cross-View remains above App-View for all seven trained detectors. Thus, the benefit of execution evidence persists under this broader positive-class definition.

Independent assessment could provide additional validation of the correspondence between our operational labels and the security meaning of observed actions, including effects outside the predefined predicates. However, developing a reliable semantic adjudication procedure across attack mechanisms is a substantial research undertaking: it requires explicit authorization criteria, consistent treatment of partial execution and unintended effects, and validation of the adjudicator itself. In fact, entire studies, including PentestJudge and AgentRewardBench, are devoted specifically to developing and validating methods for assessing agent behavior. We therefore leave this broader assessment to a dedicated future study. Our current evaluation uses explicit attack-effect criteria, excludes unresolved cases from primary scoring, and measures sensitivity to broader label policies. We will clarify this scope and incorporate the additional characterization of ambiguous sessions in the revised paper.

Reviewer_TnAvOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses the problem that existing LLM Agent security detection methods mainly rely on application-layer information and struggle to identify hidden malicious behaviors. It proposes a cross-layer security detection evaluation framework that combines application-layer and kernel-level evidence. The authors construct the ACE Benchmark, which contains 4,047 sessions, 17 threat models, and 12 attack mechanics, and systematically evaluate the detection performance of different evidence views using four detector families. The experimental results show that kernel-level evidence has independent attack detection capability, and combining application-layer and kernel-level information generally further improves detection performance, while demonstrating a certain degree of generalization to unseen attack categories and different Agent runtime environments.

Critical strengths

1.The paper considers three evidence views, App-View, Kernel-View, and Cross-View, and uses multiple detection methods to systematically compare the detection performance of evidence from different layers.

2.The ACE Benchmark constructed by the authors covers various attack entry points and attack behaviors. Meanwhile, the dataset collects paired application-layer and kernel-level runtime evidence, providing a valuable evaluation basis for studying the detection capabilities and complementarity of different evidence sources.

Critical weaknesses

1.The paper has already discussed related work on Agent security analysis based on application-layer and kernel-level evidence in the Related Work section. Compared with these existing studies, what new research findings does this paper provide, apart from constructing a larger-scale dataset and systematically comparing the detection performance of different evidence views?

2.Although the authors compare the performance of App-View, Kernel-View, and Cross-View across different attack mechanics and discuss some of the results, the current analysis mainly focuses on differences in detection performance and their intuitive explanations. It does not analyze why different attack categories have different dependencies on application-layer and kernel-level evidence. Which specific features lead to these differences?

3.The authors construct a detection representation containing 57 kernel-level features and 29 application-layer features. Why are so many features needed, and do all of these features make substantial contributions to the final detection performance? Could the difference in the number of features also affect the subsequent detection results?

4.The authors use predefined Fire Predicates to determine whether malicious attacks are successfully triggered and construct malicious labels for sessions accordingly. However, some Fire Predicates themselves rely on kernel-level system call information. Could the detection advantage of Kernel-View be influenced to some extent by the label construction process, rather than solely resulting from the stronger discriminative capability of kernel-level evidence itself?

Questions for authors

None

9 comments
AuthorsOpenReview ↗

Response [1/2] to Reviewer TnAv for [C1]

[C1] "The paper has already discussed related work on Agent security analysis based on application-layer and kernel-level evidence in the Related Work section. Compared with these existing studies, what new research findings does this paper provide, apart from constructing a larger-scale dataset and systematically comparing the detection performance of different evidence views?"

[Response]

We agree that ACE's distinct value should be more clearly emphasized. Rather than proposing another monitoring or enforcement system, ACE measures the independent and combined detection value of application- and kernel-layer evidence across attack mechanics and held-out attack families. In doing so, ACE provides both a reusable experimental foundation and empirical findings about the complementary roles of these evidence layers. Below, we distinguish our contributions from prior OS-level agent-security work and summarize the key findings.

AgentSight and ActPlane are the two closest works because both bring the OS boundary into agent security. ActPlane enables agents to generate policies that an OS-level engine enforces, with feedback to improve compliance. Its objective is policy enforcement. We study what evidence allows detectors to recognize attack effects in completed sessions. AgentSight, enables boundary tracing and LLM-based analysis of correlated application and kernel observations. Its three effectiveness case studies cover indirect prompt injection, reasoning-loop detection, and multi-agent coordination. The security case shows that a combined monitor can detect a particular injected attack. Our study asks the next set of questions: whether kernel evidence is independently useful for detection, how much it adds to application evidence, which attack mechanics benefit from each layer, and whether detection generalizes to held-out attack families.

Answering these questions systematically requires a common experimental foundation that captures both evidence layers across diverse attack mechanics. ACE provides this foundation through its paired-evidence benchmark and execution-mechanic taxonomy, which are substantial contributions in their own right. Prior benchmarks primarily capture application-layer attacks, while existing execution-level security systems do not provide the paired datasets needed to systematically compare application- and kernel-level evidence (Appendix A).

To our knowledge, ACE is the first large-scale paired application--kernel benchmark for agent security and provides the broadest coverage of execution-level attack mechanics among the benchmarks surveyed in Table 1. It comprises 4,047 executed sessions spanning 17 threat models and six delivery-vector families, including attacks involving 31 real third-party MCP servers and built-in agent tools. Each session pairs syscall traces, served tool manifests, and agent transcripts, enabling direct comparisons of evidence layers on identical executions with matched benign controls.

Beyond its scale, ACE introduces a taxonomy of 12 sensor-observable execution mechanics that organizes attacks by their underlying behavior rather than delivery vector (Section 2.6). Together, ACE's paired-evidence design, broad attack coverage, and execution-grounded taxonomy position it as a reference benchmark for developing and systematically evaluating cross-layer agent-security detectors. We will release the corpus, featurizer, detector artifacts, and evaluation protocols upon acceptance.

AuthorsOpenReview ↗

Response [2/2] to Reviewer TnAv for [C1]

In addition, the study yields four findings that, to our knowledge, no other study establishes:

  1. Kernel observations provide standalone detection signal. Under attack-family holdout, kernel-only Llama reaches 0.922 AUROC, while XGBoost reaches 0.750 using numerical kernel features without literal IP addresses or pathnames (Table 8; Appendix E.4). This establishes a role for execution evidence beyond providing context for the application transcript.

  2. The strongest evidence layer changes with the attack mechanic. Silent exfiltration through logging favors Kernel-View, whereas retrieval reference injection favors App-View. Preference manipulation reaches 0.834 AUROC with combined evidence, compared with 0.687 App-only and 0.701 Kernel-only. These mean in-distribution results across ten detectors show complementary coverage and a concrete benefit of combining the layers (Table 2).

  3. The benefit extends to attack families absent from training. Cross-View exceeds App-View for all ten detectors, with positive pointwise paired 95% intervals (Appendix H.4). Seven detectors achieve their highest AUROC with Cross-View and three with Kernel-View. We therefore find both a generalizable contribution from kernel evidence and a benefit from combining the layers that depends on the detector.

  4. Evidence representation materially affects detection. BERT on aggregated kernel summaries reaches 0.836 OOD AUROC, versus 0.517--0.578 for the tested raw-stream alternatives (Appendix I.4). Compact summaries also outperform chronological raw traces on average across three frozen judges (Appendix J.2). These results offer guidance on how to represent execution evidence for detection.

We will make these contributions and their relationship to prior work clearer in the Introduction and related-work discussion of the final paper.

AuthorsOpenReview ↗

Response [1/4] to Reviewer TnAv for [C2]

[C2] "Although the authors compare the performance of App-View, Kernel-View, and Cross-View across different attack mechanics and discuss some of the results, the current analysis mainly focuses on differences in detection performance and their intuitive explanations. It does not analyze why different attack categories have different dependencies on application-layer and kernel-level evidence. Which specific features lead to these differences?"

[Response]

We agree the connection between attack mechanics and specific features could be made clearer. To make these mechanisms concrete, we examine three paired attack/benign cases and report per-detector view comparisons across all twelve mechanics. The cases illustrate which specific features capture the relevant observations, while the comparisons show whether the view-level patterns are consistent across detector families.

Each case uses an attack and an unpatched benign session with the same server or built-in task category and prompt. We selected them to illustrate internal tool activity, manipulation of returned content, and execution amplification within a tool call.

Case S: Silent logging changes execution evidence while leaving the application input unchanged. In a memory-bank task, the malicious handler records the requested project and filename in an internal log. The attack and benign session have identical tool calls, arguments, and results. Their rendered App-View inputs and all 29 numeric application features are also identical. The raw trace nevertheless shows the malicious handler opening an additional file for append, increasing the distinct temporary-path count from 1 to 2 (Appendix G.1).

This numeric feature records the additional filesystem footprint without receiving the filename itself. Language-model detectors instead receive the observed path in the kernel summary's temporary-path list. The paired case illustrates how execution evidence can expose an internal operation absent from the recorded application input.

Case R: Retrieval injection changes the content returned to the agent. Both sessions in a document-loader task read the same spreadsheet. The attack prepends a purported system annotation instructing the agent to fetch an external verification endpoint before returning the document. The original spreadsheet content remains intact after the injected prefix.

The result system-style marker count rises from 0 to 1, as does the excess of result markers over served descriptions. The maximum result-preview length rises from 522 to 809 characters. These are specific features in Appendix G.2. Language-model detectors receive the instruction text directly in App-View, while the kernel summary preserves execution statistics without retaining the returned instruction. The case connects the injected content to both numeric application features and the text available to language-model detectors.

Case E: A small number of tool calls can generate substantial execution activity. In a health-probe task, the attack replaces a workspace document with instructions to run a continuous loop invoking stat on a process-status file. The agent reads the document and starts the loop through Bash. The benign session reads the unmodified document and searches the workspace for the actual runbook. Both sessions execute tools, but the attack makes fewer calls while generating much more work:

MeasurementBenignAttack
Tool calls42
Total recorded syscalls1,957719,059
Program-execution calls (execve)2023,887
File-open calls (openat)531239,598

Repeated stat executions in the raw trace link these measurements to the injected loop. App-View preserves the instruction and shell command for language-model detectors, while its numeric features summarize the tool trajectory and argument sizes. Kernel-View exposes work inside the Bash invocation through total syscall count and execution/open counts, with log-counts and rates also available to numeric detectors (Appendix G.1). The case shows why tool-call count alone does not measure execution intensity.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

We sincerely thank the reviewers for their careful evaluation and constructive feedback. We appreciate their recognition of the importance of studying agent security beyond application-layer observations, as well as the value of ACE's paired-evidence benchmark and systematic evaluation across attack mechanics and detector families. We are also grateful for the attention given to our experimental methodology, particularly the matched-session comparisons and robustness analyses. These elements are central to our goal of understanding when and to what extent kernel-level evidence provides complementary security signals beyond those available at the application layer.

The comments have helped us identify where we can explain the contribution, experimental choices, and scope of the findings more clearly. For convenience, we will address each reviewer's concerns individually in direct replies below and would be happy to answer any further questions or discuss any points that remain unclear.

NOTE: We will use the following abbreviations to refer to comments. Weaknesses will be abbreviated W# and questions will be abbreviated Q#. For example:

Weakness 1 -> W1

Question 1 -> Q1

AuthorsOpenReview ↗

Thank You and Reminder for Final Feedback

As the discussion period comes to a close at the end of the day tomorrow, we would like to thank all reviewers for their time and thoughtful feedback.

We welcome any further comments, questions, or concerns regarding our responses, and would be happy to address them before the discussion period closes.

Thank you again for your time and consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.