acceptodds
Under review as a conference paper at ICLR 2027

ProtoArena: Can AI Agents Discover and Exploit Real-World Cross-Component Protocol Vulnerabilities?

Abstract

Large language model (LLM) agents are increasingly used for vulnerability analysis and automated security testing. Existing cybersecurity benchmarks primarily evaluate capture-the-flag challenges, source-assisted vulnerability reproduction, and exploitation of vulnerabilities within software systems. However, many protocol vulnerabilities arise from inconsistencies in message interpretation, timing, state, or trust assumptions across multiple components. Existing benchmarks provide limited coverage of whether agents can infer and exploit such interactions through black-box observation. We introduce ProtoArena, a black-box, multi-component, dynamic benchmark comprising 80 cross-component vulnerability-exploitation environments across 11 protocol and application domains and seven attack categories. Its tasks are primarily informed by published protocol-attack mechanisms. Agents interact with running systems through scoped interfaces to achieve observable security effects without access to target source code. Three information levels assess guided reproduction, exploitation with mechanism and topology context, and effect-directed black-box exploration. A hybrid verifier combines deterministic checks with LLM review of independently collected traffic and logs. We evaluate five LLMs on ProtoArena. Claude Opus 4.8 achieves the highest observed success rates, reaching 48.8% under guided reproduction and 12.5% under effect-directed black-box exploration. In a separate DNSBomb optimization experiment, it achieves a peak downstream-response to average query-input byte-rate ratio exceeding . These results demonstrate verifiable exploitation in a subset of the evaluated environments, while revealing substantial limitations under reduced guidance and variation across protocol domains and attack categories. All experiments use controlled, explicitly authorized targets and publicly documented attack mechanisms; they neither test production systems nor disclose previously unknown vulnerabilities.

Primary area:
datasets and benchmarks
Keywords:
LLM agents, cybersecurity benchmarks, network protocol security, cross-component vulnerabilities

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_UKkFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces ProtoArena, a framework for evaluating whether LLM agents can generate observable security effects by interacting with multiple protocol participants, without access to the target source code. The authors constructed 80 environments covering 11 protocol or application domains and seven categories of attacks; the tasks are primarily derived from known attack mechanisms. The evaluation employs three information levels: L1 provides the mechanism, topology, and step-by-step guidance; L2 retains the mechanism and topology but omits the steps; and L3 provides the scope of access, constraints, and target effects. Success determination combines deterministic checks with LLM-based review of execution evidence. The paper evaluates five models using OpenHands, retaining a single result for each model-environment-information level combination. Table 3 reports that Claude Opus achieved the highest success rates: 48.8% for L1, 35.0% for L2, and 12.5% ​​for L3. Additionally, optimization experiments for DNSBomb under known mechanisms report a peak downstream response rate up to approximately 171,268 times the average query input rate, alongside a case study of out-of-domain cache poisoning. The primary contributions of this paper are the construction of the benchmark and the design of the evaluation. The evidence demonstrates that some models can produce the specified effects within the constructed environments; however, it is insufficient to establish a stable ranking of models or to prove that models can generally infer vulnerability mechanisms from black-box observations. The paper explicitly acknowledges this latter limitation.

Critical strengths
  1. The definition of the evaluation target holds clear scientific value. Section 3.1 emphasizes that merely deploying multiple services does not constitute a cross-component vulnerability task; the intended effect must rely on security-relevant interactions between components. DNSBomb—leveraging timing dependencies and discrepancies in HTTP request boundary interpretation—serves as a concrete example. This provides the benchmark with a clearly defined research object and establishes a sound direction for evaluating interaction capabilities beyond mere source code analysis.

  2. The criteria for determining success prioritize observable effects and evidentiary sources. Sections 3.3 and 4.2, along with Appendices A and E, mandate deterministic verification, independent evidence, and mechanism alignment; they reject agent self-reporting as sufficient proof and prohibit LLMs from reclassifying runs that failed initial thresholds as successful. These principles are crucial for security agent benchmarks. Furthermore, the paper explicitly limits the scope of interpretation regarding DNSBomb rates and acknowledges the limitations inherent in single-instance results and existing mechanisms, thereby avoiding the pitfall of framing observations as broad conclusions about general capabilities.

Critical weaknesses
  1. Appendix E claims that the final verdicts of the two LLM judges align with expert verification; however, it fails to report the number of verification samples, the selection method, whether experts made independent judgments, or whether initial "harness" failure cases were reviewed. Consequently, this claim does not adequately substantiate the judges' overall accuracy, particularly regarding the assessment of false negatives. Without addressing false negatives, it is difficult to determine the correctness and validity of the experimental results. Section 3.2 requires the submission of an executable PoC that achieves the desired effect within the specified budget; yet, Step 8 in Figure 5 indicates that the conversation reached the 60-turn limit and that the "Harness selects the live-session result." The figure does not demonstrate a successful replay of the submitted PoC from a clean state. Achieving an effect during interaction is a different evaluation criterion from delivering a reproducible exploit program; the current materials do not explain how this case satisfies both requirements, nor do they clarify its relationship to the main results matrix. This aspect is unclear to me.

  2. The measurements for DNSBomb also require verification. Appendix D retains only the last 20,000 lines of the log increment, using this subset to calculate query input volume, time span, and peak response rates. If early query records were truncated, the denominator and time span values ​​could be altered. The paper does not report whether truncation occurred for specific models or if recalculations based on full logs would yield consistent results. Figure 2 illustrates the process from query logs to pulse statistics—noting that query logs record only queries—whereas Appendix D relies on downstream response ([out]) records; the implementation details regarding data collection for both are not clearly explained. These questions directly concern the "success" reported in the paper and the prominently displayed attack intensity figures. While they do not prove the results incorrect, they render the current evidence insufficient to fully substantiate the reliability of the measurements.

  3. Section 3.4 states that the instances, targets, judges, and budget remain unchanged, and that L3 continues to provide the permitted operations. However, while levels L1 and L2 in Appendix B explicitly state that the agent can run an authoritative service at 127.0.0.1:53535 and describe the forwarding relationship for agent.lab, the L3 example specifies only the target service port without detailing the corresponding control permissions. While obscuring protocol mechanisms and network topology may be a reasonable way to set difficulty levels, the scope of agent control and the methods for discovering these interfaces within authorized limits must be clearly defined. Otherwise, a drop in success rates might reflect a lack of information regarding legitimate operational parameters, rather than merely the difficulty of crafting an exploit based on limited observations. Page 10 notes that the agent can read /task and /opt/scenario.json but does not show the content of these inputs across the different levels, making it impossible to verify whether information is missing or inadvertently exposed. Furthermore, L1 and L2 define a "qualifying pulse" as a one-second interval meeting a specific threshold, whereas L3 requires "distinct pulses." If the same detection logic is used for all three, the handling of this terminological discrepancy needs to be clarified. For a benchmark where information levels serve as the primary experimental variable, these are not merely peripheral implementation details; they are fundamental to ensuring fair comparisons and interpretable results.

Questions for authors
  1. Is success contingent upon PoC replay? Please clarify whether the "live-session success" shown in Figure 5 is reflected in Tables 3–5, and whether the finally submitted PoC was subsequently executed from a clean state to reproduce the effect. Is the same success criterion applied across all tasks?

  2. What is the evidence for the consistency between the verifiers and the experts? Please provide the number of cases fed into each verification stage, the scope and independence of expert verification, and the inspection results regarding cases where the harness failed. Does the consistency between the two judges and the experts refer to their initial independent assessments or their consensus following final adjudication?

  3. Are the complete inputs and privileges comparable across the three information levels? Please specify what information is disclosed by the Prompt, /task, /opt/scenario.json, and other readable files, respectively. How does DNSBomb L3 inform the agent of—or allow it to discover—control privileges over the authoritative service? Do the "one-second interval" and "distinct pulses" in the main task correspond to the same judgment criterion?

  4. Are the DNSBomb results robust against log truncation and data collection methods? For each of the five models, please provide the total query bytes, time span, peak downstream bytes, and the number of truncated logs; specify where the [out] records are generated; and present the results recalculated using complete records or independent traffic evidence.

  5. Can you provide task-by-task records supporting the overall validity of the benchmark? The current PDF primarily presents aggregate counts by category. Please provide details for the 80 environments—including source mechanisms, software versions, key configurations, attacker privileges, success conditions, and summaries of expert PoC verification—and explain the rules regarding re-run limits, actual re-run counts, and result retention for instances involving tool errors, environment failures, or incomplete evidence.

Reviewer_96mpOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces ProtoArena, a black-box, multi-component, dynamic benchmark comprising 80 cross-component vulnerability-exploitation environments across 11 protocol and application domains and seven attack categories. It evaluates agent capabilities under three information levels that provide different amounts of attack-related guidance. Five LLMs are evaluated using the OpenHands agent framework, with Claude Opus 4.8 achieving the highest overall success rate at each information level.

Critical strengths
  • A distinctive benchmark focused on evaluating agents’ security testing capabilities in black-box, multi-component network environments. This focus provides a useful setting for assessing attacks that depend on interactions among protocol components.
Critical weaknesses
  • Insufficient motivation for the real-world significance of combining black-box access with multi-component protocol testing.

Although this setting is underexplored in existing benchmarks, identifying a coverage gap alone does not establish its practical importance. The motivation needs stronger evidence connecting the chosen information and access constraints to representative real-world security risks.

  • Insufficient evidence supporting the reliability and informativeness of the benchmark results. The artifact is not currently publicly available, limiting independent validation.

Specifically: (i) although Appendix E reports agreement between the LLM judges and expert verification, the number and selection of expert-verified cases, the expert-review protocol, and the resulting error statistics are insufficiently documented; (ii) differences among environments within the same protocol domain are insufficiently documented, making substantive coverage difficult to assess; (iii) only one outcome is retained per model–environment–information-level combination, without repeated-run estimates of variability or ranking stability; and (iv) Sections 5.1–5.3 mainly describe success rates and rankings without systematically explaining performance differences, failure modes, or practical implications.

Questions for authors
  1. What real-world deployments or documented incidents demonstrate the severity and practical importance of cross-component vulnerabilities under the benchmark’s black-box access assumptions?

  2. How was the reliability of success/failure judgments validated? What cases were independently checked by experts, and what false-positive and false-negative rates were observed?

  3. How do environments within the same protocol domain differ in attack mechanism, implementation, topology, or configuration, and how do these differences contribute substantive coverage?

  4. Can repeated evaluations under identical task configurations and budgets quantify the variability in success rates and the stability of model rankings?

  5. What recurring failure modes or actionable lessons emerge from the results beyond aggregate success rates and rankings?

Reviewer_EJS1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces ProtoArena, a black-box benchmark for testing whether LLM agents can exploit network protocol vulnerabilities that arise from interactions between components, such as HTTP desynchronization, DNSBomb and SMTP smuggling. It contains 80 controlled environments built from published attack mechanisms, spanning 11 protocol and application domains (DNS, TLS/PKI, email, HTTP, CDN, SSO, BGP, among others) and 7 attack categories. Success is judged from traffic and service logs collected outside the agent’s workspace. Deterministic rule checks come first, followed by an LLM review stage.

Critical strengths

The motivation and the pointed-out gap is real and interesting. Existing cyber benchmarks mostly test CTFs, protocol attacks that depend on how several participants interact (HTTP desync, DNSBomb, SMTP smuggling) are not well covered, and present significant technical challenges.

Success is defined by effects, using evidence the agent cannot write to. The agent’s own claims (attack plan, reported score, PoC output) are explicitly not enough to count as success. For exploitation tasks, where agents could easily report success they did not achieve, this is the right principle.

Critical weaknesses

The evaluation is somewhat limited. There is one run per model–environment–level, with no repetition. This cannot rule out internal randomness of the language model.

The paper does not provide sufficient information for the benchmark. The 80 environments appear only as counts in Table 2. There is no per-task list of source attacks, or reference PoCs, and no description of how expert validation was done. A reader cannot judge task quality or difficulty.

Questions for authors

NA

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.