acceptodds
Under review as a conference paper at ICLR 2027

Automedon: Taking the Reins of Web Agents through Full-Adaptive Manipulation

Abstract

Large language model-based web agents increasingly execute user tasks through autonomous browser interactions, but their reliance on webpage content exposes them to prompt injection. Existing attacks are zero-adaptive or partial-adaptive. They fix payloads in advance or adapt only to limited observed states, and thus fail under the execution dynamics of long-horizon tasks. We present Automedon, the first full-adaptive manipulation framework for black-box web agents. Automedon builds on two insights. (i) Step-to-step webpage transitions form an observable footprint of the agent's decision-making, and (ii) attacker-controlled browser extensions provide a persistent attack surface for modifying what the agent observes. To steer an agent through a long-horizon malicious task, Automedon coordinates a Planner, a Generator, and an Evaluator for explore-exploit planning, state-conditioned payload generation, and transition-based verification, respectively. Across four datasets, Automedon achieves an average ASR of 0.99, outperforming seven baselines while preserving benign task accuracy within 0.04 of the optimal value. Experimental results also demonstrate that it remains effective across different backend LLMs, observation functions, agent frameworks, and countermeasures. Our code is available at https://anonymous.4open.science/r/Automedon.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Web Agent Security, Large Language Model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_VSNNOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Automedon, a full-adaptive prompt injection framework that manipulates black-box web agents into executing multi-step malicious tasks. Unlike existing static or partially adaptive attacks, Automedon leverages browser extensions to continuously monitor webpage transitions and dynamically adjust malicious payloads. It consists of three components: a Planner for selecting intermediate attack goals, a Generator for producing context-aware payloads, and an Evaluator for verifying attack progress. Experiments across four datasets demonstrate an average 99% attack success rate, significantly outperforming existing baselines while largely preserving benign task performance. The framework also shows strong effectiveness across different LLMs, agent architectures, and defense mechanisms.

Critical strengths
  1. The paper introduces a full-adaptive prompt injection framework that continuously adjusts attack strategies based on webpage state transitions. This addresses the limitations of existing static and partially adaptive attacks, particularly in long-horizon web agent tasks.

  2. The proposed method achieves an impressive average attack success rate of 99% across four datasets, significantly outperforming seven baselines. The experiments also demonstrate strong generalization across different LLM backends, agent architectures, and defense mechanisms.

  3. The paper provides extensive experiments, including ablation studies, comparisons with existing defenses, and evaluations under different configurations. Additionally, the real-world case study using a browser extension demonstrates the practical feasibility of the proposed attack framework.

Critical weaknesses
  1. The proposed attack assumes that the adversary already controls a browser extension installed in the victim agent’s browser, allowing continuous observation and modification of webpage content. This represents a relatively powerful attacker capability and may limit the applicability of the proposed framework to more realistic scenarios where attackers only control specific webpage elements or external content.

  2. Despite the impressive experimental results, most evaluations are conducted on locally hosted websites using a relatively small number of task pairs. Furthermore, the real-world validation is limited to a single browser-use agent and a specific attack scenario. Additional experiments on more diverse websites, complex tasks, and realistic deployment environments would strengthen the paper’s claims about practical effectiveness and generalizability.

Questions for authors

N/A

Reviewer_3afgOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies prompt injection attacks against web agents. The paper presents Automedon, a full-adaptive manipulation framework for web agents. The paper assumes that the adversary controls one browser extension, which can then manipuate webpages as the adversary desires. Based on the agent's action, Automedon can generate new attacks that achieve an ASR of 0.99.

Critical strengths
  1. The generated attacks achieve high ASR under a few defenses considered in the paper.
Critical weaknesses
  1. The adversary model considered in this paper is that the adversary controls a browser extension; I cannot make sense of this adversarial model. If the adversary controls a browser extension, the security consequences are severe because extensions operate inside the browser’s security boundary, bypassing standard web protections like the Same-Origin Policy. The adversary can steal credentials, take over accounts, inject malicious javascripts. Prompt injection is less serious compared to these.

  2. The baselines used in the experiments seem too simple. Each baseline is one template, re-injected on every page. Most have fixed templates. Two include a little LLM generation: Pop-up generates its button label, and InjecAgent generates its text once. The paper cites the MUZZLE paper (Syros et al. USENIX Security 2026), which similarly does adaptive red-teaming of web agents against IPI attacks, but does not compare with it.

  3. The technical contribution is thin. Each module is a single prompted LLM call, Section 3.3 and Insight #1 restate generic properties of adaptive black-box attacks.

Questions for authors

None

Reviewer_7h3TOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Automedon uses a malicious browser extension to redirect web agents. It breaks the attacker’s objective into smaller goals, inserts instructions into each page, and checks page changes to decide what to do next. A graph stores previous attempts. The paper reports high attack success across four datasets and shows that the agent can finish the malicious task before returning to the user’s original task.

Critical strengths
  • The attack usually succeeds without preventing the original task from finishing. Table 1 reports ASRs of 1.00, 1.00, 0.98, and 1.00. Benign-task accuracy matches the no-attack baseline on three datasets and decreases from 0.88 to 0.84 on Shopping.
  • The attacker needs no access to the user’s task or the agent’s reasoning. It uses page changes as feedback. In the browser-use example, it gets the agent to post a comment and then resume the requested title lookup.
  • The ablations test the main parts of the method. Removing the planner, generator, evaluator, or graph lowers ASR in Figure 6. The appendix also tests different backend models and ways of presenting webpages to the agent.
Critical weaknesses
  • The paper does not show whether the graph is better than a plain history of previous attempts. Figure 6 removes the graph but does not explain what past observations and outcomes remain available. This matters because DoomArena, AgentLAB, and MUZZLE already use observation history, coordinated injections, or feedback from failed runs [1–3]. Automedon’s more specific contribution is using only page changes during one continuing run.
  • Some demonstrated harms may already be possible through the extension alone. An extension that can read and change pages may be able to post the Reddit comment or edit the shopping cart directly. The paper does not establish what using the victim agent adds. The browser-use example also relies on induced waits, leaving the reliability of injection timing unclear.
  • The newer-model results cover only Reddit. Tables 6–10 all use that dataset, while the main experiments and browser-use example use GPT-4.1-mini. The 192 task pairs span only three websites, and the expanded Reddit and GitLab sets reuse WASP’s malicious goals. These results do not yet establish transfer to unfamiliar websites or attack objectives.
  • The PromptArmor test does not tell the guard what the user asked. Appendix H uses an observation-only GPT-4.1-mini prompt. Knowing that the user requested a title lookup could help a guard reject instructions to post a comment. The original PromptArmor paper discusses task context and tests GPT-4.1 and o4-mini [5]. Restrictions on which actions the agent may take, as studied in CaMeL [6], are also untested.

[1] Léo Boisvert et al. DoomArena: A Framework for Testing AI Agents Against Evolving Security Threats. COLM 2025. First arXiv version: April 18, 2025.

[2] Tanqiu Jiang, Yuhui Wang, Jiacheng Liang, and Ting Wang. AgentLAB: Benchmarking LLM Agents against Long-Horizon Attacks. arXiv:2602.16901. First version: February 18, 2026.

[3] Georgios Syros et al. MUZZLE: Adaptive Agentic Red-Teaming of Web Agents Against Indirect Prompt Injection Attacks. USENIX Security 2026. First arXiv version: February 9, 2026.

[4] Zeyi Liao et al. RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments. ICLR 2026. First arXiv version: May 28, 2025.

[5] Tianneng Shi et al. PromptArmor: Simple yet Effective Prompt Injection Defenses. arXiv:2507.15219. First version: July 21, 2025.

[6] Edoardo Debenedetti et al. Defeating Prompt Injections by Design. IEEE SaTML 2026. First arXiv version: March 24, 2025.

Questions for authors
  1. Does a plain list of previous attempts work as well as the graph? Keep the same observations, goals, outcomes, failed payloads, generator, and budget.
  2. What prevents the extension from posting the Reddit comment itself? If nothing does, explain what using the victim agent improves, such as reliability or the effort needed to carry out the attack.
  3. Can you repeat the model comparison beyond Reddit? Use a public computer-use agent available before submission on GitLab and Shopping, or a small RedTeamCUA set [4], without retuning the framework’s prompt templates.
  4. Does ASR remain high when PromptArmor receives the original user request and uses GPT-4.1 or o4-mini? Report benign-task accuracy as well, so the comparison captures any cost of blocking legitimate actions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.