- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Automedon, a full-adaptive prompt injection framework that manipulates black-box web agents into executing multi-step malicious tasks. Unlike existing static or partially adaptive attacks, Automedon leverages browser extensions to continuously monitor webpage transitions and dynamically adjust malicious payloads. It consists of three components: a Planner for selecting intermediate attack goals, a Generator for producing context-aware payloads, and an Evaluator for verifying attack progress. Experiments across four datasets demonstrate an average 99% attack success rate, significantly outperforming existing baselines while largely preserving benign task performance. The framework also shows strong effectiveness across different LLMs, agent architectures, and defense mechanisms.
Critical strengths
-
The paper introduces a full-adaptive prompt injection framework that continuously adjusts attack strategies based on webpage state transitions. This addresses the limitations of existing static and partially adaptive attacks, particularly in long-horizon web agent tasks.
-
The proposed method achieves an impressive average attack success rate of 99% across four datasets, significantly outperforming seven baselines. The experiments also demonstrate strong generalization across different LLM backends, agent architectures, and defense mechanisms.
-
The paper provides extensive experiments, including ablation studies, comparisons with existing defenses, and evaluations under different configurations. Additionally, the real-world case study using a browser extension demonstrates the practical feasibility of the proposed attack framework.
Critical weaknesses
-
The proposed attack assumes that the adversary already controls a browser extension installed in the victim agent’s browser, allowing continuous observation and modification of webpage content. This represents a relatively powerful attacker capability and may limit the applicability of the proposed framework to more realistic scenarios where attackers only control specific webpage elements or external content.
-
Despite the impressive experimental results, most evaluations are conducted on locally hosted websites using a relatively small number of task pairs. Furthermore, the real-world validation is limited to a single browser-use agent and a specific attack scenario. Additional experiments on more diverse websites, complex tasks, and realistic deployment environments would strengthen the paper’s claims about practical effectiveness and generalizability.
Questions for authors
N/A