- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes AdvSim2Real, a framework for improving the robustness of web agents against adaptive prompt injection attacks through adversarial training in a simulated web environment. The framework consists of two training stages: (1) a curriculum agent dynamically generates tasks of appropriate difficulty to improve the web agent’s task completion capabilities, and (2) an adversarial agent generates prompt injection attacks to improve the defender’s robustness through co-evolution. Both stages operate within a frozen web world model, with an LLM judge providing feedback for reinforcement learning. The authors introduce a success-flip reward mechanism that rewards attacks only when they cause previously successful tasks to fail. Experiments on 150 web tasks demonstrate improvements in both clean task completion and robustness against adaptive attacks, including a 33.6% relative improvement against an unseen adversary. The paper also shows that task completion improvements learned in simulation can partially transfer to a real browser environment.
Critical strengths
-
The paper is generally well-written and clearly organized. The motivation, methodology, and experimental setup are presented in a logical and easy-to-follow manner, making the proposed framework straightforward to understand.
-
The paper provides extensive experiments, including evaluations under multiple adaptive attackers, comparisons with baseline models, ablation studies, and sim-to-real transfer experiments. These results offer useful insights into the effectiveness and limitations of the proposed approach.
-
The combination of adaptive curriculum learning and adversarial training within a web world model is an interesting approach. The success-flip reward mechanism is also a reasonable design for encouraging attackers to generate meaningful adversarial examples.
Critical weaknesses
-
Although the paper demonstrates improvements in task completion when transferring from simulation to a real browser, the robustness against prompt injection attacks is evaluated exclusively within the simulated environment. Without evaluating the trained defender against attacks in real browser environments, it remains unclear whether the reported robustness improvements generalize to realistic scenarios.
-
The evaluation is conducted on only 150 synthetic web tasks, primarily focused on form-filling operations. These tasks may not adequately represent the complexity and diversity of real-world web agent interactions. Evaluations on established benchmarks such as WebArena or more realistic websites would strengthen the paper’s claims about generalizability.
-
The framework relies heavily on a frozen world model and an LLM judge for both training and evaluation. As acknowledged by the authors, the world model can generate inaccurate states, and the judge can incorrectly assess task completion. These inaccuracies may introduce unreliable reward signals and potentially overestimate the defender’s robustness. A more rigorous evaluation using deterministic success criteria would strengthen the conclusions.
Questions for authors
N.A
2 comments
Dear Reviewer 5ZU5,
Thank you again for your careful review and positive comments on the paper's clarity, methodology, and experimental evaluation. In our response, we
- demonstrated that adversarial robustness transfers to real Chromium, with 10–13 percentage-point gains over capability-only training,
- clarified the diversity of our tasks and the limitations of evaluating on WebArena,
- audited world-model and judge errors and confirmed our findings using a deterministic browser checker.
As the discussion period ends soon, we'd be grateful to hear whether these results address your concerns. We are happy to answer any further questions.
We thank the reviewer for the positive assessment of the paper's clarity and experiments.
W1: Robustness is evaluated only in simulation
We evaluated the attacks in real Chromium. Each injection is rendered as a static HTML notice at the step where the reactive adversary injected it in simulation, and success is scored by the deterministic strict checker, with no world model or LLM judge. Results are strict success in %, 3 seeds × 150 tasks.
The robustness transfers to the real browser. Robust iter 3 beats Capability iter 3 by 10–13 points under every adversary (paired 95% CIs exclude zero), while the two are tied on clean tasks. The gap is larger in the browser than in the world model, so the simulator does not overestimate robustness.
W2: 150 synthetic form-filling tasks; WebArena
WebArena uses a different action space from the one our executor and the WebWorld-14B world model operate in, so evaluating there would measure the action-space mismatch rather than robustness. Our tasks are built so that robustness can be verified exactly. Each specifies required final values, protected fields, forbidden controls and a deterministic checker, which is what made the browser evaluation in W1 possible without an LLM judge. The tasks span five skill strata (conditional policy, derived values, conjunctive selection, relational join and ordered repair), with reference solutions of 4–9 actions.
W3: Reliance on the world model and LLM judge
The browser results in W1 use only the deterministic checker, and they confirm the paper's conclusion with a larger effect. We also audited the world model and judge against a deterministic field check: