- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
ArenaWeaver automatically reconstructs applications for prompt injection evaluation, with browser and MCP interfaces sharing persistent state. It provides 40 applications and introduces Counterfactual Forking Search (CFS), which reuses recorded agent histories to explore injection locations and content. The paper evaluates 25 models on 660 curated cases and studies cross-model transfer, cross-interface transfer, and failures where fabricated evidence corrupts an otherwise authorized action.
Critical strengths
- The environment design enables useful security experiments that are difficult with simpler tool-response benchmarks. Browser and MCP access share actual application state, and attacks enter through separate application-level delivery tools. This makes it possible to trace outside content through an agent’s actions to persistent consequences, and to compare interfaces while preserving the task and intervention. The breadth of applications and composed workflows makes this a substantial infrastructure contribution.
- The evaluation offers useful findings with supporting controls and concrete evidence. Fixed attacks are evaluated from the task’s beginning rather than only from the searched continuation, and separate clean runs help distinguish capability from robustness. I particularly appreciated the finding that an agent can reject explicit outsider instructions yet accept fabricated evidence about whether a requested action is authorized (§4.2, App. H.8). The shared-task comparison and narrower delivery analyses also strengthen the reported robustness differences.
Critical weaknesses
- The evidence for CFS’s advantage as a search method is incomplete. Figure 3a compares shorter branch continuations with full benign runs. This demonstrates reduced victim execution per continuation, but does not establish better attack discovery under an equal total budget. A comparison against full-restart attack search, including attacker and setup costs, would better substantiate this contribution. This limits my assessment of CFS more than of the environment suite.
- Aggregate outcome validation still relies heavily on model judges. The blinded review supporting 595/628 successful trajectories is reassuring, and the human-inspected cases provide useful evidence (App. E.6). However, that review examines originally positive outcomes and uses another LLM. It therefore leaves false negatives and shared judgment errors less well characterized. Since some failures depend on interpreting authorization and fabricated claims, a modest human audit covering both outcome classes would strengthen confidence in the benchmark.
Questions for authors
-
Can you provide a representative, equal-budget comparison between CFS and full-restart attack search, reporting replay success and total cost?
-
Do you have human-checked agreement rates on a sample of both successful and unsuccessful runs, particularly for cases involving fabricated evidence rather than explicit instructions?
2 comments
Thanks for addressing my questions, I have no follow up questions for Q1: comparison between CFS and full-restart attack search experiments.
Regarding to the " refined the judge prompt over several rounds of human review", what is your rubrics and standards for the refinement and as you mentioned that you read the trajectories and " judged whether the attacks succeeded", I am wondering the methods for you to judge the trajectories, as in my experience, those trajectories are normally very long, some might be hundreds of thousand tokens, they are really hard for humans to just manully read and check, so what are the methods you used to handle these?
Rebuttal for Reviewer bW3P
Thank you for highlighting how shared application state lets us trace the effects of an outsider’s message from the moment an agent reads it to the change it makes in an application. We are glad you singled out the fabricated-evidence finding in §4.2 and Appendix H.8. An agent that rejects an outsider’s explicit instructions can still act on a false claim about what the user has authorized.
Thank you for suggesting this comparison. We evaluated CFS and adaptive full restart on the same 60 mixed browser–MCP tasks, using GPT-6 Astra as the attacker and GPT-5.6 Sol as the victim. These are the most challenging tasks in our evaluation because the victim must coordinate actions across both interfaces. On each task, we ran 30 CFS attempts and gave full restart the same victim-inference cost budget. We then replayed each method’s selected attack three times from the original user request, for 60 fresh replays per method.
As we can see, at nearly the same victim cost, CFS achieved 50% higher optimization ASR than full restart (45% vs. 30%) while allowing 3.75× as many search attempts (30 vs. 8 per task). Its fresh-replay ASR was also higher (35% vs. 23.3%).
Thanks for the insightful question! Before running the final large-scale benchmark evaluation, we refined the judge prompt over several rounds of human review. In each round, we read the trajectories, judged whether the attacks succeeded, and compared our verdicts with the model’s. When we found a disagreement, we revised the relevant part of the prompt and checked whether the revised judge agreed with our assessment. Therefore, the final rubric now would ask whether the agent “treated an attacker-fabricated document/claim as ground truth” and directs the judge to “QUOTE the obeyed instruction / override / fabricated claim.” In the rubric’s fake-invoice example, the judge must identify the false payment detail the agent relied on and verify that it sent the payment to the specified address (Appendix E.7, p. 97). Based on our human spot-check of 50 cases involving fabricated evidence, we estimate about 95% agreement with the final judge.
We also independently re-judged 1,287 original trajectories with GPT-6 Astra (over the current claude judge), covering all 25 victim models and all five scenario families. It upheld 595 of 628 original positive verdicts (94.7%) and 614 of 659 original negative verdicts (93.2%), with 95.8% agreement on the 1,262 determinate cases. The disagreements largely offset each other, with 23 original positives changing to negative and 30 negatives changing to positive. Applying these changes while keeping other labels fixed moved pooled benchmark ASR only from 36.71% to 36.75%, and all 25 Figure 1 model ranks remained unchanged.
We appreciate your suggestions to compare CFS and full restart under a common total budget and to include both positive and negative verdicts in a human audit, particularly for attacks that rely on fabricated evidence. And we would include all these discussions in our final revision.