- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes TraceEdit, a self-evolving attack framework for GUI agents. It coordinates five environment-editing channels and introduces a counterfactual persistence reward that measures whether malicious influence remains after adversarial edits are removed. The attacker is trained with constrained reinforcement learning to balance attack success, persistence, and stealth. Experiments across six benchmarks and four GUI agents show improvements over thirteen baselines, supported by ablations and robustness analyses.
Critical strengths
1 The paper addresses an important measurement problem: stopping edits does not necessarily stop exposure to an attack. 2 The appendix provides substantial controls, including full-DSL versions of prior attackers, channel-by-reward ablations, environment restoration, and replay checks. 3 Evaluation extends beyond attack success to human detectability, judge agreement, transfer, and defenses, offering several useful perspectives.
Critical weaknesses
- I have a major concern regarding the counterfactual rollback protocol. Section 3.2 applies Undo immediately after the last edit, while Table 7 states that pending timers are canceled. However, in Tables 36–37, scheduled navigation occurs at step 6, after the rollback point at step 5, yet the examples still report RHLremoved=3. Could the authors clarify how these navigations survive rollback? This seems important for the validity of the central persistence metric.
2 The human stealth results do not agree across presentations. Figure 12 labels its vertical axis 1-HDR, with TraceEdit’s benchmark points around 0.83–0.91. These imply HDR values around 9–17%, whereas Tables 1 and 32 report 27.9% under the same stated setting. Are different subsets or annotation units being used? This matters because stealth is a central claim.
Questions for authors
I find this to be a very solid and carefully executed work. I would appreciate the authors' responses to my questions above, and I am open to raising my score if these concerns are adequately addressed.