- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper presents Secure-CUA a react based computer use agent that holds security guarantees under an assumed trusted/untrusted observation partition. Untrusted regions are masked and before any action is executed a one step program is established from this masked content. This content is then accessed through an isolated query model, whose responses may influence the action only as specified by the committed program. The paper formalizes generation integrity and grounding integrity and finally evaluates benign utility of their CUA on 400 WebArena tasks, achieving 53.55% average task success compared with 55.12% for the unprotected baseline.
Critical strengths
- It is interesting to try to understand how can we try to establish system security guarantees in a react based model. While I see major flaws in terms of establishing this system as system secure, this may be a practical variant that is more usable.
- The separation between action generation and visual grounding is useful. It highlights that correctly selecting a semantic action does not ensure that GUI commands execute it on the intended target.
- The benign utility results look great.
Critical weaknesses
-
New terminology, generation integrity and grounding integrity, is introduced, however, its relationship to the existing notions of control-flow integrity (CFI) and data-flow integrity (DFI) is not sufficiently explained. It is not quite clear why these more functional security terms were introduced instead of arguing with CFI and DFI. What additional security property does this formulation establish? They seem to describe contracts for different functional stages of the system, rather than clearly distinct security properties.
-
It is unclear that generation integrity and grounding integrity establish full system security guarantees, particularly against branch steering and DFI violations. The isolated query model processes untrusted content, and its Boolean, numeric, or enumeration responses can determine branches within the committed transaction. For example, a third-party seller on a marketplace could put an injection into a correctly masked ticket listing description. When the isolated query model reads this description to answer whether the ticket is refundable, the injection could cause it to return an incorrect Boolean. The committed transaction could then select the non-refundable ticket while both integrity properties still hold: the interpreter follows a permitted branch, and grounding correctly executes the selected action.
-
It is unclear that generation integrity and grounding integrity can even be correctly established in the implementation. The paper should distinguish proving that these properties compose from demonstrating that its components satisfy them. For example, the grounding guarantee assumes accurate grounding. If the intended action is to type into the Description field, but the grounding model returns coordinates inside the Title field, both fields may pass an editability check. Which checks establish that the selected element is the intended semantic target, rather than merely a valid target for that operation? Similarly, how is it established that untrusted content cannot reach the planner through later observations or runtime feedback? Adaptive attacks should test the limits of this.
-
It is unclear that all content on a website can be clearly marked as untrusted or trusted, and that this distinction is sufficient for security. First, the paper should explain more precisely how DOM labeling translates into masking of the visually rendered website as HTML and JavaScript dynamically change the page. It describes remasking and stability checks, but their effectiveness needs evaluation. The paper should also clarify whether website vulnerabilities such as XSS are excluded: if attacker-controlled content can execute JavaScript and modify DOM elements outside the labeled region, the assumed boundary between trusted and untrusted content may no longer hold.
-
Second, a website owner’s incentives are not necessarily aligned with the user’s agent. A retailer may want to upsell a more expensive product or preselect an upgrade. Secure-CUA masks some marketing content, but what justifies trusting the remaining owner-controlled content? This is why DFI defenses introduce policy constraints on the resulting actions. However, policies need to change over state changes: they either continue holding, need to be combined with new policies, or expire. For example, permission to change a payment recipient before confirmation should not remain available afterward. This is why previous work first establishes CFI by committing to a plan ahead of time, and then addresses DFI through policies governing data flows and action parameters within that plan. The plan provides the execution context needed to determine which policies apply at each stage. In the current implementation, it is unclear how equivalent guarantees can be established: each transaction commits only to the next action, and untrusted query responses can select its branches, without an explicit specification enforcing security constraints across subsequent state transitions.
-
A major problem is that a lot of the security hinges on manually labeling the right website elements as content to be masked, and this labeling problem is underplayed. The paper assumes complete labels, but how can correctness be ensured when deployment may require labeling hundreds of websites and maintaining those labels as websites change? Fatigue can cause misses, and complicated website structures may make the origin of an element unclear. Even if an external attacker controls only a plain-text issue title or comment, the website may render it in multiple places, such as previews, notifications, or accessible button labels, any of which could be missed during labeling.
-
Say we give up on making humans manually label websites and instead use an LLM for this. We then need a separate security argument for the labeling process. First, directly exposing the labeller to untrusted content reintroduces the prompt-injection problem. Second, sanitizing the DOM before labeling, as explored by UCM (Nikolić et al.), may remove context useful for distinguishing trusted from untrusted content. At the same time, the remaining parts of the DOM may not reveal which data could be supplied by a third party. For example, an owner-written heading and a seller(third-party)-supplied title may have identical DOM structures, without their provenance being recoverable from the sanitized DOM. Automatic labeling therefore cannot simply replace manual labels while inheriting the same guarantee.
-
On that note, the contribution beyond Nikolić et al. could be made clearer. It seems currently the contribution is adding per-action transactions that constrain how query responses influence actions, as well as handling of free-form strings?
-
The overhead of this defense is surprisingly large in terms of tokens.
-
The RQs focus on benign utility, access to untrusted information, and overhead, without sufficiently evaluating whether Secure-CUA’s implementation satisfies its security assumptions under attack. The claim that the system is secure by design does not establish complete labeling, correct grounding, or safe influence from endorsed query responses. The evaluation should include the cases above, with precise attacker permissions, adaptive attacks, and ablations showing which mechanism prevents each failure. It should report both attack success and whether legitimate tasks can still be completed under attack.
Questions for authors
See Weaknesses.
Thank you for the new attack results and the grounding control — I consider W2 resolved. My concern on W1 stands, however: for a paper whose central claim is security, the experimental section should be organized around demonstrating how secure the framework is, and the current evidence — a small set of known attack scenarios appended in the rebuttal — does not carry that weight. I maintain my rating.