acceptodds
Under review as a conference paper at ICLR 2027

Cross-Surface Prompt Injection against Safety-Aligned Mobile GUI Agents

Abstract

Mobile GUI agents increasingly act on behalf of users in real app workflows. This autonomy makes the agent's interpretation of on-screen content a direct security concern. Existing indirect prompt injection attacks mostly place the decisive malicious objective on the same surface the agent is already reading. In mobile workflows, this same-surface assumption creates two obstacles: (1) attacker-controllable fields are often short and cannot host structured interaction, and (2) direct requests for credentials, private data, authorization, or other high-risk actions expose malicious intent early. To overcome these limitations, we propose CrossSurf, a cross-surface prompt injection attack that decouples navigation from the downstream attack objective. The design of our attack is inspired by the observation that the same constrained field can still carry a short navigation request and link that opens an external continuation page from the source surface. CrossSurf optimizes two artifacts: a Stage I jump instruction that makes the cross-surface transition appear consistent with the benign workflow, and a Stage II attacker-controlled page that presents the guidance, structured fields, and action target needed for the downstream objective. We construct a benchmark with 117 mobile test cases from 39 source apps across eight domains and 6 risk categories. Across five representative victim GUI agents, CrossSurf raises ASR from 5.0% for a direct single-stage baseline to 30.9%, showing that cross-surface injection exposes vulnerabilities missed by same-surface attacks. Prompt-only defenses help modestly, while task-alignment and multimodal action-guard wrappers are more effective after the jump but do not eliminate the risk. Our code and data are available at https://anonymous.4open.science/r/CrossSurf/.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Mobile GUI Agents, Indirect Prompt Injection, LLM Security

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_vV8XOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces a new prompt injection attack scheme, which is called "cross-surface prompt injection". The basic idea is separating the attack into two stages, including Stage I (jump instruction injection) which misleads the agent into a attacker-controlled page, and Stage II (attack page optimization) where the actual attack happens with the whole compromised page. Evaluation was conducted on a self-constructed benchmark with 117 mobile test cases from 39 source apps across eight domains and 6 risk categories, showing clear improvement of attack success rate.

Critical strengths

This paper introduces a simple but effective attack scheme with potentially higher success rate, which is potentially helpful for future design of agent models and harness.

Critical weaknesses
  • The approach is too straightforward as a research paper, lacking sufficient technical depth.
  • The attack may be easily mitigated by restricting and monitoring page jumps.
Questions for authors
  • What is the main technical challenge tackled in this paper?
  • Could you briefly discuss the takeaway messages for mobile agent/system designers? For example, what could they do to avoid such attacks?
3 comments

We thank Reviewer vV8X for the constructive feedback. All revisions are highlighted in blue in the updated PDF.

W1 & Q1: Main technical challenge tackled in this paper.

The core challenge is constructing an executable attack path under asymmetric interface control (§4.1), where naive redirection fails due to three coupled tensions:

  1. Stealth vs. persuasion under tight source constraints: Stage I must fit into a short third-party field (e.g., SMS, comment, calendar note) without altering surrounding UI controls. In Appendix A.6, 6/42 direct objective disclosures fail character-limit screening. The trigger must stay plausible enough to avoid refusal while still persuading the agent to leave the app.
  2. Coherence vs. actionability on the destination: Stage II must balance contextual continuity (ConsII) with a clean, executable UI route. Figure 5 (Minimal-Low 41.67% vs. Structured-High 70.37%) and Table 20 (removing ConsII drops completions from 73 to 61) show that both are necessary.
  3. Stage-specific black-box failure attribution: In a two-step pipeline, end-to-end failure (ASR = 0) is ambiguous. Editing the destination cannot fix a missed jump, and editing the jump cannot fix a post-jump refusal. Decoupling arrival (JSR) from post-arrival (ASR) diagnosis directs refinement to the responsible artifact.

Empirical results confirm this difficulty. CrossSurf outperforms Generic Cross-Surface by +9.7 pp average ASR across five agents (30.9% vs. 21.2%, Table 1). Table 3 ablations show that removing Stage I drops JSR/ASR by 7.6/7.5 pp, while removing Stage II drops ASR by 8.0 pp at similar JSR. Matched controls (§5.5, Tables 11 & 14) further show continuity ×\times structure interaction on form-complete pages and a +6.48 pp advantage for deferred disclosure at equal destination functionality and budgets.


W2 & Q2: Mitigation via restricting page jumps and takeaways for mobile agent/system designers.

In closed workflows that never require external pages, blocking all browser and WebView transitions structurally eliminates the attack (S⊆JS \subseteq J). However, in realistic open workflows where users expect agents to follow external links, naive jump blocking destroys benign utility. This motivates our selective navigation (D2) and pre-action authorization (D3) controls (Section 5.5, Table 5; Table 19):

PolicyOriginal JJ / SSAdapted JJ / SSClean: no ext. (/108)Clean: ext. (/108)Wrong blocks (/216)Latency
D0: no added gate135 / 73—889200.0 s
D1: complete gate0 / 00 / 08701090.1 s
D2: selective nav.94 / 50124 / 668483131.4 s
D3: D2 + action auth.94 / 16120 / 388377203.1 s

Takeaways for mobile agent and system designers (§5.5, Appendices A.5–A.7):

  1. Block jumps completely in closed workflows: D1 eliminates attacks (0/216 original and adapted) with minimal clean loss on non-external tasks (87/108 vs. 88/108).
  2. Decouple inspection from action authority in open workflows: Where external pages are needed (D1 drops clean completion to 0/108), selective navigation alone leaves high risk (50/216 original; 66/216 adapted). Adding pre-action authorization (D3) cuts compromise to 16/216 (original) and 38/216 (adapted) while retaining 77/108 external-step utility.
  3. Preserve provenance and check before data entry: Controllers should track resolved origin/redirect metadata across browser and WebView switches (94.4% coverage in our adapter) and verify operation, data, and recipient before typing, pasting, uploading, or committing state.

Dear Reviewer vV8X,

Thank you for your constructive feedback. We would like to briefly summarize how the rebuttal addresses your questions about the technical challenge and the implications for mobile agent designers.

  1. We clarified the technical challenge of diagnosing failures across two interfaces with different constraints. The revised discussion explains how the evaluation measures navigation and post-navigation outcomes separately (W1, Q1).
  2. We evaluated your suggestion to block external navigation. Complete blocking prevented all compromises in the evaluated trials, but it also prevented completion of all 108 clean tasks that required an external step (W2, Q2).
  3. For workflows that require external pages, we evaluated selective navigation with pre-action authorization. This reduced compromises and retained completion on 77 of the 108 clean tasks that required an external step (W2, Q2).

We would be happy to clarify any remaining questions. If the revisions address your concerns, we would appreciate your reconsideration of the rating.

Best regards,
The Authors

Thank the authors for the rebuttal. I still think this technical contribution is too straightforward to meet the bar of ICLR. How to balance contextual continuity and attack success rate might be a good place for innovation, but the current investigation into this is too shallow. I'd keep my rating.

Reviewer_PdgGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies a two-stage prompt-injection attack on mobile GUI agents. It first directs the agent to an external page, then uses that page to induce an unsafe action. Experiments across multiple applications and agents show that this approach is more effective than the evaluated direct-injection baseline.

Critical strengths
  • The paper addresses a relevant security problem in mobile agents. The evaluation across multiple agent configurations shows that malicious content can redirect agents away from the user’s intended task.
Critical weaknesses
  • The technical contribution appears incremental. The main idea is to separate a benign-looking navigation request from the harmful instructions on an external page, then refine both through feedback. The experiments support the effectiveness of this design, including its improvement over baseline. However, the paper makes a stronger case for an effective attack strategy than for a substantial methodological advance.

  • There is no defensive lesson from the paper. Although the paper has evaluated three defenses, none of them are designed adaptively against the proposed attack. The attack bypassing existing defense is not surprising but what is more important is what lessons can we learn for this advanced attack. This paper has no discussion or furehter evaluation for this point.

Questions for authors
  1. What is the main new security insight beyond showing that an agent can be redirected to a malicious page?
  2. What is the attack robustness against adaptive defenses?
2 comments

We thank Reviewer PdgG for the constructive feedback. All revisions are highlighted in blue in the updated PDF.

W1: Technical contribution relative to prior work.

Beyond prior indirect prompt injection studies (Table 15), CrossSurf makes three concrete contributions:

  1. Asymmetric mobile setting at scale: Unlike single-surface web-DOM attacks (EIA, EVA) or static/cross-app phishing precedents (WASP, Hijacking JARVIS, MUZZLE), CrossSurf models a constrained boundary across 39 real Android apps (117 cases). The attacker controls only a short third-party field on a benign app and must induce a transition to a separately built interactive page.
  2. Automated two-stage black-box optimization: CrossSurf jointly optimizes a workflow-consistent jump trigger (Stage I) and a feedback-diagnosed interactive destination (Stage II). Across five agents (Table 1), this raises average ASR from 5.0% (Direct) and 21.2% (Generic Cross-Surface) to 30.9%. Ablations in Table 3 confirm that both Stage I (+7.6 pp JSR, +7.5 pp ASR) and Stage II (+8.0 pp ASR) are essential.
  3. Causal mechanism isolation and adaptive defense evaluation (§5.5, Appendices A.6–A.7): Matched controls prove that deferred disclosure outperforms early disclosure at equal destination functionality and budget (+6.48 pp, Table 14). They also show that continuity interacts with page structure on form-complete pages (Figure 5, Table 11), and that pre-action authorization cuts compromise from 50/216 to 16/216 at fixed navigation (Table 5).

W2 (part 1) & Q1: New security insights beyond redirection.

Redirection only exposes the agent to a page. Downstream harm depends on interface organization and action authority (§1, §3, §5.5):

  1. Task relevance is not action authorization: In Table 5, D2 and D3 share the exact same navigation filter (94/216 arrivals on original artifacts). Adding a pre-action authorization check cuts terminal compromise from 50/216 (23.15%) to 16/216 (7.41%), and from 66/216 to 38/216 after adaptation. Permitting a relevant page visit must not imply authority for downstream data entry or state changes.
  2. Workflow continuity acts as an implicit authority cue: From the same post-jump checkpoint with all pages 100% functional (Figure 5, Table 11), making the destination look like a seamless continuation raises completion by +12.50 pp (minimal) and +18.98 pp (structured). Thus, GUI agents conflate semantic and visual continuity with user authorization even across untrusted WebView boundaries.

W2 (part 2) & Q2: Adaptive defenses and defensive lessons.

To complement the wrapper comparison in Table 4, Section 5.5 (Table 5) and Appendices A.5–A.7 (Tables 17 & 19) evaluate three frozen controller policies on the 36-task control suite (216 attack trials across Mobile-Agent-v2 + GPT-4o/Qwen2.5-VL-72B; 216 clean trials split evenly by external-step requirement):

  • D1 (complete gate): blocks all target browser and WebView transitions.
  • D2 (selective navigation): uses a frozen GPT-4o-mini classifier (τ=0.50\tau=0.50) and an ADB/WebView origin-redirect adapter (94.4% coverage) with only the original task and trusted metadata.
  • D3 (D2 + pre-action authorization): shares D2's exact navigation policy and additionally checks operation, data, and recipient before any data entry or state change.

We evaluate both original and adapted artifacts (re-optimized against each frozen defense under fixed objectives, permissions, and query budgets):

PolicyOriginal JJ / SSAdapted JJ / SSClean: no ext. (/108)Clean: ext. (/108)Wrong blocks (/216)Latency
D0: no added gate135 / 73—889200.0 s
D1: complete gate0 / 00 / 08701090.1 s
D2: selective nav.94 / 50124 / 668483131.4 s
D3: D2 + action auth.94 / 16120 / 388377203.1 s

Key defensive lessons:

  1. Closed vs. open workflows: Complete blocking (D1) eliminates the attack (0/216) with minimal loss on closed tasks (87/108 vs. 88/108), but it breaks external-continuation tasks (0/108).
  2. Pre-action authorization in open workflows: D3 sharply reduces compromise vs. D2 alone (50→16 original; 66→38 adapted) while keeping external-step utility at 77/108 (with 26/216 user confirmations).
  3. Substantial mitigation with residual adaptive risk: Even after defense-aware re-optimization, D3 cuts terminal compromise by 42.4% relative to D2 (38/216 vs. 66/216) and 47.9% relative to D0 (38/216 vs. 73/216). However, the remaining 17.59% ASR shows that provenance-preserving controllers must be evaluated under adaptive attacks rather than static wrappers alone.

Dear Reviewer PdgG,

Thank you for your constructive feedback. We would like to highlight three additions that address your questions about the paper’s contribution and defensive implications.

  1. We expanded the comparison with prior work and added matched controls to clarify the contribution of studying cross-surface behavior in mobile workflows (W1).
  2. We clarified the main security implication: an agent should evaluate permission to visit a task-relevant page separately from authorization to enter data or change state on that page (W2, Q1).
  3. We added an adaptive defense evaluation. In that evaluation, adding a pre-action authorization check to selective navigation reduced compromises from 66 of 216 trials to 38 of 216 (W2, Q2).

We would be happy to clarify any remaining questions. If the revisions address your concerns, we would appreciate your reconsideration of the rating.

Best regards,
The Authors

Reviewer_TGQcOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces CrossSurf, a cross-surface prompt injection attack against safety-aligned mobile GUI agents. The key idea is to decouple navigation from the harmful objective: a Stage-I "jump instruction" embedded in a constrained attacker-controllable field (SMS/email body, comment, note) asks the agent to open a workflow-consistent continuation link, and a Stage-II attacker-controlled page then presents the structured fields and action target for the downstream objective. Both stages are optimized with VLM-generated candidates scored on attack success (JSR/ASR) plus workflow-consistency signals, with iterative feedback editing for the page. The authors build a benchmark of 117 cases across 39 apps, 8 domains, and 6 risk categories and evaluate 5 mobile GUI agents. CrossSurf raises average ASR from 5.0% (direct injection) to 30.9% and average CNSR to 53.7%, with cross-agent transfer and defense results (Task Shield reduces CNSR to 25.0%).

Critical strengths
  1. The cross-surface decomposition is a clean and useful reframing of indirect prompt injection for mobile agents. By separating a benign-looking navigation request from the downstream objective, the paper isolates two distinct failure points (jump-following vs. post-jump action completion) that prior same-surface attacks conflate, and shows each is independently exploitable and independently defendable (Stage-I / Stage-II ablations).
  2. The empirical foundation is unusually thorough for this space. A 117-case benchmark spanning 8 domains and 6 risk categories, five diverse agents (general VLMs + UI-specialized), controlled baselines, ablations, cross-agent transfer, and defense evaluation — all under well-defined metrics (JSR/ASR/CNSR plus TCclean/TCattack that separate safety from task capability).
Critical weaknesses
  1. The core mechanism remains "follow an injected link," so the novelty is a re-decomposition of known indirect-injection attacks rather than a fundamentally new vulnerability class. The conceptual advance over EVA/WASP/Hijacking JARVIS is real but moderate, and it is not shown that cross-surface delivery produces harm that same-surface attacks could not reach with comparable attacker effort.
  2. Artifacts are generated by a very strong proprietary model (GPT-5.4), and consistency signals rely on VLM judges without reported human calibration. This raises the concern that measured ASR partly reflects attacker-side capability and judge bias rather than a fundamental victim vulnerability, and it limits reproducibility for non-API users.
Questions for authors
  1. How sensitive are results to the attacker-side model? Would generating artifacts with a weaker or comparable-strength model substantially change JSR/ASR? This determines whether the threat is a property of the victim or of the attacker's tooling.
  2. The consistency judges (ConsI/ConsII) are central to your selection signals. Did you validate them against human judgments (e.g., inter-annotator agreement), and how robust are the final artifacts to judge noise?
  3. Beyond the ablations, can you decompose how much of the Stage-II gain comes from providing a structured form (surface expansion) versus the workflow-consistency framing alone? A form-complete but low-consistency control page would isolate the two mechanisms.
3 comments

We thank Reviewer TGQc for the detailed and constructive feedback. All revisions are highlighted in blue in the updated PDF.

W1: Novelty relative to prior work and why cross-surface delivery differs fundamentally from same-surface attacks.

Relative to prior work (Table 15), CrossSurf captures a structural capability and trust-boundary asymmetry unique to real mobile app workflows:

  1. Structural expansion beyond same-surface limits: In native mobile apps (e.g., SMS, Gmail, Calendar), an unprivileged attacker controls only a short, passive text field. They cannot inject forms, buttons, or multi-step UI widgets. In our timing-control screening (Appendix A.6), 6/42 tasks (14.3%) could not even fit early objective disclosure due to source character limits. Crossing into a WebView or browser page upgrades the attacker from length-bounded text to full interactive UI control. It also defers the risky request past source-side inspection, because a source-only filter sees only a benign navigation prompt. This structural shift explains the 6× gap over Direct Injection (30.9% vs. 5.0% ASR, Table 1). It also explains why selective navigation alone (D2) still leaves 50/216 (original) and 66/216 (adapted) compromise until D3 enforces post-jump pre-action authorization (Table 5).
  2. Automated joint optimization across 39 real apps: Unlike single-surface web-DOM attacks (EIA, EVA) or static/cross-app phishing precedents (WASP, Hijacking JARVIS, MUZZLE), CrossSurf automates black-box optimization of both the constrained source trigger and the feedback-refined destination page across 117 cases. It outperforms Generic Cross-Surface by +9.7 pp ASR (30.9% vs. 21.2%, Table 1). Ablations in Table 3 confirm that both stages (+7.6/+7.5 pp for Stage I; +8.0 pp for Stage II) are essential.
  3. Matched-effort timing advantage even with equalized destinations (Tables 14 & 19): We also artificially granted early disclosure the identical functional destination and 96-query budget across the 36 eligible tasks (matched source lengths: 41 vs. 39 chars):
Objective presentationArrivals / trialsSuccesses / trialsJSRASRMean queries / task
Early + matched destination123/21659/21656.94%27.31%58.2
Deferred + matched destination135/21673/21662.50%33.80%61.4

Deferred disclosure still outperforms early disclosure by +6.48 pp (95% paired CI [1.2, 11.8] pp). Thus, cross-surface delivery combines a structural capability and bypass advantage with an additional +6.48 pp timing advantage even when destination functionality is held equal.


W2 (part 1) & Q1: Sensitivity to the attacker-side construction model.

Because cross-agent transfer (Table 2) replays fixed artifacts, we directly replaced the construction model (generation, diagnosis, and editing together) across GPT-5.4, GPT-4o, and Qwen2.5-VL-72B-Instruct. Victims (Mobile-Agent-v2 + GPT-4o/Qwen2.5-VL-72B), judges, and budgets remained fixed (Tables 12 & 19):

Construction modelArrivals / trialsSuccesses / trialsJSRASRInvalid rateCost / task
GPT-5.4 (Reference)135/21673/21662.50%33.80%10.2%0.84 USD
GPT-4o130/21664/21660.19%29.63%12.1%0.62 USD
Qwen2.5-VL-72B-Instruct101/21644/21646.76%20.37%23.3%0.58 USD

Even with GPT-4o (29.63%) or open-weight Qwen2.5-VL-72B-Instruct (20.37%), CrossSurf still induces substantial terminal compromise. Stronger construction models mainly improve search efficiency (invalid rate: 10.2% vs. 23.3%, Table 19). This shows that the vulnerability lies in the victim agents rather than being an artifact of GPT-5.4 alone.

W2 (part 2) & Q2: Human calibration of ConsI/ConsII and robustness to judge noise.

Sections 4.2–4.3 define ConsI/ConsII as heuristic selection signals rather than authorization checks. In a blinded calibration (Table 18, Appendix A.7), three researchers rated 144 balanced artifacts (72 Stage I, 72 Stage II) under rubric v1.0:

CheckScopeResult
Human–human agreementPre-adjudication (144 artifacts)Continuity α=0.78\alpha=0.78 [0.71, 0.84]; usability α=0.81\alpha=0.81 [0.75, 0.87]; auth. κ=0.84\kappa=0.84 [0.78, 0.90]
Judge–human agreementFrozen scores vs. human ratingsConsI cont. ρ=0.74\rho=0.74; ConsII cont./usab. ρ=0.68/0.79\rho=0.68 / 0.79; ConsII vs. auth. κ=0.21\kappa=0.21
DisagreementsPre-adjudication & auth. gap26/144 (18.1%) pre-adjudication; 11/72 (15.3%) actionable-implausible; 19/72 (26.4%) high-ConsII unauthorized

The low ConsII–authorization agreement (κ=0.21\kappa=0.21) confirms that plausibility does not imply authorization.

To test robustness to judge noise, we evaluate both fixed-pool re-selection (Table 13, 72 pools of 8 candidates) and full sequential-search reruns from scratch (Table 20):

ConfigurationFixed-pool changed / 72Fixed-pool fresh S/NS/N (ASR)Full-search valid / 36Full-search fresh J/NJ/N, S/NS/N
Reference (GPT-5.4)0/7273/216 (33.80%)35/36135/216, 73/216
Claude Sonnet 4.5 judge29/7271/216 (32.87%)34/36131/216, 69/216
Without ConsI18/7268/216 (31.48%)35/36122/216, 65/216
Without ConsII25/7263/216 (29.17%)33/36133/216, 61/216

Replacing the judge with Claude Sonnet 4.5 changes 29/72 fixed-pool choices, while fresh ASR shifts by under 1 pp (33.80% to 32.87%; full search: 69/216). This confirms that attack effectiveness is robust to judge noise. Full reruns also show that ConsI primarily drives arrivals (135 to 122), whereas ConsII primarily drives post-jump completion (73 to 61).


Q3: Decomposing Stage-II gains: page structure vs. workflow continuity.

We implemented the requested 2×22\times 2 form-complete control from the identical post-jump checkpoint (Figure 5, Section 5.5; Table 11, Appendix A.6). All four cells keep the identical objective, fields, buttons, endpoint, and working action route:

Complete interfaceLow continuityHigh continuityContinuity contrast
Minimal complete90/216 (41.67%)117/216 (54.17%)+12.50 pp
Structured complete111/216 (51.39%)152/216 (70.37%)+18.98 pp

Per-cell 95% Wilson CIs are [35.3,48.3]%[35.3, 48.3]\%, [47.5,60.7]%[47.5, 60.7]\%, [44.8,58.0]%[44.8, 58.0]\%, and [64.0,76.1]%[64.0, 76.1]\%. Structure contrasts are +9.72 pp (low continuity) and +16.20 pp (high continuity). Even when all forms are equally functional, structured UI alone adds +9.72 pp. Meanwhile, workflow-consistent framing acts as an implicit trust cue that amplifies action completion on structured pages by +18.98 pp (reaching 70.37%). This shows a clear positive interaction between surface expansion and semantic continuity.

Dear Reviewer TGQc,

Thank you for your detailed feedback. We would like to briefly summarize the additions in our rebuttal.

  1. We expanded the comparison with prior work and added a matched experiment that holds destination functionality and the evaluation budget fixed (W1).
  2. We evaluated sensitivity to three construction models, including an open-weight model. We also added a blinded human calibration of the consistency judges and tested how sensitive the results were to judge choice (W2, Q1–Q2).
  3. We added the requested form-complete control to distinguish the effects of interface structure and workflow continuity (Q3).

We would be happy to clarify any remaining questions. If the revisions address your concerns, we would appreciate your reconsideration of the rating.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.