acceptodds
Under review as a conference paper at ICLR 2027

Benchmarking and Jailbreaking Phone-use Agents for Social Media Misuse

Abstract

Phone-use agents operate mobile apps through native GUIs, giving them access to functions that social platforms rarely expose through public APIs, such as reporting. Abuse of these functions relies heavily on human labor, and phone-use agents could substantially reduce its cost. Yet existing benchmarks rely on mock apps that offer limited support for abuse relevant functions, and they cover only a narrow range of real-world abuse scenarios. In this work, we take a pioneering step toward systematically evaluating this risk with BadPhoneAgent, a benchmark dedicated to misuse of phone-use agents on social media. It provides a high-fidelity environment spanning 11 social media apps and 18 abuse relevant functions, alongside 150 manually constructed test cases grounded in platform policies and organized into 8 categories and 35 subcategories. Replaying the same tasks on a physical phone reproduces the outcomes observed in our environment (91.5% agreement), indicating that the risks it measures also arise in real apps. To further expose this risk, we propose View Editing for Intent Laundering (VEIL), which exploits the underexplored gap between the model's observation and the phone state. VEIL reaches attack success rates (ASR) of 88% on GPT-6-Astra and 64.7% on Claude Fable 5, while these models refuse only 8.0% and 1.3% of requests, respectively. More concerning, several open-source models surpass these frontier commercial models under VEIL at speed comparable to human baseline. These results reveal the potential for large-scale social media abuse by phone-use agents and provide a foundation for developing safer phone-use agents.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Phone-use agents

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_GBbjOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies the misuse risk of phone-use agents on social media. The key observation is that, by operating apps through native GUIs, these agents can access functions platforms rarely expose via public APIs (reporting, danmaku, direct messaging)—abuse that traditionally relied on human labor and that agents could make far cheaper and more scalable. Existing benchmarks either use mock apps lacking such functions or cover only narrow abuse scenarios. The paper makes two contributions. BadPhoneAgent is a high-fidelity evaluation environment spanning 11 Chinese/English social apps and 18 abuse-relevant functions, with 150 policy-grounded test cases (8 categories, 35 subcategories); a semi-automated framework replicates real apps on a lightweight virtual platform, enabling large-scale repeated evaluation without real accounts and direct task-success checks from app state. VEIL (View Editing for Intent Laundering) is an attack exploiting a structural flaw: agents judge safety from what they observe, yet act on the true phone state. Controlling the local framework, the attacker modifies the model’s inputs (instruction, history, screenshot) without changing app state by (a) splitting a harmful request into individually benign subtasks and clearing history between them, and (b) using SIFT/template matching (no model inference) to mask screenshot regions linking the subtasks. Main findings: substantial risk exists even without attacks (GPT-6-Astra completes 52% of harmful tasks while refusing only 44%; several open-source models refuse none); VEIL raises GPT-6-Astra’s ASR from 52% to 88% and Qwen3.8-27B to 94.7%—surpassing all commercial models at 1.46× human speed—while dropping refusal rates sharply (Claude Fable 5 from 44.7% to 1.3%); real-device validation shows 91.5% virtual–physical agreement (κ=0.80) with VEIL still effective; ablations confirm all three components cumulatively raise ASR, supported by an information-theoretic argument; and two existing defenses, effective on direct requests, are weakened by VEIL.

Critical strengths
  1. The attack mechanism captures a real and fundamental architectural flaw. The observation–action gap, where the agent judges safety based on what it observes yet acts on the true underlying state, is an attack surface specific to phone-use agents and conceptually very clean.
  2. Real-device validation is the biggest plus. 600 virtual–real device pairs, 91.5% agreement, and κ=0.80 strongly address the question most often raised against benchmark papers, namely whether the virtual environment reflects real-world risks.
  3. The attack components are cleverly designed and low-cost. Screenshot matching uses SIFT+RANSAC / normalized cross-correlation, runs on pure CPU in tens of milliseconds with no model inference required, and the paper deliberately argues that a masked screenshot is not itself a sign of an attack (cloud-based MLLMs already routinely mask for privacy). This makes the attack both practical and hard to defend against by merely “detecting that an edit occurred.”
Critical weaknesses
  1. Insufficient defense exploration weakens the constructive value. Only two off-the-shelf defenses (a safety reminder and OS-Sentinel) are tested, and both fail under VEIL. The paper proposes no new defense idea targeting the observation–action gap, not even conceptual ones (e.g., app-side state verification, cross-subtask consistency checks, action–observation consistency validation). An attack paper claiming to "promote safety" should at least outline viable defensive directions.

  2. Attack novelty may be a combination of known techniques. Task decomposition plus multi-turn history isolation is essentially close to existing jailbreak ideas such as Crescendo; the genuinely new element is "screenshot masking" as visual-channel sanitization. Reviewers may see the core contribution as a framing of the observation–action gap rather than a fundamentally new attack primitive. The incremental novelty needs to be argued more clearly.

  3. On-device ASR is "attempted ASR," which risks overstatement. To prevent real harm, the on-device experiments intercept actions without actually submitting them, so on-device ASR measures whether the agent attempts execution, not whether the operation takes effect. Platform-side rate limiting, CAPTCHAs, and risk controls are not evaluated at all (the authors concede in A.1 that no forced logouts or CAPTCHAs were encountered during testing, so they were not assessed). The "feasibility of abuse at scale" conclusion therefore skips the interception stage most likely to occur in real deployment.

  4. Weak rigor in the human speed baseline. Only 3 volunteers, with each task timed only on the second (post-familiarization) run—a tiny sample with no reported variance. Figures such as "1.46× the human baseline" rest on this and are not very convincing.

  5. Room for improvement in the judge and statistical rigor. Task success is determined by scripts from app state (a good choice), but refusal judgment relies on humans plus DeepSeek-V4-Flash (97.7% agreement, but no κ and no disagreement analysis).

Questions for authors
  1. Have you explored any defense that is more principled with respect to the observation–action gap? For example, app/OS-side action–intent consistency verification, cross-subtask state-correlation checks, or requiring safety judgments to be based on the true app state rather than the view the model observes? Even a conceptual discussion would substantially strengthen the contribution.

  2. On-device ASR is "attempted" (actions are intercepted and never actually submitted). If actions were truly submitted, to what extent would platform-side risk controls (rate limiting, CAPTCHAs, account bans, report-validity review) reduce the actual abuse success rate? Could you provide even a small-scale evaluation that actually reaches the platform?

  3. Compared with Crescendo (multi-turn gradual escalation) and general task-decomposition jailbreaks, is VEIL's core increment simply "sanitization of the screenshot visual channel"? Could you quantify this more clearly through ablation—specifically, how large is the gap from existing multi-turn jailbreak methods when screenshot editing is removed and only decomposition + history isolation are used?

  4. Is the 0% refusal rate of GUI-Owl / Qwen3-VL because these specialized GUI models have no safety alignment at all, or because the harness/prompt format fails to trigger their safety behavior? This bears on whether the conclusion should be attributed to the "model" or to the "evaluation setup."

  5. Can a design with 3 volunteers and single-run timing support the speed conclusions? Could you add more participants and report variance, or at least state the speed claims more conservatively?

  6. The 97.7% agreement between humans and DeepSeek-V4-Flash was measured on how many samples and which configurations? What is κ? Do disagreement cases concentrate in a particular task type?

  7. Once the code and data are released, could these 150 policy-aligned cases be used directly for real abuse, or be absorbed into model training data and thereby lose their validity? Are there plans for access control or a gated release?

6 comments
AuthorsOpenReview ↗

Responses (1/3)

We thank Reviewer GBbj for the thorough review and the constructive suggestions. We respond to each question in detail below.

  • Q1 and weakness1: On defense exploration.

Have you explored any defense that is more principled with respect to the observation–action gap? For example, app/OS-side action–intent consistency verification, cross-subtask state-correlation checks, or requiring safety judgments to be based on the true app state rather than the view the model observes? Even a conceptual discussion would substantially strengthen the contribution.

Thank you for this valuable and constructive suggestion. We feed the judge model GPT-6-Sol the agent's full execution trajectory, including every action (e.g., tapping and typing) together with the corresponding app state as unedited screenshots, and ask it whether the task is harmful. This simulates a cross-subtask state check on the app/OS side. On VEIL executions with Qwen3.8-27B (ASR=94.7%), the judge identifies 63.3% of all 150 task executions as harmful, suggesting that app-side state verification and cross-subtask consistency checks are promising directions.

However, such defenses still face practical challenges. Subtasks issued across time need to be linked before they can be checked together, and social media platforms handle an enormous volume of user activity, which makes judging every trajectory costly. Evaluating app-side and system-level defenses is beyond the scope of this work, but we have added this discussion to the revised manuscript and hope it inspires future work.

  • Q2 and weakness3. On-device ASR is "attempted" (actions are intercepted and never actually submitted). If actions were truly submitted, to what extent would platform-side risk controls reduce the actual abuse success rate? Could you provide even a small-scale evaluation that actually reaches the platform?

Thank you for this valuable suggestion. We conducted an additional real-device evaluation with actual submission on 137 reversible tasks under VEIL with Qwen3.8-27B (excluding 13 irreversible tasks, such as reporting). All content submitted by the agent in this evaluation was successfully published and visible to another account, without being blocked or removed by the platforms. We have added these results and discussion to the revised manuscript and hope they inspire future work.

  • Additional clarification on Weakness3. On real device evaluation.

We would also like to clarify that by "at scale" we mean automating the GUI interactions that such abuse has so far relied on human labor to perform. Our focus is on the extent to which phone-use agents reduce this labor, with their limited safety alignment raising concerns about misuse. Rate limiting and account bans also constrain abuse carried out by humans. Evaluating these controls is beyond the scope of our work. To avoid potential misunderstanding, we have revised the related claims in the manuscript to the "potential for automated misuse in real-world apps" and discuss these factors in the limitations.

AuthorsOpenReview ↗

Responses (2/3)

  • Q3 and weakness2. Could you quantify this more clearly through ablation, specifically how large the gap from existing multi-turn jailbreak methods is when screenshot editing is removed and only decomposition and history isolation are used?

We compare our decomposition and history isolation with Crescendo (a multi-turn gradual-escalation attack) and DrAttack (a task-decomposition jailbreak). The results are shown below.

ModelBaselineCrescendoDrAttackOurs (without screenshot editing)
GPT-6-Astra52.0%52.7% (+0.7)65.3% (+13.3)84.7% (+32.7)
Qwen3.8-27B67.3%70.0% (+2.7)75.3% (+8.0)91.3% (+24.0)

Even without screenshot editing, our decomposition with history isolation clearly outperforms Crescendo and DrAttack. This shows that VEIL's gains do not come only from sanitizing the screenshot visual channel.

  • Additional clarification on Weakness2. About the novelty of our work

We would like to clarify that our novelty lies in benchmarking an overlooked real-world misuse risk and jailbreaking frontier models through an underexplored attack surface.

Existing benchmarks lack apps and functions needed to evaluate social media misuse by phone-use agents, such as reporting. Our benchmark covers them through a high-fidelity simulator and 150 test cases manually built from platform policies. With it, we find substantial risks even without attacks, with GPT-6-Astra and Claude Fable 5 completing 52.0% and 42.7% of harmful tasks.

On the attack side, we identify and exploit an underexplored attack surface in phone-use agents, namely the observation-state gap. Exploiting this gap enables effective attacks on models with stronger safeguards, which task decomposition and history isolation alone cannot achieve. As shown in the General Response, on GPT-6-Astra with a safety prompt, decomposition with history isolation raises ASR by 17%, whereas adding screenshot editing adds 43.3%.

  • Q4. Is the 0% refusal rate of GUI-Owl / Qwen3-VL because these specialized GUI models have no safety alignment at all, or because the harness/prompt format fails to trigger their safety behavior? This bears on whether the conclusion should be attributed to the "model" or to the "evaluation setup."

Thank you for this insightful question. Our additional experiments show that both the evaluation setup and the model contribute to the 0% refusal rate.

We evaluate two configurations. In S1, the default setting in the paper, each model runs with its official system prompt [1,2] for phone-use tasks. In S2, we remove the GUI-related system prompt and send each harmful request as a plain conversational query.

ModelRefusal rate (S1)Refusal rate (S2)
GUI-Owl-1.5-8B0.0%41.3%
Qwen3-VL-8B0.0%78.0%

Without the GUI system prompt, the refusal rates of GUI-Owl and Qwen3-VL rise to 41.3% and 78.0%, respectively. This indicates that the phone-use prompt format affects safety behavior.

Under S2, GUI-Owl also refuses far less often than Qwen3-VL. A possible reason is that GUI-Owl is fine-tuned from Qwen3-VL on GUI data [2], and prior work shows that fine-tuning even on benign data can degrade safety alignment [3]. We have added this analysis to the revised manuscript.

[1] Qwen3-VL Technical Report

[2] Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents

[3] Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!

AuthorsOpenReview ↗

Responses (3/3)

  • Q5 and weakness4. Can a design with 3 volunteers and single-run timing support the speed conclusions? Could you add more participants and report variance, or at least state the speed claims more conservatively?

Thank you for this valuable suggestion. We extended the study to five volunteers, each performing every task three times. The first run is used to familiarize the volunteer with the task, and the last two are timed. The resulting human execution time is 42.57 ± 4.1 seconds per task (mean and standard deviation across the ten runs).

We have also stated the speed claims more conservatively. For example, instead of comparing the agent's speed with humans, we now report the raw execution times for reference to avoid potential misunderstandings.

  • Q6 and weakness5. The 97.7% agreement between humans and DeepSeek-V4-Flash was measured on how many samples and which configurations? What is κ? Do disagreement cases concentrate in a particular task type?

The 97.7% agreement is measured on all runs that the evaluation scripts classify as unsuccessful attacks (2,438 in total) across every experiment in the paper. Human labels were produced by two annotators working independently, who agreed on 98.9% of runs, and a third reviewer resolved the remaining disagreements.

Cohen's κ between DeepSeek-V4-Flash and the human labels is 0.92.

The disagreements do not concentrate in a single category. Across Harassment, Fraud and Scams, Illicit Transactions, Hate and Extremism, Dangerous Behavior, Privacy Violations, Platform Manipulation, and Misinformation, the numbers of disagreements are 12, 11, 4, 6, 5, 4, 10, and 4, respectively.

  • Additional clarification on Weakness5. Human annotation of refusal rates.

All refusal rates reported in the paper come from the human annotation described above, not from DeepSeek-V4-Flash (in Section 3.3). We report the DeepSeek-V4-Flash results only as a reference for future work, to show how accurate an LLM judge would be for researchers who cannot run human annotation.

  • Q7. About the release of code and dataset.

Thank you for this valuable suggestion. We plan to make the simulator publicly accessible. For the jailbreak samples, we follow common practice in safety research and release them only to applicants who agree to use them for research purposes and not to train models that are evaluated on this benchmark.

  • On presentation.

We thank the reviewers for their feedback on presentation. We have fixed the noted typos and inconsistent citation styles, and we will carefully polish the writing throughout the paper to improve clarity and readability in the revised manuscript.

Reviewer_ZhizOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on evaluating the safety risks of phone-use agents in social media misuse, where agents can automate harmful activities through native mobile interfaces. Existing benchmarks lack realistic environments and comprehensive coverage of social media abuse scenarios. To address these limitations, the authors introduce BadPhoneAgent, a benchmark featuring high-fidelity environments across 11 social media apps, 18 functions, and 150 harmful tasks spanning 8 categories. The paper further proposes View Editing for Intent Laundering (VEIL), a jailbreak method that exploits the gap between an agent's observations and the actual phone state by decomposing harmful requests into seemingly benign subtasks, isolating interaction histories, and masking relevant visual context. Experiments across ten commercial and open-source models reveal substantial misuse risks, with VEIL significantly increasing attack success rates and reducing safety refusals. Additional real-device evaluations show 91.5% agreement with virtual outcomes, supporting the benchmark's practical relevance.

Critical strengths
  1. This paper rises a realistic and well-validated benchmark that supports 11 Chinese and English social media apps and 18 functions, including reporting, danmaku, and comment replies, which are largely missing from previous mock-app environments. The benchmark is also carefully constructed, with 150 harmful tasks grounded in platform-specific policies and reviewed by three independent reviewers, achieving 92% agreement. I particularly appreciate that task success is evaluated directly from application states rather than relying on an LLM judge, while safety refusals are manually annotated. Moreover, the authors validate the environment through 600 paired evaluations on real devices, achieving 91.5% outcome agreement (κ = 0.80) and 98.5% refusal agreement. These efforts make the benchmark a valuable contribution to evaluating phone-use agent safety.
  2. The paper reveals an important weakness in existing phone-use agents, which their safety behavior varies considerably across different types of harmful requests. As shown in Section 5.1, GPT-6-Astra and Claude Fable 5 refuse all or most dangerous-behavior requests (100% and 77.8%, respectively), but neither refuses platform-manipulation requests. Furthermore, four evaluated open-source models show no safety refusals across the benchmark. These findings suggest that current models may be better at recognizing explicitly harmful content than identifying the misuse of seemingly ordinary social media functions, such as abusive reporting or engagement manipulation. I find this observation particularly valuable, as it highlights a concrete safety gap that deserves more attention in future agent development.
Critical weaknesses

1. The novelty of VEIL is not sufficiently distinguished from existing compositional jailbreak methods.

The core idea of decomposing harmful requests into seemingly benign subtasks has been explored in prior works, such as @article{jones2024adversaries, title={Adversaries can misuse combinations of safe models}, author={Jones, Erik and Dragan, Anca and Steinhardt, Jacob}, journal={arXiv preprint arXiv:2406.14595}, year={2024} } and @inproceedings{li2024drattack, title={DrAttack: Prompt decomposition and reconstruction makes powerful LLMs jailbreakers}, author={Li, Xirui and Wang, Ruochen and Cheng, Minhao and Zhou, Tianyi and Hsieh, Cho-Jui}, booktitle={Findings of the Association for Computational Linguistics: EMNLP 2024}, pages={13891--13913}, year={2024} }. However, these methods are neither adequately discussed nor compared against. Moreover, Table 1 shows that most of VEIL's improvement comes from task decomposition and history isolation. For GPT-6-Astra, these two components increase ASR from 52.0% to 84.7%, while the additional screenshot editing only improves it to 88.0%. Although combining these techniques in phone-use agents is meaningful, the distinct methodological contribution of VEIL remains insufficiently demonstrated.

2. Lack of related work.

The related-work section overlooks several relevant studies on mobile GUI agent security, such as @inproceedings{chen2026ghostei, title={GhostEI-Bench: Do Mobile Agent Resilience to Environmental Injection in Dynamic On-Device Environments?}, author={Chen, Chiyu and Song, Xinhao and Chai, Yunkai and Yao, Yang and Zhao, Haodong and Li, Lijun and Li, Jie and Teng, Yan and Liu, Gongshen and Wang, Yingchun}, booktitle={International Conference on Learning Representations}, volume={2026}, pages={131563--131587}, year={2026} } and @article{chen2026mobileworldsafety, title={MobileWorldSafety: Benchmarking GUI Agent Safety Against Environmental Injection Attacks in Android Apps}, author={Chen, Sujin and Li, Lijun and Du, Tianyi and Shao, Jing}, journal={arXiv preprint arXiv:2608.17659}, year={2026} }. Although these works investigate different threat models, they are relevant to positioning BadPhoneAgent within the existing mobile agent safety evaluation landscape. Together with the missing discussion of compositional jailbreak research, these omissions make it difficult to fully assess the novelty of the proposed benchmark and attack framework. A more comprehensive comparison would help clarify the paper's contributions relative to existing studies.

Questions for authors

Q1. How does VEIL fundamentally differ from existing compositional jailbreak approaches, such as DrAttack? In particular, what new attack capabilities does VEIL introduce beyond decomposing harmful requests into seemingly benign subtasks?

Q2. Table 1 shows that task decomposition and history isolation already account for much of VEIL's improvement. For example, GPT-6-Astra's ASR reaches 84.7% before screenshot editing and increases to 88.0% afterward. Could the authors provide a more detailed analysis of cases where screenshot editing is necessary for a successful attack? This would help establish the contribution of the proposed observation-state mismatch mechanism.

Q3. Since harmful final actions are intercepted during real-device evaluation (§5.3, Appendix A.7), to what extent do the reported results support the claim of large-scale social media misuse?

2 comments

We are grateful to the reviewer for the detailed and thoughtful review. Please find our point-by-point responses below.

  • Q1 and Weakness 1. How does VEIL fundamentally differ from existing compositional jailbreak approaches such as DrAttack, and what new attack capabilities does it introduce beyond decomposing harmful requests into seemingly benign subtasks?

VEIL differs fundamentally in that it identifies and exploits an underexplored observation–state gap in phone-use agents to carry out effective attacks. We have added a comparison with DrAttack, and the results show that compositional jailbreak approaches alone struggle to improve ASR, even on open-source models. By exploiting this gap, VEIL raises ASR by 36.0% and 27.3% on the two models, well above the 13.3% and 8.0% achieved by DrAttack.

ModelBaselineDrAttackVEIL
GPT-6-Astra52.0%65.3% (+13.3)88.0% (+36.0)
Qwen3.8-27B67.3%75.3% (+8.0)94.7% (+27.3)

The new capability VEIL introduces is a stronger attack against models with more conservative safeguards. As shown in Q2, full VEIL raises the ASR of Claude Fable 5 and Claude Opus 5.5 by 22.0% and 32.0%, whereas task decomposition with history isolation gives only 4.0% and 5.3%. Similarly, when GPT-6-Astra is protected by a prompt defense, full VEIL improves ASR by 60.7%, while task decomposition with history isolation gives only 17.3%.

  • Q2 and Weakness 1. For example, GPT-6-Astra's ASR reaches 84.7% before screenshot editing and increases to 88.0% afterward. Could the authors provide a more detailed analysis of cases where screenshot editing is necessary for a successful attack?

Screenshot editing becomes particularly important when the model has stronger safeguards. For example, once GPT-6-Astra is equipped with a safety prompt, its ASR reaches only 42.0% before screenshot editing but rises to 85.3% afterward.

We also add results on two models with more conservative safeguards, Claude Fable 5 and Claude Opus 5.5. Taking Claude Opus 5.5 as an example, full VEIL raises ASR by 32.0%, whereas task decomposition with history isolation gives only 5.3%.

ModelBaselineVEIL w/o screenshot editingFull VEIL
GPT-6-Astra (+ safety prompt)24.7%42.0% (+17.3)85.3% (+60.7)
Gemini 3.8 Flash (+ safety prompt)24.7%50.0% (+25.3)82.7% (+58.0)
Claude Fable 542.7%46.7% (+4.0)64.7% (+22.0)
Claude Opus 5.535.3%40.7% (+5.3)67.3% (+32.0)

Table: Additional ablation on screenshot editing when the model has more safeguards.

  • Q3. Since harmful final actions are intercepted during real-device evaluation, to what extent do the reported results support the claim of large-scale social media misuse?

Thank you for this constructive comment. To address this question, we conducted an additional real device evaluation ( Qwen3.8-27B + VEIL), actually submitted actions for 137 reversible tasks (excluding 13 irreversible tasks, such as reporting). For all 137 tasks, the outcomes matched those recorded before interception, suggesting consistency with the intercepted results.

Furthermore, by "large-scale" we mean automating the GUI interactions that such abuse has so far relied on human labor to perform. Our focus is on how far phone-use agents reduce the human labor such abuse requires, and their limited safety alignment makes this a risk worth noting. To avoid potential misunderstanding, we have revised the related claims in the manuscript to the "potential for automated misuse in real-world apps".

  • Weakness 2. On related work.

Thank you for this constructive comment. We now discuss GhostEI-Bench and MobileWorldSafety in the related work. Their threat model concerns injected content in the external environment affecting benign users, whereas ours concerns the evaluation and jailbreaking of a user who deliberately misuses a phone-use agent. For compositional attacks [1, 2], we add a quantitative comparison with DrAttack, and the results show that compositional jailbreak approaches alone struggle to improve ASR, even on the open-source model Qwen3.8-27B (see our responses to Q1 and Q2). We discuss this in the related work and highlight that the difference lies in exploiting the underexplored observation–state gap to jailbreak effectively, even on models such as GPT-6-Astra and Claude Fable 5.

[1] DrAttack: Prompt decomposition and reconstruction makes powerful LLMs jailbreakers

[2] Adversaries can misuse combinations of safe models

I appreciate the authors' clarifications and keep my positive rating unchanged.

Reviewer_eVvuOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces BadPhoneAgent, a benchmark and virtual environment for evaluating social media misuse by phone-use agents, and VEIL, a jailbreak attack that exposes an attack surface specific to phone-use agents. BadPhoneAgent provides 150 manually constructed test cases, and the environment spans 11 social media apps and 18 abuse relevant functions. VEIL exploits the underexplored gap between the model’s observation and the phone state, and reaches attack success rates (ASR) of 88% on GPT-6-Astra and 64.7% on Claude Fable 5, while these models refuse only 8.0% and 1.3% of requests, respectively.

Critical strengths

• The problem is timely and important, given that phone-use agents are becoming increasingly powerful and popular.

• The benchmark is solid with 11 social media apps, 18 abuse relevant functions, and 150 manually constructed test cases of 8 categories and 35 subcategories.

• VEIL attack provides a new attack surface by exploiting the gap between the observation view and the true phone state.

• The evaluation is solid with real-device validation and ablations.

Critical weaknesses

• VEIL assumes that the attacker can run the framework locally and modify the inputs it sends to the model. Such an assumption may be too strong for commercial agent products.

• Some typos should be fixed, such as extra whitespaces in Introduction and Section 3.3, and missing “.” in Related Works. The citation styles are not consistent, e.g., in Introduction vs. in Related Works.

• This paper https://arxiv.org/abs/2606.27944 looks closely related but is not cited/discussed.

Questions for authors

• Is the assumption “attacker can run the framework locally and modify the inputs it sends to the model” too strong for commercial agent products?

• How does this work compare with the arXiv:2606.27944 paper?

2 comments

We sincerely thank the reviewer for the time and valuable feedback. We address each concern below.

  • Q1 and Weakness 1. Is the assumption that "the attacker can run the framework locally and modify the inputs it sends to the model" too strong for commercial agent products?

We agree that this assumption is strong for commercial products, since our work mainly targets open-source agent frameworks. Such frameworks are widely adopted for phone-use agents. For example, Open-AutoGLM has over 26k GitHub stars [3], and they allow users to freely modify the inputs sent to the model. However, VEIL can also be applied to commercial desktop agents, such as Codex and Claude Code, when they operate a phone connected via USB, capturing screenshots and executing actions through ADB.

In this setting, history isolation uses a new session for each subtask. Screenshot editing can be implemented through hooks[1,2], a mechanism offered by many commercial agents, including Codex and Claude Code. A hook binds a fixed event, such as taking a screenshot, to a user-specified script, such as screenshot editing. This lets us edit each screenshot before it is passed to the model.

In the revised manuscript, we detail the implementation in Appendix A.6, including code in Figure 6 and a Codex case study in Figure 7. We also refine the threat model to clarify the attacker’s ability to edit screenshots supplied to the model and isolate context across subtasks.

[1] https://learn.chatgpt.com/docs/hooks

[2] https://code.claude.com/docs/en/hooks

[3] https://github.com/zai-org/Open-AutoGLM

  • Q2 and Weakness 3. Comparison with the arXiv paper.

Thank you for pointing us to this paper. It is concurrent work, but it differs from ours in scope, evaluation methodology, and attack methodology. In terms of scope, we focus on social media misuse, a high-impact setting that existing benchmarks rarely cover. Our benchmark covers a broader range of social media functions across both Chinese- and English-language apps, whereas arXiv:2606.27944 focuses on Chinese-language apps and basic social interactions. In terms of evaluation, we provide a simulated, dynamic, and reproducible environment in which success is measured directly from app state, enabling scalable and repeated evaluation. arXiv:2606.27944 instead relies on manual on-device testing, which makes its evaluation costly to reproduce and to extend to new models. In terms of attacks, arXiv:2606.27944 evaluates only plain natural-language harmful instructions, while we exploit an underexplored observation–state gap in phone-use agents to jailbreak frontier models such as Claude Fable 5 and GPT-6-Astra and substantially reduce their refusal rates. We have added this discussion to the Related Work section of the revised manuscript.

  • Weakness 2. Typos and citation style.

Thank you for pointing out these typos. We have carefully proofread the revised manuscript, corrected them, and made the citation style consistent throughout.

Thank the authors for the clarificaitons. I retain my positive rating.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We first thank all reviewers for their time and constructive feedback. We are grateful for the reviewers’ positive feedback on:

  • Our realistic and well-validated benchmark for a timely and important problem (eVvu, Zhiz), with real-device validation that strongly addresses its real-world fidelity (GBbj) and previously missing abuse-relevant functions (Zhiz,GBbj).

  • Our VEIL attack, which captures a real and fundamental architectural flaw (GBbj).

  • Our particularly valuable observation, which highlights a concrete safety gap that deserves more attention in future agent development (Zhiz).

Below we address the shared concern about the VEIL attack (Zhiz, GBbj).

  • Comparison with multi-turn and compositional jailbreaks.

VEIL differs fundamentally in that it uncovers and exploits the underexplored observation–state gap. Compared with the multi-turn attack Crescendo and the compositional attack DrAttack, VEIL achieves a substantially higher ASR, e.g., +36% on GPT-6-Astra versus +0.7% and +13% for Crescendo and DrAttack, respectively.

ModelBaselineCrescendoDrAttackVEIL
GPT-6-Astra52.0%52.7% (+0.7)65.3% (+13.3)88.0% (+36.0)
Qwen3.8-27B67.3%70.0% (+2.7)75.3% (+8.0)94.7% (+27.3)
  • Necessity of exploiting the observation–state gap.

We ablate screenshot editing under stronger safety mechanisms, covering models with an additional safety prompt and models with stronger built-in safety alignment. Screenshot editing keeps VEIL effective both when an extra defense is deployed (+60% vs. +17% on GPT-6-Astra with a safety prompt) and on models with stronger safeguards (+32% vs. +5% on Claude Opus 5.5).

ModelBaselineVEIL w/o screenshot editingFull VEIL
GPT-6-Astra (+ safety prompt)24.7%42.0% (+17.3)85.3% (+60.7)
Gemini 3.8 Flash (+ safety prompt)24.7%50.0% (+25.3)82.7% (+58.0)
Claude Fable 542.7%46.7% (+4.0)64.7% (+22.0)
Claude Opus 5.535.3%40.7% (+5.3)67.3% (+32.0)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.