acceptodds
Under review as a conference paper at ICLR 2027

Do Phone-Use Agents Respect Your Privacy?

Abstract

We study whether phone-use agents respect privacy while completing benign mobile tasks. This question has remained hard to answer because privacy-compliant behavior is not operationalized for phone-use agents, and ordinary apps do not reveal exactly what data agents type into which form entries during execution. To make this question measurable, we introduce MyPhoneBench, a verifiable evaluation framework for privacy behavior in mobile agents. We operationalize privacy-respecting phone use as permissioned access, minimal disclosure, and user-controlled memory through a minimal privacy contract, iMy, and pair it with instrumented mock apps plus rule-based auditing that make unnecessary permission requests, deceptive re-disclosure, and unnecessary form filling observable and reproducible. Across five frontier models on 10 mobile apps and 300 tasks, we find that task success, privacy-compliant task completion, and later-session use of saved preferences are distinct capabilities, and no single model dominates all three. Evaluating success and privacy jointly reshuffles the model ordering relative to either metric alone. The most persistent failure mode across models is simple data minimization: agents still fill optional personal entries that the task does not require. These results show that privacy failures arise from over-helpful execution of benign tasks, and that success-only evaluation overestimates the deployment readiness of current phone-use agents. All code, mock apps, and agent trajectories are publicly available at https://anonymous.4open.science/r/MyPhoneBench-3BD7.

Primary area:
datasets and benchmarks
Keywords:
Phone-use Agents, Mobile GUI Agents

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_VVCFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents MyPhoneBench, a benchmark for evaluating privacy behavior in phone-use agents during benign tasks. It introduces the iMy privacy contract and instrumented Android applications to assess over-permissioning, unnecessary data disclosure, and optional-field filling. Experiments on five frontier models across 10 apps and 300 tasks reveal trade-offs between task success and privacy compliance, with form minimization emerging as the most challenging dimension. The paper also examines cross-session preference reuse and highlights limitations of success-only agent evaluation.

Critical strengths
  1. Important and timely research problem. The paper addresses privacy risks in GUI-based phone-use agents, an important yet underexplored setting. Its focus on privacy violations arising during benign task execution, rather than adversarial attacks, provides a valuable perspective on agent privacy evaluation.

  2. Fine-grained and verifiable privacy auditing. The paper introduces field-level auditing that captures agents' data-handling behavior during execution, including unnecessary permission requests, repeated disclosures, and optional-field filling. This provides better visibility into privacy violations than conventional task-success-based evaluations.

  3. Substantial engineering effort and reproducibility. The authors develop 10 instrumented Android mock applications, integrate a privacy contract and structured privacy probes, and implement deterministic evaluation mechanisms. The resulting benchmark represents a meaningful engineering effort and provides a reproducible foundation for studying privacy behavior in mobile agents.

Critical weaknesses
  1. Limited validity of the privacy definition. The benchmark relies heavily on predefined rules, particularly the LOW/HIGH classification and task-necessity labels. While the authors acknowledge that this is a simplified privacy taxonomy, I am still concerned about how these rules are established. Who determines whether certain data should be classified as LOW or HIGH? How is task necessity defined? Without further justification or validation, it is unclear whether the benchmark captures realistic privacy expectations or mainly evaluates compliance with its own predefined rules.

  2. Potential bias in privacy evaluation. Privacy scores are calculated only over the probes encountered during execution. This means that agents reaching different stages of a task may be evaluated on different privacy situations. Although PQSR partially addresses the issue of early task failures, it does not fully resolve differences in probe exposure, even among successful agents. I would like to see more analysis of how task progress, UI layout, and probe placement affect the reported privacy scores and model rankings.

  3. The analysis lacks depth. Most findings are based on comparisons of scores across models and applications, with limited investigation into why these differences occur. For example, the authors attribute unnecessary optional-field filling to a completion-oriented bias, but this could also result from misunderstanding the UI, failing to recognize optional fields, or following general task-completion instructions. The current experiments demonstrate that these failures occur, but provide limited evidence explaining their causes. I expected more controlled analysis to draw meaningful conclusions beyond model rankings.

  4. Missing important baselines. The paper evaluates five frontier models using essentially the same iMy contract and agent framework, but does not explore whether simple privacy-aware interventions could improve their behavior. For example, how would the results change with explicit privacy-aware prompting or a rule-based privacy filter? Without such comparisons, it is difficult to tell whether the observed failures reflect limitations of the models themselves or simply insufficient privacy guidance during execution. These baselines would also help clarify what the benchmark teaches us about improving agent privacy.

  5. Unclear connection between cross-session preference reuse and privacy evaluation. I find the inclusion of cross-session preference reuse somewhat difficult to justify within the main scope of this privacy benchmark. The experiments mainly test whether agents can save and correctly reuse user preferences, which seems closer to evaluating memory functionality than privacy protection. An agent could achieve a high score on this metric while still storing information without proper consent or ignoring deletion requests. If user-controlled memory is intended to be a core privacy dimension, I would expect evaluations of consent, deletion, modification, or unauthorized retention. Otherwise, it is unclear why successful preference reuse should be treated as a privacy-related capability alongside over-permissioning, trap resistance, and form minimization.

Questions for authors

Please see the Critical Weaknesses section for specific concerns. I appreciate the authors' effort to make agent privacy measurable through concrete form-filling scenarios. However, this simplification also limits the benchmark's ability to capture the complexity of privacy risks in real-world agent interactions. More importantly, the analysis remains largely descriptive, with limited insight into why privacy failures occur, while privacy evaluation is also heavily intertwined with task completion. Overall, despite the meaningful problem setting and engineering effort, I believe the paper still falls short of the depth and rigor expected of a high-quality benchmark.

Reviewer_2iVyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces MyPhoneBench, a benchmark for evaluating privacy-related behavior in mobile GUI agents during benign task execution. It combines the iMy privacy contract with instrumented Android applications and deterministic auditing of unnecessary data-access requests, re-disclosure, and optional personal-field filling. The authors report 300 tasks across 10 mock applications and evaluate five frontier models on task success, privacy-qualified task completion, and cross-session preference reuse. The results reveal differences across these capabilities, with unnecessary optional-field filling remaining a common failure mode.

Critical strengths

Auditable evaluation framework. The explicit privacy contract, instrumented applications, and deterministic checks make otherwise difficult-to-observe data-handling behavior measurable. In particular, recording intermediate form edits, including entries that are never submitted, supports privacy auditing beyond final-state task success. Jointly reporting task completion and privacy outcomes is also useful when models differ in their ability to reach privacy-sensitive interactions.

Critical weaknesses

1. Incomplete positioning relative to prior smartphone privacy research. Section 6 does not discuss SAPA-Bench [1], an earlier peer-reviewed benchmark covering privacy recognition, localization, sensitivity assessment, and risk warnings. MyPhoneBench's focus on auditable execution and voluntary data minimization is different and potentially valuable, but a targeted comparison is needed to establish the remaining evaluation gap and the additional value of iMy and field-level auditing. This uncertainty affects my assessment of the contribution's significance. MLA-Trust [2], anonymization-based protection [3], and screenshot-based leakage/profiling evaluation [4] provide further context with different objectives or threat models. I suggest these as additional background; missing comparisons to arXiv-only or contemporaneous work are not a basis for my recommendation.

2. Limited validation of the operational privacy judgments and their generality. Sections 2.1 and 2.4 motivate the minimal HIGH/LOW contract and three probes, and Section 2.3 describes schema/template checks and manual review. However, evidence that these judgments capture contextual and user-dependent privacy requirements remains limited. Data sensitivity and the necessity of disclosure may vary with the task and the user's preferences; the PQSR threshold sweep does not test sensitivity to such policy choices.

The mock applications and always-grant simulator deliberately isolate voluntary overreach, as the authors acknowledge. The experiments nevertheless leave behavior under permission denial or differing user preferences unresolved. Similarly, five models in a shared framework support a controlled comparison, but provide limited evidence about specialized GUI agents or alternative agent frameworks. These concerns restrict how broadly the findings can be interpreted. A focused validation of the privacy judgments would be more informative than simply increasing the number of models or tasks.

Overall assessment. I lean toward weak rejection. The auditable framework is useful, but the contribution's positioning and the validation of its operational privacy judgments need strengthening. A convincing clarification of the added evaluation value and supporting evidence for the privacy judgments could change my recommendation.

References

[1] Lin, Z., Li, J., Pan, S., Shi, Y., Yao, Y., & Xu, D. (2026). Mind the Third Eye! Benchmarking Privacy Awareness in MLLM-powered Smartphone Agents. Proceedings of the AAAI Conference on Artificial Intelligence, 40(42), 35626–35634. https://doi.org/10.1609/aaai.v40i42.40874

[2] Yang, X., Chen, J., Luo, J., Fang, Z., Dong, Y., Su, H., & Zhu, J. (2025). MLA-Trust: Benchmarking Trustworthiness of Multimodal LLM Agents in GUI Environments. arXiv:2506.01616. https://arxiv.org/abs/2506.01616

[3] Zhao, L., Zou, Z., Li, S., & Liu, Z. (2026). Anonymization-Enhanced Privacy Protection for Mobile GUI Agents: Available but Invisible. arXiv:2602.10139. https://arxiv.org/abs/2602.10139

[4] Cen, Q., Cong, T., Song, D., He, X., Song, J., Xu, K., & Li, Q. (2026). PriMobiBench: Characterizing Visual Privacy Leakage in VLM-Driven Mobile GUI Agents. arXiv:2609.13873. https://arxiv.org/abs/2609.13873

Questions for authors
  1. Could the authors clarify MyPhoneBench's additional evaluation capability relative to prior smartphone privacy-awareness benchmarks such as SAPA-Bench, identifying the behaviors that iMy and field-level execution auditing make measurable? This would help establish the significance of the contribution.

  2. Beyond schema/template checks and manual review, what evidence validates HIGH/LOW labels and task-specific data necessity against user expectations? Are there existing results or a small targeted analysis showing whether alternative plausible user preferences change the principal findings?

Permission-denial experiments and specialized open-source GUI-agent baselines would be useful extensions. A comprehensive new evaluation is not required during rebuttal; focused evidence addressing the two questions above would be more useful for reassessing the submission.

Reviewer_daHVOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposed the privacy implications of phone-use agent on a simulated environment. The paper use 10 mock apps from 9 domains with 300 tasks to show how the agents respect user privacy during the task execution instead of just task success rate. The key finding reveals that different metrics results in different models/agents as winner with Form minimization as one of the major privacy risks.

Critical strengths

The paper studies the Phone-use privacy implication which is timely and well-motivated. The simulated environment allows the trackable task execution in the form-based 10 mock apps. The paper evaluates both the privacy risks during the task execution by an llm agent and task success rate which is usually measured at the end, to understand how both the task success and privacy could be correlated in the phone-use. The privacy contract is light weight and minimal for easy to evaluate during the task execution.

Critical weaknesses

Major weakness:

  1. The paper is lack several key details that are required for understanding the impact of this study. Studied on a very limited scope both in the phone usability (such as form-based, naive version of the apps) and privacy risks (3 probes: OP, TR, FM).
  2. Work only on the tractable form-based privacy compliances. Are there input filed beyond textboxes eg., drop-down list, multi-page forms etc? Does the privacy implications in the description textbox (Visit details in Fig 3) also included in the privacy evaluation such as including the phone number in the description to call back?
  3. Paper uses a narrowed set of phone use privacy risks/probes. Many risks such as privacy risk of the stored preference in iMy, filling work phone number in the food order app, retaining information without appropriate consent, etc could be potential even in this limited simulation environment.
  4. I would like to see details on what the privacy contract would look like? Would it be based on the task ? Does the agent know the LOW/HIGH labels of these keys from the iMy privacy contract? How does it differ from the visual iMy shown in Fig 3a.
  5. How does this study reflect the real-world phone-use case? How far the results generalize beyond the controlled mock apps? Even a smaller validation study using realistic application workflows or independently designed simulated environments beyond andriodworld could help establish whether the observed behavior transfers beyond the constructed benchmark.
  6. The table shows that Qwen 3.5 Plus has a privacy-qualified success rate of 47.6%, compared with 47.2% for Claude Opus 4.6 and 45.2% for Kimi K2.5 in PQSR Table 1. These numbers are very close to make clear judgement. Adding statistical significance could help here
  7. Introduction mentioned AutoGLM, UI-TARS, and OpenClaw but not tested in Table 1.
  8. Definition of LOW and HIGH profiles keys with respect to the underlying task is not explained and motivated clearly. These labels are simplified taxonomy on privacy implication and its effect on the context is unclear. If the phone number is considered LOW, does it mean it can be used anywhere/any textbox in the form underlying the task? How will this affect the form minimization privacy probes?
  9. Both the probes/risks and LOW/HIGH levels are utilized together in a complicated way for the privacy evaluation. For example, it is unclear why we penalize the agent for filling LOW level keys on the optional when LOW definition says it is ok to use without permission. Also unclear whether typing into a field that is later cleared or never submitted counts as penalty.
  10. I believe Figure 1 misrepresents the capability of the environment. For example, the verification code from the phone automatically filled would not be supported in the action space in Appendix A. Is this one of the mock apps and the real trajectory from an agent in Table 1?
  11. In Section 2.3, How the data is generated is missing? There are a very few details on the target state and the seeded database without any clear explanation in the main paper. What are the distributions of the privacy risks/probes across the domain and tasks? Average number of probes per task?

Minor Weakness:

  1. The related works could use a section on LLM Agent privacy implications when using memory as iMy looks like an external memory that agent have access to via read/save actions.
  2. Table 1 caption says “300 tasks,” but I think it should be 250 as per the explanation before Table 1.
  3. Naming is inconsistent (iMy/IMY, MyPhoneBench/PhoneUse)
  4. “Later session transfer” is used but never mentioned in the plot. I believe these are the cross-session metrics.
  5. Will a human be able to solve these task with 100% success rate with better 100% privacy. No accompanying small user study exist to validate the findings from the human perspective.
Questions for authors
  1. How does this deterministic rules looks like for cases such as agent typed the data of birth field wrong or added an empty space character?
  2. In Section 2.1, How does agent/environment use the privacy contract? Is it through the read and save profile tools? Since save profile use the “level”, does this corresponds to the Definition of LOW and HIGH profiles keys
  3. How do you verify whether the agent successfully taken the actions? For example, for ordering food, does clicking “submit” button with all the field (required and/or optional) is considered successful.
  4. What happens when the agent asks the user for a specific question when the paper use only the deterministic user.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.