acceptodds
Under review as a conference paper at ICLR 2027

OS-Native Agents: Managing User Tasks in the Dynamic Mobile World

36 views

Abstract

Mobile agents can perform an increasingly broad range of tasks on behalf of users. Yet the mobile world continues to change after an agent has acted: application and system states evolve, new information arrives, and users revise their plans. In conventional mobile interaction, the operating system (OS) delivers relevant changes and governs access to device resources, while users decide which ongoing tasks require attention and how to respond across applications. Delegating this coordination to an agent requires OS support across interactions, including access to relevant state changes, cross-application capabilities, and execution under user authorization. We therefore introduce OS-native agents, which are recognized by the operating system and integrated into its native state, capability, and trust mechanisms. The OS provides system-level interfaces through which the agent observes runtime application and system state, discovers and composes structured capabilities exposed by applications and system services, and executes actions across apps and services through an OS contract for identity, authorization, and audit. We realize OS-native agents on Android and introduce DynamicWorld, a replayable benchmark for managing multiple ongoing tasks in a shared Android environment, with controlled changes in applications, system state, and user requests.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
mobile agent, task management, agent harness, Simulation environment

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_JLTfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the problem that traditional mobile agents mainly perform single or short-term operations and struggle with complex tasks that require long-term persistence, cross-app coordination, and adjustment in response to changes in device state and external events. The authors propose an OS-native agent framework for mobile operating systems that integrates task management with the OS environment, enabling the agent to maintain persistent workflows and to perform task scheduling, state tracking, and execution adjustment according to environmental changes. They also construct the DynamicWorld benchmark to evaluate agents' continuous task execution capability in dynamic mobile environments. This work extends the focus of mobile agent research from single UI operations to persistent workflow management in dynamic environments, provides a corresponding system framework and evaluation benchmark, and lays a research foundation for improving the adaptability and reliability of mobile agents in real-world complex scenarios.

Critical strengths
  1. The research problem has practical value. It focuses on long-term task management in dynamic mobile environments, expanding the application scenarios of mobile agents.
  2. The system design is relatively complete. It combines OS-level state awareness, function invocation, and permission control to achieve persistent cross-app task management.
  3. A new evaluation benchmark is proposed. DynamicWorld covers a variety of dynamic scenarios and can evaluate agents' continuous task execution capability.
  4. The experimental results are promising. Task success rates are notably better than those of compared methods while reducing the number of model calls.
Critical weaknesses
  1. There is a lack of ablation studies on key components, making it impossible to determine the true contribution of each part of the OS-Native framework.
  2. What are the fundamental architectural differences between OS-Native Agents and AOHP, beyond persistent task management and the introduction of DynamicWorld?
  3. Does the advantage of OS-Native Agents in DynamicWorld depend on the benchmark apps exposing more convenient interfaces specifically for this framework?
  4. Security and privacy mechanisms mainly rely on existing Android capabilities, and there is insufficient security evaluation.
  5. There is a lack of repeated experiments and statistical stability analysis.
Questions for authors

First, what is the main source of the performance improvement? The authors need to conduct ablation experiments within the same system to clarify how much the native state interface, AppFunctions, event recovery, and persistent task context each contribute to the performance gain. Second, what is the real technical innovation of OS-Native Agents relative to existing systems such as AOHP? If the main novelty lies in persistent task management and the DynamicWorld benchmark, these two contributions should be argued more focusedly, rather than presenting existing Android OS mechanisms themselves as the main innovation.

4 comments

We thank the reviewer for the comments.

Updated system. Since submission, we have improved the agent's harness: its context management, its skill texts, and its handling of user interaction. The OS mechanisms, the task design, the benchmark, and the baselines are unchanged. We also add Gemini 3.5 Flash as a third model, ablations of each module, and repeated runs for stability. We will update the paper with all results below.

Metrics. Success of each system is the mean ± SD over three runs; each ablation is one full run. Every comparison reports the paired difference Δ, its 95% confidence interval from a bootstrap over scenarios, and p from an exact permutation test over scenarios.

W1 / Q1. How much do the native state interface, AppFunctions, event recovery, and persistent task context each contribute?

Each ablation turns off one part of the GPT-5.4 system and keeps everything else fixed: the model, the harness, the apps, the event timelines, the simulated user, the judge, and the scoring. Native app-record reads and AppFunctions are removed together, because both let the agent read and change app records without the screen; device state and notifications stay, because they serve event handling.

Part named by the reviewerAblationWhat the agent does instead
Event recovery− ProactiveApps, notifications, and device conditions no longer wake the agent; it schedules its own checks and rereads app state at each check
Native state interface and AppFunctions− EfficientApp functions and the OS's calendar, contacts, messages, notes, photos, and file tools are turned off; it reads and changes app records through the app screens, while device state, notifications, and change subscriptions stay available
Persistent task context− PersistentEvery message and change goes to one long session; changes are not routed to tasks, and tasks are never closed
ConfigurationSuccessΔ vs. Full95% CIpCalls / scenarioTokens / scenario
Full (OS-native)78.4 ± 3.0%———142.22.48M
− Proactive60.2%−18.2 pp[−28.6, −8.2]0.005174.8 (+23%)3.45M (+39%)
− Efficient71.6%−6.8 pp[−14.8, +0.7]0.12201.7 (+42%)4.22M (+70%)
− Persistent69.3%−9.1 pp[−16.7, −2.6]0.0394.4 (−34%)3.44M (+39%)

A change is any app, system, or third-party event after delegation that concerns the task; the user's own later instructions are not counted. Success by the number of changes (GPT-5.4):

Later changes (tasks)0 (19)1 (22)2 (23)≥3 (24)Relative drop, 0 → ≥3
OS-native82.5 ± 12.2%78.8 ± 9.5%76.8 ± 2.5%76.4 ± 6.4%7%
MobileClaw59.6 ± 8.0%43.9 ± 6.9%55.1 ± 9.1%31.9 ± 2.4%46%
MobileWorld49.1 ± 6.1%24.2 ± 5.2%13.0 ± 4.3%4.2 ± 7.2%91%
OS-native − Proactive (one run)78.9%68.2%43.5%54.2%31%
OS-native − Efficient (one run)78.9%68.2%65.2%75.0%5%
OS-native − Persistent (one run)84.2%68.2%73.9%54.2%36%

OS-native's success falls by 7% (relative) from no changes to three or more, MobileClaw's by 46%, and MobileWorld's by 91%. Without event delivery or without the task layer, OS-native falls by 31% and 36%; without native interfaces, by only 5%.

  • Event recovery decides whether the agent acts in time. Without it, success drops 3.5 pp on tasks without later changes and 22.2 pp on tasks with them ([−35.6, −9.9], p = 0.006).
  • Persistent task context keeps the agent on track as changes accumulate. One long session falls 12.1 pp behind on tasks with later changes ([−21.9, −3.3], p = 0.03); its context grows from about 25K to 60–76K tokens, while Full stays at 20–25K.
  • Native state and AppFunctions lower the cost of each step. Through the screen, the agent needs 42% more calls and 70% more tokens.

W2 / Q2. What is the fundamental difference from AOHP?

We agree with the reviewer's reading: the main novelty lies in persistent task management and DynamicWorld, and the Android mechanisms are not the innovation. We argue the two as one contribution: autonomous task management is the design, the OS mechanisms are the means, and DynamicWorld is the test.

AndroidAOHPOurs
ProactiveNotification listeners, broadcasts, alarmsEvent streams: a notification buffer and sensor streams the agent subscribes toMonitors bound to tasks; a change log that keeps every notification, app event, and device change; event triage that decides per task to ignore, record, or act, and wakes only the affected task
EfficientAppFunctions, content providers, system servicesSystem-exposed tools through API, CLI, structured UI, or GUI as fallback; skills package reusable capabilitiesSystem-exposed tools: app functions, OS tools for calendar, contacts, messages, notes, photos, and files, and system actions, with GUI as fallback; skills load only the functions they declare
ControlledPermissions, identity, trusted confirmationPer-operation information-flow policy, trusted vault, taint trackingCalls sorted by declared side effect; grants issued outside the model, per task, revoked when it closes; notification redaction and on-device profile filling
PersistentStorage, keep-aliveSystem memory for preferences, task state, histories, and policy decisions (profile, task-local, sensitive)Task records (goal, constraints, progress, linked app records); routing each message to its task's session; rebuilding context from the record on wake-up; closing the task, which removes its monitors and grants

Both systems build on largely the same OS mechanisms. They differ in what the mechanisms are tied to: AOHP ties them to the agent while it handles a request; we tie them to the user's task, from delegation until it closes. We also do not modify the OS: Android already gives apps every mechanism this needs. As W1 shows, tying events and state to the task is what keeps the agent's success level as changes accumulate.

W3. Does the advantage depend on the benchmark apps exposing more convenient interfaces for this framework?

No. Without native interfaces, our agent reads and changes app records through the same screens as MobileClaw, with the same model:

Comparison (GPT-5.4)Δ95% CIp
OS-native without native interfaces (− Efficient) vs. MobileClaw+24.6 pp[+11.1, +37.3]0.004

The interfaces also were not designed around our agent:

ItemHow it is built
AppFunction schemasFollow published Skill and API references, such as Variflight, AMap, and DiDi
App logicGUI pages and AppFunctions call the same business operations on the same records under the same rules
Task constructionEvery task was executed once through the GUI and once through AppFunctions, and kept only if both completed it
ScoringReads app records, not which interface was used

W4. Security and privacy mechanisms mainly rely on existing Android capabilities; the security evaluation is insufficient.

The mechanisms do come from Android. What we add is that they serve the task: a task's monitors and checks last only as long as the task. To evaluate them, we ran checks on a real device in the form of AOHP's security evaluation (its Table 2). Each check runs the full code path, from the harness to the agent application to the app. A scripted model issues the calls, so the outcome depends only on the checks: they hold even if the model is misled, for example by injected text, into making the call.

CheckExpected behaviorResult
Ordinary actionsWriting a note goes through without asking the userPass
Sensitive actionsA tap on a payment button requires the user's trusted confirmation; after a refusal, a coordinate tap on the same button requires it again, and the order stays unpaidPass
Unsupported accessA function outside the declared interface set is refused before it reaches the appPass
Sensitive eventsCard numbers, ID numbers, one-time codes, and passwords in notifications are redacted before they reach the modelPass
Task closingClosing a task removes its monitors; a later matching notification wakes nothingPass

W5. Repeated experiments and statistical stability.

Every system runs three times with every model on all 88 tasks with the same event timelines, 27 full runs in total.

ModelSystemSuccessModel calls / scenarioTokens / scenarioUser requests / scenario
GPT-5.4MobileWorld21.2 ± 2.9%102.81.52M1.72
MobileClaw47.0 ± 5.7%290.73.40M2.29
OS-native78.4 ± 3.0%142.22.48M1.63
GPT-5.4-miniMobileWorld4.5 ± 2.3%61.10.80M1.12
MobileClaw17.0 ± 3.4%196.51.68M2.71
OS-native36.7 ± 5.7%100.01.64M1.10
Gemini 3.5 FlashMobileWorld44.3 ± 2.3%120.52.17M1.38
MobileClaw51.9 ± 3.5%340.95.19M2.37
OS-native73.9 ± 1.1%170.04.46M1.25

Each task is averaged over its three runs and paired with the same task in the baseline:

ModelComparisonΔ95% CIp
GPT-5.4OS-native vs. MobileClaw+31.4 pp[+20.0, +43.0]0.0004
GPT-5.4OS-native vs. MobileWorld+57.2 pp[+45.8, +68.1]<0.0001
GPT-5.4-miniOS-native vs. MobileClaw+19.7 pp[+13.6, +25.4]0.0002
GPT-5.4-miniOS-native vs. MobileWorld+32.2 pp[+23.5, +40.3]0.0001
Gemini 3.5 FlashOS-native vs. MobileClaw+22.0 pp[+9.0, +36.9]0.007
Gemini 3.5 FlashOS-native vs. MobileWorld+29.5 pp[+15.9, +43.6]0.002

All six differences are significant, and every interval lies above zero. We will add the per-run tables and the tests to the appendix.

Area_Chair_g1C9OpenReview ↗

Reviewer response to authors?

Dear reviewer JLTf,

Author-reviewer discussions end on November 18th. If possible, please respond to author comments by that time so that authors have opportunity for any additional responses.

Thank you.

Reviewer_bTRzOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper observes that existing mobile agents are largely evaluated on single-turn task execution, whereas the phone continues to change after an agent acts. It introduces the concept of an OS-native agent: an agent recognized by the operating system and integrated into its native state observation, capability discovery, and identity/authorization/audit mechanisms, with an Android implementation built on an extended DeepSeek Harness. Alongside the system, it releases DynamicWorld, a replayable benchmark of 88 tasks across 16 scenarios and 18 apps that evaluates initial delivery, revision, cancellation, and later preservation under scripted application, system, and user changes. On GPT-5.4, the reported OS-native system reaches 76.1% task success versus 40.9% for MobileClaw and 23.9% for MobileWorld. The problem framing and layered benchmark design are useful, but the current evidence does not isolate which design choices drive the reported gains, baseline configuration is not documented well enough to rule out confounds, and statistical and reproducibility safeguards are missing.

Critical strengths
  • The problem is well identified and well motivated. The paper frames ongoing cross-app task management as an OS problem along state, capability, and trust, and Figure 1 makes the distinction concrete by contrasting user-managed follow-up with an OS-native agent that receives the delay event. This matches a real gap as mobile agents move from single instructions to long-lived delegations.
  • The benchmark targets the right failure modes. Tasks in a scenario share persistent application records, so earlier actions change what later decisions require; replays fix the same event timeline for every system; and temporal scoring captures "fixed, then broken again". This is closer to real task-management failure than static single-goal tasks.
Critical weaknesses
  • §4.3 and the Metrics do not report the number of repeated runs, and Tables 3–5 report only one binary success value per cell. LLM agents are subject to sampling variance, and several cells rest on small samples. Repeated runs with mean, spread, and a paired significance test are needed, with intervals reported for small categories.
  • §4.3 states that "a local LLM judge evaluates open-ended text or images," but the judge model, rubric prompt, human annotation subset, and agreement measure are not reported.
Questions for authors

Please see the above comments

3 comments

We thank the reviewer for the detailed comments.

Updated system. Since submission, we have improved the agent's harness: its context management, its skill texts, and its handling of user interaction. The OS mechanisms, the task design, the benchmark, and the baselines are unchanged. We also add Gemini 3.5 Flash as a third model, ablations of each module, and repeated runs for stability. We will update the paper with all results below.

Metrics. Success of each system is the mean ± SD over three runs; each ablation is one full run. Every comparison reports the paired difference Δ, its 95% confidence interval from a bootstrap over scenarios, and p from an exact permutation test over scenarios.

W1. Number of repeated runs; mean, spread, and a paired significance test; intervals for small categories.

Every system runs three times with every model on all 88 tasks with the same event timelines, 27 full runs in total.

ModelSystemSuccessModel calls / scenarioTokens / scenarioUser requests / scenario
GPT-5.4MobileWorld21.2 ± 2.9%102.81.52M1.72
MobileClaw47.0 ± 5.7%290.73.40M2.29
OS-native78.4 ± 3.0%142.22.48M1.63
GPT-5.4-miniMobileWorld4.5 ± 2.3%61.10.80M1.12
MobileClaw17.0 ± 3.4%196.51.68M2.71
OS-native36.7 ± 5.7%100.01.64M1.10
Gemini 3.5 FlashMobileWorld44.3 ± 2.3%120.52.17M1.38
MobileClaw51.9 ± 3.5%340.95.19M2.37
OS-native73.9 ± 1.1%170.04.46M1.25

Paired test. Tasks in one scenario share a phone, its records, and its events, so the 88 tasks are not independent. We average each task over its three runs and take the per-task difference between two systems. The 95% confidence interval comes from a bootstrap that resamples whole scenarios (5,000 draws), and p from an exact permutation test that flips the sign of each scenario's difference (all 2^16 assignments).

ModelComparisonΔ95% CIp
GPT-5.4OS-native vs. MobileClaw+31.4 pp[+20.0, +43.0]0.0004
GPT-5.4OS-native vs. MobileWorld+57.2 pp[+45.8, +68.1]<0.0001
GPT-5.4-miniOS-native vs. MobileClaw+19.7 pp[+13.6, +25.4]0.0002
GPT-5.4-miniOS-native vs. MobileWorld+32.2 pp[+23.5, +40.3]0.0001
Gemini 3.5 FlashOS-native vs. MobileClaw+22.0 pp[+9.0, +36.9]0.007
Gemini 3.5 FlashOS-native vs. MobileWorld+29.5 pp[+15.9, +43.6]0.002

All six differences are significant, and every interval lies above zero.

Small categories. The same paired comparison for the three smallest categories (GPT-5.4):

Category (tasks)OS-nativeMobileClawΔ95% CIp
Routine execution (5)80.0 ± 0.0%40.0 ± 20.0%+40.0 pp[+6.7, +73.3]0.25
Intent clarification (8)75.0 ± 12.5%25.0 ± 0.0%+50.0 pp[+25.0, +75.0]0.03
Authorized execution (11)81.8 ± 0.0%45.5 ± 9.1%+36.4 pp[+12.1, +63.6]0.047

W2. Judge model, rubric prompt, human annotation subset, and agreement measure.

ItemDetail
What programs checkEvery structured condition, such as records, times, and amounts
What the judge decidesOnly open-ended text and images, against each task's rubric and the saved evidence
Judge modelGPT-5.6 Terra
Rubric reviewEvery generated rubric was reviewed for requirement coverage, factual correctness, and acceptable solutions: 85 of 88 passed unchanged, 3 were revised
Human annotationTwo annotators independently checked 100 sampled judge decisions against the same rubric and evidence
AgreementWith the judge: 97 and 96 of 100; with each other: 99 of 100 (Cohen's κ = 0.85)
PromptsWe will include the judge prompt and the rubric prompt in the appendix

Ablations. Each ablation turns off one part of the GPT-5.4 system and keeps everything else fixed: the model, the harness, the apps, the event timelines, the simulated user, the judge, and the scoring.

AblationWhat is removedWhat the agent does instead
− ProactiveNative event deliveryApps, notifications, and device conditions no longer wake the agent; it schedules its own checks and rereads app state at each check
− EfficientNative reads and writes of app records: app functions and the OS's calendar, contacts, messages, notes, photos, and file toolsIt reads and changes these records through the app screens; device state, notifications, and change subscriptions stay available
− PersistentThe task layerEvery message and change goes to one long session; changes are not routed to tasks, and tasks are never closed
ConfigurationSuccessΔ vs. Full95% CIpCalls / scenarioTokens / scenario
Full (OS-native)78.4 ± 3.0%———142.22.48M
− Proactive60.2%−18.2 pp[−28.6, −8.2]0.005174.8 (+23%)3.45M (+39%)
− Efficient71.6%−6.8 pp[−14.8, +0.7]0.12201.7 (+42%)4.22M (+70%)
− Persistent69.3%−9.1 pp[−16.7, −2.6]0.0394.4 (−34%)3.44M (+39%)

Event delivery carries most of the gain, and the task layer adds to it; native interfaces mainly lower cost.

Baseline configuration. The same ablation addresses confounding. Without native interfaces, our agent reads and changes app records through the same screens as MobileClaw, with the same model:

Comparison (GPT-5.4)Δ95% CIp
OS-native without native interfaces (− Efficient) vs. MobileClaw+24.6 pp[+11.1, +37.3]0.004

So the gap over MobileClaw does not come from giving our agent better access to the apps. All systems run the same 16 scenarios and 88 tasks with the same apps, event timelines, simulated user, and judge, and the named model performs all agent inference, including GUI execution:

SystemPlanning and memoryObservation and actionResumes on changes
MobileWorldConversation historyScreenshots; GUI actionsScheduled reviews
MobileClawPersistent memory and skills; a GUI model operates the phoneScreenshots and UI tree; GUI actionsNotification delivery and scheduled reviews (added by us)
OS-nativeTask recordsApp state and app functions; GUI when no interface existsOS events routed to the task

We will list each system's model, call limits, and authorization settings in the appendix.

Reproducibility. Every system replays the same fixed event timeline in each scenario, and scores are computed from saved app records, delivered messages, and the judge's decisions on saved evidence, so any run can be rescored. We will release every run of every configuration.

Ethics flag. No person takes part in task execution; the user is simulated. Scenario data are synthetic. A questionnaire on daily activities only decided which activities the scenarios cover, and none of its content enters the data. Personal profiles are fixtures, and all payments and orders happen inside simulated apps on a test device. No real account, payment method, or third party is involved. We will add this statement to the paper.

Thanks for the authors' detailed response. I am sorry for selecting the wrong ethics option. I will raise my score.

Reviewer_paWKOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces OS-native agents for managing ongoing user delegations as application state, device conditions, and user requests change. The Android realization connects native state observation, capability discovery, and authorization mechanisms to a persistent agent harness. The authors also introduce DynamicWorld, comprising 88 tasks across 16 scenarios, which evaluates required outcomes and action constraints over time. With GPT-5.4, the proposed system achieves 76.1% task success, compared with 40.9% for MobileClaw and 23.9% for MobileWorld, while reducing model calls relative to MobileClaw.

Critical strengths
  • fulfillment of a user’s goal over time. DynamicWorld evaluates revisions, preservation, stopping conditions, and authorization windows, capturing failures that terminal-state evaluation alone can miss.
  • The integrated system demonstrates substantial improvements with both evaluated foundation models. Reporting model calls, tokens, and user requests alongside task success helps characterize the practical trade-offs of the architecture.
Critical weaknesses
  • The contribution relative to existing OS-level agent frameworks needs sharper delineation. AOHP already provides agent-oriented OS interfaces, service composition, and security mechanisms. The strongest contribution here appears to be their integration with persistent delegation management and the accompanying dynamic benchmark, rather than the individual native mechanisms. A more explicit separation of inherited support and newly introduced mechanisms would clarify the technical advance.
  • The complete-system comparisons establish an overall advantage but provide limited attribution. Native state access, structured capabilities, event handling, and harness design change together. Moreover, among the 41 tasks initially completed by both OS-native and MobileClaw, full successes are 35 and 31, respectively. This is useful evidence, but it leaves the specific advantage for continued fulfillment less firmly established than the aggregate success gap.
Questions for authors
  • Which mechanisms are newly implemented beyond existing OS-level frameworks, and which are reused or integrated?
  • Can matched comparisons independently disable native event delivery or structured capabilities while retaining the same harness?
  • How variable are the results across repeated runs or event schedules, particularly for continued fulfillment after successful initial delivery?
4 comments

We thank the reviewer for the careful reading.

Updated system. Since submission, we have improved the agent's harness: its context management, its skill texts, and its handling of user interaction. The OS mechanisms, the task design, the benchmark, and the baselines are unchanged. We also add Gemini 3.5 Flash as a third model, ablations of each module, and repeated runs for stability. We will update the paper with all results below.

Metrics. Success of each system is the mean ± SD over three runs; each ablation is one full run. Every comparison reports the paired difference Δ, its 95% confidence interval from a bootstrap over scenarios, and p from an exact permutation test over scenarios.

W1 / Q1. Which mechanisms are newly implemented beyond existing OS-level frameworks, and which are reused or integrated?

We agree that the native mechanisms themselves are not our contribution. Our contribution is autonomous task management: the agent keeps managing a user's task after the user has left, until the task ends. The OS mechanisms are the means.

AndroidAOHPOurs
ProactiveNotification listeners, broadcasts, alarmsEvent streams: a notification buffer and sensor streams the agent subscribes toMonitors bound to tasks; a change log that keeps every notification, app event, and device change; event triage that decides per task to ignore, record, or act, and wakes only the affected task
EfficientAppFunctions, content providers, system servicesSystem-exposed tools through API, CLI, structured UI, or GUI as fallback; skills package reusable capabilitiesSystem-exposed tools: app functions, OS tools for calendar, contacts, messages, notes, photos, and files, and system actions, with GUI as fallback; skills load only the functions they declare
ControlledPermissions, identity, trusted confirmationPer-operation information-flow policy, trusted vault, taint trackingCalls sorted by declared side effect; grants issued outside the model, per task, revoked when it closes; notification redaction and on-device profile filling
PersistentStorage, keep-aliveSystem memory for preferences, task state, histories, and policy decisions (profile, task-local, sensitive)Task records (goal, constraints, progress, linked app records); routing each message to its task's session; rebuilding context from the record on wake-up; closing the task, which removes its monitors and grants

Both systems build on largely the same OS mechanisms. They differ in what the mechanisms are tied to: AOHP ties them to the agent while it handles a request; we tie them to the user's task, from delegation until it closes. We also do not modify the OS: Android already gives apps every mechanism this needs.

W2 (first part) / Q2. Can matched comparisons independently disable native event delivery or structured capabilities while retaining the same harness?

Yes. Each ablation turns off one part of the GPT-5.4 system and keeps everything else fixed: the model, the harness, the apps, the event timelines, the simulated user, the judge, and the scoring.

AblationWhat is removedWhat the agent does instead
− ProactiveNative event deliveryApps, notifications, and device conditions no longer wake the agent; it schedules its own checks and rereads app state at each check
− EfficientNative reads and writes of app records: app functions and the OS's calendar, contacts, messages, notes, photos, and file toolsIt reads and changes these records through the app screens; device state, notifications, and change subscriptions stay available
− PersistentThe task layerEvery message and change goes to one long session; changes are not routed to tasks, and tasks are never closed
ConfigurationSuccessΔ vs. Full95% CIpCalls / scenarioTokens / scenario
Full (OS-native)78.4 ± 3.0%———142.22.48M
− Proactive60.2%−18.2 pp[−28.6, −8.2]0.005174.8 (+23%)3.45M (+39%)
− Efficient71.6%−6.8 pp[−14.8, +0.7]0.12201.7 (+42%)4.22M (+70%)
− Persistent69.3%−9.1 pp[−16.7, −2.6]0.0394.4 (−34%)3.44M (+39%)
  • Event delivery decides whether the agent acts in time. The loss is largest on Timely action (88.9% → 57.1%) and Plan adaptation (83.3% → 63.6%). In T048, the agent set its next check for 23:10, but the item came back in stock at 19:10 and the user withdrew the request at 20:10.
  • Structured capabilities decide the cost of each step. Through the screen, the agent needs 42% more calls and 70% more tokens.
  • The task layer keeps the agent on track as changes accumulate. One long session carries the whole history, which grows from about 25K to 60–76K tokens within a scenario; Full rebuilds each session from the task record and stays at 20–25K.

W2 (second part) / Q3. Is the advantage in continued fulfillment firm, and how variable are the results across repeated runs or event schedules?

Continued fulfillment after initial delivery. A change is any app, system, or third-party event after delegation that concerns the task; the user's own later instructions are not counted. On the 69 tasks that receive at least one change, a task is initially successful when its first delivery is correct (for a task that asks for a single delivery, that delivery is the first one), and kept when it then stays correct through all later changes (GPT-5.4, mean over runs):

Metric (69 tasks)OS-nativeMobileClawMobileWorldΔ vs. MobileClaw95% CIpΔ vs. MobileWorld95% CIp
Initial success83.6%53.6%34.8%+30.0 pp[+16.9, +42.8]0.001+48.8 pp[+37.0, +59.4]<0.001
Initial success kept to the end77.3%42.8%14.5%+34.5 pp[+19.6, +48.4]0.001+62.8 pp[+50.2, +74.2]<0.001
Kept rate (of initial successes)92.5%79.7%41.7%+12.8 pp[+2.8, +23.4]0.01+50.8 pp[+33.0, +68.9]<0.001

The kept rate is a ratio and cannot be paired task by task, so its interval and p come from the same scenario bootstrap.

The 35-vs.-31 comparison in the submission (85% vs. 76% of 41 shared tasks) came from one pair of runs. Over runs, on the tasks that both systems first delivered, OS-native fully succeeds more often in every pair of runs: 90–94% against 71–89%. The gap also starts before the first delivery: on the 26 tasks whose first delivery is due only after a change, OS-native makes 22.7 initial deliveries and MobileClaw 11.5.

Across event schedules. The 16 scenarios differ in their timelines. Success by the number of changes after delegation, counted as above (GPT-5.4):

Later changes (tasks)0 (19)1 (22)2 (23)≥3 (24)Relative drop, 0 → ≥3
OS-native82.5 ± 12.2%78.8 ± 9.5%76.8 ± 2.5%76.4 ± 6.4%7%
MobileClaw59.6 ± 8.0%43.9 ± 6.9%55.1 ± 9.1%31.9 ± 2.4%46%
MobileWorld49.1 ± 6.1%24.2 ± 5.2%13.0 ± 4.3%4.2 ± 7.2%91%
OS-native − Proactive (one run)78.9%68.2%43.5%54.2%31%
OS-native − Efficient (one run)78.9%68.2%65.2%75.0%5%
OS-native − Persistent (one run)84.2%68.2%73.9%54.2%36%

From no changes to three or more, OS-native's success falls by 7% (relative), MobileClaw's by 46%, and MobileWorld's by 91%. Without event delivery or without the task layer, OS-native falls by 31% and 36%, closer to MobileClaw; without native interfaces, by only 5%. On the 69 tasks with later changes, removing event delivery costs 22.2 pp (95% CI [−35.6, −9.9], p = 0.006) and removing the task layer 12.1 pp ([−21.9, −3.3], p = 0.03). The gap over MobileClaw grows from +22.8 pp with no changes to +44.4 pp with three or more (95% CI [+20.6, +65.5], p = 0.01). On the 69 tasks with later changes, OS-native also leads MobileClaw with the other two models: +23.7 pp with Gemini 3.5 Flash (95% CI [+7.3, +41.8], p = 0.02) and +17.4 pp with GPT-5.4-mini ([+10.3, +24.5], p < 0.001).

Across runs. Success over three runs:

ModelSystemSuccessModel calls / scenarioTokens / scenarioUser requests / scenario
GPT-5.4MobileWorld21.2 ± 2.9%102.81.52M1.72
MobileClaw47.0 ± 5.7%290.73.40M2.29
OS-native78.4 ± 3.0%142.22.48M1.63
GPT-5.4-miniMobileWorld4.5 ± 2.3%61.10.80M1.12
MobileClaw17.0 ± 3.4%196.51.68M2.71
OS-native36.7 ± 5.7%100.01.64M1.10
Gemini 3.5 FlashMobileWorld44.3 ± 2.3%120.52.17M1.38
MobileClaw51.9 ± 3.5%340.95.19M2.37
OS-native73.9 ± 1.1%170.04.46M1.25

Thank you for the repeated evaluations and the expanded analysis of continued fulfillment. The retention analysis, the comparisons on jointly initially successful tasks, and the additional model results strengthen the evidence that the benefit extends beyond initial execution.

The breakdown by subsequent changes is also informative, although it measures performance across different task groups rather than sensitivity to alternative event timings within the same scenario. Please preserve this distinction and note that the ablations are based on single runs.

Taken together, the additional evidence addresses my main concerns, and I maintain my weak-accept recommendation with greater confidence.

Thank you for clarifying the contribution relative to existing OS-level frameworks and for providing the matched ablations. The distinction between reusing native mechanisms and organizing them around persistent user delegations makes the contribution substantially clearer.

The ablations also address my concern about component attribution. The results support the importance of event delivery and the persistent task layer, while the structured-interface ablation provides stronger evidence for efficiency benefits than for a reliable success-rate improvement.

Please incorporate these distinctions into the revision and clearly separate results obtained with the updated harness from those in the original submission. Overall, this response substantially reduces my concerns about the architectural contribution.

Comments on the paper

1 comment

We sincerely thank all reviewers for their constructive comments and the time they spent on our paper. We have carefully revised the paper based on their suggestions and uploaded the newest version.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.