acceptodds
Under review as a conference paper at ICLR 2027

Transition and Recovery Alignment via Evidence-Grounded Contracts for Mobile GUI Agents

Abstract

Mobile GUI agents execute natural-language instructions through sequences of interface actions, but long-horizon execution remains vulnerable to silent transition failures, state drift, and ineffective recovery. A plausible action may fail to produce the intended state transition, causing subsequent decisions to be conditioned on a state that was never reached and allowing these errors to compound over time. Existing systems often address such failures through online reflection or auxiliary verification, introducing additional reasoning and model calls during execution. We introduce TRAVEC, a framework that moves transition prediction, verification, and recovery from the deployment-time execution loop into training. The key abstraction is an evidence-grounded transition contract, which specifies the task-relevant state change expected after an action, the observable evidence used to verify that change, and an optional recovery conditioned on transition failure. From heterogeneous mobile GUI trajectories, we construct 550K verifiable supervision items to support training a Contract Policy and a Verifier that audit self-generated trajectories and retain audited training steps. The resulting audited experience is then used to optimize the Action Policy through supervised learning and offline GRPO. These auxiliary components are used only during training, while deployment retains a single action-only policy with no additional verification or free-form reasoning. Across four offline and two online benchmarks, TRAVEC achieves strong performance while maintaining low inference latency, suggesting that transition-aware supervision can be incorporated into policy learning while preserving action-only deployment.

Primary area:
foundation or frontier models, including LLMs
Keywords:
GUI Agents, Multimodal Agents, Mobile Agents

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Ca3KOpenReview ↗
Rating
2
Confidence
4(+2)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces TRAVEC, a training framework for mobile GUI agents based on evidence-grounded transition contracts. Each contract specifies the current state and preconditions, an action, its expected state change, observable verification evidence, and optional failure-conditioned recovery. The authors construct a reported 550K supervision items and train role-specific Action, Contract, and Verifier policies. During training, these auxiliary policies and a trajectory Critic audit collected interactions, which support action-policy optimization through SFT and GRPO. Deployment uses only an action-only policy without auxiliary verification or free-form reasoning. Evaluation covers four offline and two online benchmarks, with component and reward ablations and manual audits of annotation and verifier quality.

Critical strengths
  1. The paper addresses an important practical problem: a syntactically valid and well-grounded GUI action does not necessarily produce the intended task-relevant transition. Organizing expected changes, observable evidence, and recovery conditions into a common training interface is a useful system design. Using auxiliary components only during training also preserves the low inference cost of an action-only deployment policy.

  2. The evaluation covers both offline action prediction/grounding and online task completion. The reported ScreenSpot-Pro and MobileWorld improvements are encouraging. Appendix A.5 explicitly requires candidate-specific execution evidence collected from the same decision-state snapshot, avoiding the inappropriate reuse of a reference action's outcome for unexecuted alternatives. The manual annotation and verifier audits provide additional, although limited, evidence about supervision quality.

Critical weaknesses
  1. The training/evaluation separation and supervision counts are not sufficiently clear to establish the experimental protocol. Section 4.2 states that the 400 AndroidWorld training tasks are disjoint from the evaluation task templates, while Appendix B.3 reports evaluation on all 116 described AndroidWorld templates. The paper does not explain which additional templates generated the training tasks or whether “disjoint” instead refers to different parameter instantiations of shared templates. Separately, Figure 5(b) includes 50,000 parser-only MobileViews records and 2,853 evaluation-only ScreenSpot/ScreenSpot-Pro records within the reported 350,000 contract/state items. Excluding these leaves 297,147 items. These observations do not establish leakage, but they leave both the generalization setting and the actual training-data count unresolved.

  2. The experiments do not sufficiently isolate or directly validate the central contribution. A matched Qwen3.5-9B Action-SFT-only baseline is missing, and the action-only reward ablation does not report online task completion. Consequently, it is difficult to separate the contribution of structured contracts from domain adaptation, trajectory filtering, and standard action-alignment optimization. Moreover, on AndroidWorld, 35.345% versus 34.483% corresponds to 123 versus 120 successful episodes out of 348; without paired statistical analysis or repeated training runs, this small margin does not establish a reliable advantage over the strongest baseline. The paper also lacks quantitative measurements of silent transition failures, state drift, and recovery effectiveness, so the aggregate benchmark results do not directly demonstrate the proposed mechanism.

Questions for authors

Clarification: What templates generate the 400 AndroidWorld training tasks? Please specify their relationship to the 116 evaluation templates and distinguish template-level separation from parameter-instance separation. Are the training templates newly constructed?

Clarification: Does the 350K contract/state count, and consequently the 550K total, include parser-only and evaluation-only records? Please provide the number of examples actually used to optimize each policy and explain how the 60K preference pairs enter training.

4 comments
AuthorsOpenReview ↗

Response to Reviewer Ca3K (1/3)

We are grateful to Reviewer Ca3K for the careful and detailed review. We also appreciate the reviewer's recognition of the importance of the problem we study and our use of candidate-specific execution evidence. In response to the comments, we have clarified the experimental protocol and supervision counts, added the requested matched training controls, and conducted a direct evaluation of transition failures and recovery. We address each comment individually below.


W1: The training/evaluation separation and supervision counts are not sufficiently clear to establish the experimental protocol. Section 4.2 states that the 400 AndroidWorld training tasks are disjoint from the evaluation task templates, while Appendix B.3 reports evaluation on all 116 described AndroidWorld templates. The paper does not explain which additional templates generated the training tasks or whether “disjoint” instead refers to different parameter instantiations of shared templates. Separately, Figure 5(b) includes 50,000 parser-only MobileViews records and 2,853 evaluation-only ScreenSpot/ScreenSpot-Pro records within the reported 350,000 contract/state items. Excluding these leaves 297,147 items. These observations do not establish leakage, but they leave both the generalization setting and the actual training-data count unresolved.

A1: We thank the reviewer for the careful reading. We clarify both points below.

Training/evaluation separation. The 400 AndroidWorld training tasks were independently generated using MobileForge's public task-generation pipeline and exploration trajectories, without using the official evaluation templates as generation references. These tasks are newly constructed instructions rather than parameter instantiations of the 116 official evaluation templates. The term "disjoint" in Section 4.2 refers to task-template-level separation, rather than different parameter instantiations of shared templates. Since training and evaluation take place within the same AndroidWorld application ecosystem, we characterize this setting as in-environment adaptation. MobileWorld is reserved exclusively for evaluation, providing an additional assessment on a different mobile benchmark. We have clarified the task-generation procedure and the evaluation setting in Section 4.2.

Supervision counts. As the reviewer notes, the 350,000 contract/state items in Figure 5(b) include 50,000 MobileViews records used only for parser construction and 2,853 ScreenSpot/ScreenSpot-Pro records reserved for evaluation. Excluding these sources leaves 297,147 records used to construct training supervision, as detailed below.

SourceRecordsPurpose
AITW160,000Training supervision
AndroidControl (training split)60,000Training supervision
CMGUI41,486Training supervision
AMEX35,661Training supervision
Subtotal297,147Training supervision
MobileViews50,000Parser construction only
ScreenSpot / ScreenSpot-Pro2,853Evaluation only
Total350,000

The reported 550K total comprises 497,147 constructed records and pairs, together with 52,853 parser-only and evaluation-only records. Figure 5(b) identifies the purposes of these sources, but the main text did not separate them from the training supervision. We have made this distinction explicit and report the counts consistently throughout the revised manuscript.


W2: The experiments do not sufficiently isolate or directly validate the central contribution. A matched Qwen3.5-9B Action-SFT-only baseline is missing, and the action-only reward ablation does not report online task completion. Consequently, it is difficult to separate the contribution of structured contracts from domain adaptation, trajectory filtering, and standard action-alignment optimization. Moreover, on AndroidWorld, 35.345% versus 34.483% corresponds to 123 versus 120 successful episodes out of 348; without paired statistical analysis or repeated training runs, this small margin does not establish a reliable advantage over the strongest baseline. The paper also lacks quantitative measurements of silent transition failures, state drift, and recovery effectiveness, so the aggregate benchmark results do not directly demonstrate the proposed mechanism.

AuthorsOpenReview ↗

Response to Reviewer Ca3K (2/3)

A2: We thank the reviewer for these detailed and constructive suggestions. To address these concerns, we have added two controlled training comparisons and a direct evaluation of transition failures and recovery.

Matched controls. We have added a Qwen3.5-9B Action-SFT-only baseline and report online results for the action-only reward configuration in Table 4. All three configurations share the same backbone, Action-SFT data, and evaluation settings.

Training configurationAndroidControl High ↑AndroidControl Low ↑ScreenSpot ↑ScreenSpot-Pro ↑AndroidWorld ↑MobileWorld ↑
Action-SFT-only50.25977.53285.45647.81812.9315.983
SFT + GRPO with action-only rewards54.89078.81692.06062.99816.6678.547
TRAVEC-9B64.30993.33792.68967.80535.34515.385

All values are percentages. Performance increases from Action-SFT-only to action-only GRPO and then to full TRAVEC across all six metrics. Compared with Action-SFT-only, TRAVEC improves AndroidWorld and MobileWorld success by 22.414 and 9.402 percentage points, respectively. Compared with action-only GRPO, which uses the same SFT initialization, rollout data, and GRPO training schedule, TRAVEC improves the two online success rates by 18.678 and 6.838 percentage points. These results provide more direct evidence of the additional benefits of process-aware training beyond standard action supervision and action-only GRPO. The full comparison is reported in Section 4.7 and Appendix E.1 of the revised manuscript.

AndroidWorld comparison. We appreciate the reviewer's observation regarding the small numerical margin between TRAVEC and ForgeQwen3-8B on AndroidWorld. We interpret their performance on this benchmark as comparable, while the controlled comparisons above and the results on MobileWorld, which is not used during training, provide additional evidence of TRAVEC's effectiveness.

Direct evaluation of transition failures and recovery. We inspected the complete execution trajectories of each model on all 117 MobileWorld tasks, including task goals, action sequences, and interface evidence before and after each step.

ModelTask success ↑Transition failure rate ↓Recovery rate after first failure ↑Task success after failure ↑
Qwen3.5-9B5.13% (6/117)76.11%2.63% (3/114)2.63% (3/114)
OpenMobile-8B8.55% (10/117)49.59%13.27% (15/113)5.31% (6/113)
TRAVEC-9B (ours)15.38% (18/117)39.59%35.65% (41/115)13.91% (16/115)

The transition failure rate measures the proportion of executed steps that fail to produce their intended task-relevant state changes. The recovery rate measures the proportion of trajectories with at least one confirmed transition failure in which subsequent actions successfully resolve the first failure, while task success after failure is measured on the same trajectories.

TRAVEC achieves the lowest transition failure rate and the highest recovery rate and post-failure task success among the three models. Compared with the base Qwen3.5-9B model sharing the same backbone, TRAVEC improves the recovery rate and task success after failure by 33.02 and 11.28 percentage points, respectively. These results provide direct quantitative evidence of the deployed Action Policy's failure recovery capability, complementing the qualitative examples of silent transition failures and state drift in Appendix C.

We further break down these metrics by grouping each model's trajectories by the number of steps it executed (1–20 and 21+ steps).

ModelExecution lengthTrajectoriesTask success ↑Transition failure rate ↓Recovery rate after first failure ↑Task success after failure ↑
Qwen3.5-9B1–20 steps885.68% (5/88)64.93%2.35% (2/85)2.35% (2/85)
Qwen3.5-9B21+ steps293.45% (1/29)93.59%3.45% (1/29)3.45% (1/29)
OpenMobile-8B1–20 steps6811.76% (8/68)42.96%15.63% (10/64)6.25% (4/64)
OpenMobile-8B21+ steps494.08% (2/49)55.54%10.20% (5/49)4.08% (2/49)
TRAVEC-9B (ours)1–20 steps6016.67% (10/60)28.17%36.21% (21/58)13.79% (8/58)
TRAVEC-9B (ours)21+ steps5714.04% (8/57)47.04%35.09% (20/57)14.04% (8/57)

Transition failures become more frequent in longer trajectories for all three models, while TRAVEC maintains similar recovery rates across the two groups (36.21% and 35.09%) and achieves the lowest transition failure rate and the highest task success in both. These evaluations are reported in Section 4.7 and Appendix E of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer Ca3K (3/3)

Q1: What templates generate the 400 AndroidWorld training tasks? Please specify their relationship to the 116 evaluation templates and distinguish template-level separation from parameter-instance separation. Are the training templates newly constructed?

A3: As clarified in A1, the 400 AndroidWorld training tasks were not instantiated from a predefined set of templates. Instead, they consist of newly generated task instructions produced from exploration trajectories using MobileForge's public task-generation pipeline, without using the 116 official evaluation templates as generation references. The separation is therefore at the task-template level rather than merely between parameter instantiations of shared templates. Training and evaluation use the same AndroidWorld applications, so we characterize the setting as in-environment adaptation.


Q2: Does the 350K contract/state count, and consequently the 550K total, include parser-only and evaluation-only records? Please provide the number of examples actually used to optimize each policy and explain how the 60K preference pairs enter training.

A4: Yes. As detailed in A1, the 350K contract/state count includes 50,000 parser-only and 2,853 evaluation-only records, and the reported 550K total comprises 497,147 constructed records and pairs together with these 52,853 records.

Examples used to optimize each policy. The SFT training inputs for each policy are summarized below.

PolicyTraining inputsSupervision target
Action Policy297,147 base recordsNormalized actions
Contract PolicyThe same 297,147 base recordsComplete contracts, including applicable recovery branches
VerifierCandidate supervision constructed from 60,000 preference pairs and 20,000 calibration recordsQuality scores, verdicts, and confidence

Role of the preference pairs. Each preference pair contrasts an accepted contract with a negative candidate under the same decision input. For Verifier SFT, the candidate contracts in these pairs, together with their structured scores and positive or negative labels, are converted into pointwise supervision and used alongside the 20,000 calibration records. The pairs are not used to train the Action or Contract policies.

GRPO data. The GRPO stage further optimizes the Action Policy using rollouts on the 400 training tasks, with four attempts per task (1,600 attempts in total). Following Section 3.4 and Appendix A.3, we retain tasks that are not solved in all four attempts, select one attempt per retained task, and keep the decision steps that pass the Critic, execution-validity, and Verifier filters. For each retained step, five candidate actions are sampled and evaluated using execution evidence collected from the same restored decision state, and the resulting records are fixed for the GRPO updates.

We have stated these training inputs and clarified the use of the preference pairs explicitly in the revised manuscript.

AuthorsOpenReview ↗

A gentle reminder that the deadline for the Author/Reviewer discussion is approaching

Dear Reviewer Ca3K,

We sincerely appreciate the time and effort you dedicated to reviewing our submission and providing valuable and thoughtful feedback. Your comments have been instrumental in helping us improve the clarity and rigor of our work.

We have carefully considered each of your comments and have done our best to provide comprehensive responses, clarifications, and the requested experiments. As the author–reviewer discussion period is drawing to a close, we would be truly grateful to receive any feedback you might have on our responses before the deadline.

Best regards,

The Authors

Reviewer_4DpfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TRAVEC trains mobile GUI agents using evidence-grounded transition contracts that specify preconditions, actions, expected state changes, observable evidence, and failure-conditioned recovery. Contract and Verifier policies, together with a trajectory Critic, audit interaction data and provide supervision for policy optimization. Candidate actions are executed separately from the same restored state before their records are used for GRPO updates. At deployment, only an action policy remains. The paper reports 550K supervision items and evaluates TRAVEC-9B on four offline and two online benchmarks. Results show substantial gains on several offline metrics and improved MobileWorld success, while the AndroidWorld advantage over the strongest listed baseline is small.

Critical strengths

1.The separation of training and deployment has practical value. Transition contracts organize state changes, verification evidence, and recovery supervision for training-time auditing. The design uses process supervision to improve task performance while maintaining inference latency close to that of the base model.

2.The evaluation offers broad coverage and diverse evidence. Four offline and two online benchmarks cover action prediction, GUI grounding, and multi-step execution. Performance gains, component ablations, and human audits provide preliminary support for effectiveness, component contributions, and supervision quality.

Critical weaknesses

1.The claimed recovery benefits are not directly established for the deployed policy. The paper emphasizes transition verification and failure recovery, but its quantitative evaluation primarily measures action accuracy and overall task success, without systematically assessing recovery after failure or reductions in ineffective loops. Examples and component ablations provide indirect support, but do not establish that the gains arise from the capabilities motivating the method.

2.The contribution beyond alternative training approaches remains insufficiently established. Although the paper includes component and reward ablations, comparisons with representative alternative training methods under matched backbone, data, and training-budget conditions are missing. The comparisons with external models therefore do not clearly separate the benefits of the proposed framework from differences in training data and resources.

Questions for authors

1.The paper emphasizes recovery from failures and the avoidance of repeated ineffective actions, but does not report targeted quantitative evaluations of these capabilities. Beyond overall task success rates, can the authors provide direct evidence that the trained policy improves in these aspects?

2.Although the paper compares several publicly available models under a unified evaluation setup, the differences in their base models, training data, and training investment make it difficult to disentangle the respective contributions of the newly added data and the proposed process-based auditing framework. Is it possible to conduct a comparison with representative open-source methods using the same base model, training data, and training budget, so as to eliminate the confounding effects of data scale and training investment?

3.This paper compares various reward ratios, yet it does not sufficiently explain the rationale behind the selection of these candidate ratios or the internal weighting of individual reward components. Please clarify the design considerations and indicate whether these values were determined based on empirical experience, preliminary experiments, or a systematic parameter search.

4.Semantic interpretations and numerical usage rules are provided for the quality scores; however, further clarification is required regarding the specific scoring criteria for each level, the reliability of the scores, and the rationale and robustness of the confidence threshold used to veto a step following a Verifier rejection.

5.Appropriately add a discussion of limitations to ensure the completeness of the paper.

7 comments
AuthorsOpenReview ↗

Response to Reviewer 4Dpf (1/4)

We sincerely appreciate Reviewer 4Dpf's thorough review and valuable suggestions, which have helped us improve the paper. We have carefully considered each comment and respond below with additional experiments and clearer explanations of our approach.


W1: The claimed recovery benefits are not directly established for the deployed policy. The paper emphasizes transition verification and failure recovery, but its quantitative evaluation primarily measures action accuracy and overall task success, without systematically assessing recovery after failure or reductions in ineffective loops. Examples and component ablations provide indirect support, but do not establish that the gains arise from the capabilities motivating the method.

A1: We thank the reviewer for highlighting the importance of directly evaluating failure recovery and ineffective loops. We agree that action accuracy and overall task success alone do not fully capture these capabilities. To address this concern, we have conducted an additional evaluation of the deployed Action Policy on all 117 MobileWorld tasks, which are not used during training.

For each model, we inspected the complete execution trajectories, including task goals, action sequences, and interface evidence before and after each step. We report the following metrics:

  • Transition failure rate ↓: The proportion of executed steps that fail to produce their intended task-relevant state changes.
  • Recovery rate after first failure ↑: Among trajectories with at least one confirmed transition failure, the proportion in which subsequent actions successfully resolve the first failure, as verified by the resulting interface state.
  • Task success after failure ↑: Among the same trajectories, the proportion that ultimately complete the task successfully.
  • Loop rate ↓: The proportion of trajectories containing at least one repeated ineffective loop, defined as an ineffective action or state cycle that the agent repeats a second time without self-correcting.
ModelTask success ↑Transition failure rate ↓Recovery rate after first failure ↑Task success after failure ↑Loop rate ↓
Qwen3.5-9B5.13% (6/117)76.11%2.63% (3/114)2.63% (3/114)19.66% (23/117)
OpenMobile-8B8.55% (10/117)49.59%13.27% (15/113)5.31% (6/113)49.57% (58/117)
TRAVEC-9B (ours)15.38% (18/117)39.59%35.65% (41/115)13.91% (16/115)23.08% (27/117)

Recovery after failure. TRAVEC achieves a recovery rate of 35.65%, compared with 13.27% for OpenMobile-8B, the strongest baseline on MobileWorld, and 2.63% for the base Qwen3.5-9B model. Its task success rate after failure reaches 13.91%, compared with 5.31% and 2.63%, respectively. TRAVEC also achieves the lowest transition failure rate among the three models.

Ineffective loops. TRAVEC's loop rate is 23.08%, compared with 49.57% for OpenMobile-8B, a relative reduction of 53.45%. While the base Qwen3.5-9B model has a slightly lower loop rate of 19.66%, potentially reflecting its shorter trajectories (88 of 117 ending within 20 steps), it achieves only 5.13% task success, compared with 15.38% for TRAVEC.

Source of the gains. These results directly evaluate failure recovery and ineffective loops in the deployed Action Policy, without auxiliary verification or reflection at inference time. Compared with the base Qwen3.5-9B model sharing the same backbone, TRAVEC improves the recovery rate and task success after failure by 33.02 and 11.28 percentage points, respectively. Together with the matched-backbone training comparisons in A2, these findings provide further evidence that TRAVEC's performance gains are associated with improved failure recovery and process-aware training. These results are reported in Section 4.7 and Appendix E of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer 4Dpf (2/4)

W2: The contribution beyond alternative training approaches remains insufficiently established. Although the paper includes component and reward ablations, comparisons with representative alternative training methods under matched backbone, data, and training-budget conditions are missing. The comparisons with external models therefore do not clearly separate the benefits of the proposed framework from differences in training data and resources.

A2: We thank the reviewer for this suggestion. Since external checkpoints differ in backbone, training data, and resources, a matched comparison can more clearly isolate the contribution of the framework itself. We have therefore added a controlled comparison in which the backbone, Action-SFT data, and evaluation settings are fixed, while the training procedures vary.

  • Action-SFT-only: Qwen3.5-9B fine-tuned with action supervision on the same records used for TRAVEC's Action SFT.
  • SFT + GRPO with action-only rewards: The same SFT model further optimized with GRPO using only the action-type and action-argument rewards. This is the action-only configuration in Table 4, for which we additionally report online results.
  • TRAVEC: The full framework with contract, verification, and recovery signals.
Training configurationAndroidControl High ↑AndroidControl Low ↑ScreenSpot ↑ScreenSpot-Pro ↑AndroidWorld ↑MobileWorld ↑
Action-SFT-only50.25977.53285.45647.81812.9315.983
SFT + GRPO with action-only rewards54.89078.81692.06062.99816.6678.547
TRAVEC-9B64.30993.33792.68967.80535.34515.385

All values are percentages. All three configurations use Qwen3.5-9B.

Performance increases from Action-SFT-only to action-only GRPO and then to full TRAVEC across all six metrics. Compared with Action-SFT-only, TRAVEC improves AndroidWorld and MobileWorld success by 22.414 and 9.402 percentage points, respectively. Compared with GRPO using action-only rewards, TRAVEC improves the two online success rates by 18.678 and 6.838 percentage points and AndroidControl High and Low by 9.419 and 14.521 percentage points.

These comparisons provide additional evidence of TRAVEC's benefits beyond standard action supervision and action-only GRPO. The Action-SFT-only baseline establishes the performance achievable through action supervision alone. The action-only GRPO configuration further controls for the backbone, SFT initialization, rollout data, and GRPO training schedule, allowing us to isolate the additional contribution of process-aware rewards. This configuration follows the action-matching reward design adopted in related GRPO-based GUI agent methods, such as GUI-R1 and MobileForge. The full comparison is reported in Section 4.7 and Appendix E.1 of the revised manuscript.


Q1: The paper emphasizes recovery from failures and the avoidance of repeated ineffective actions, but does not report targeted quantitative evaluations of these capabilities. Beyond overall task success rates, can the authors provide direct evidence that the trained policy improves in these aspects?

A3: We thank the reviewer for this important question. As detailed in A1, we have directly evaluated the deployed Action Policy's failure recovery and ineffective loops using recovery rate after the first failure, task success after failure, and loop rate. To provide further insight, we additionally analyze recovery and loop behavior for each model across two execution-length groups (1–20 and 21+ executed steps).

ModelExecution lengthTrajectoriesRecovery rate after first failure ↑Loop rate ↓
Qwen3.5-9B1–20 steps882.35% (2/85)13.64% (12/88)
Qwen3.5-9B21+ steps293.45% (1/29)37.93% (11/29)
OpenMobile-8B1–20 steps6815.63% (10/64)41.18% (28/68)
OpenMobile-8B21+ steps4910.20% (5/49)61.22% (30/49)
TRAVEC-9B (ours)1–20 steps6036.21% (21/58)23.33% (14/60)
TRAVEC-9B (ours)21+ steps5735.09% (20/57)22.81% (13/57)

TRAVEC achieves the highest recovery rate in both execution-length groups, with similar rates of 36.21% and 35.09%, respectively. Its loop rate also remains similar across the two groups (23.33% and 22.81%), whereas it increases from 13.64% to 37.93% for Qwen3.5-9B and from 41.18% to 61.22% for OpenMobile-8B. In trajectories with 21+ steps, TRAVEC achieves the lowest loop rate among all three models. Although the base Qwen3.5-9B has a lower loop rate in the 1–20 step group, TRAVEC achieves substantially higher recovery performance in the same execution-length group (36.21% vs. 2.35%).

AuthorsOpenReview ↗

Response to Reviewer 4Dpf (3/4)

Q2: Although the paper compares several publicly available models under a unified evaluation setup, the differences in their base models, training data, and training investment make it difficult to disentangle the respective contributions of the newly added data and the proposed process-based auditing framework. Is it possible to conduct a comparison with representative open-source methods using the same base model, training data, and training budget, so as to eliminate the confounding effects of data scale and training investment?

A4: We thank the reviewer for the question. As detailed in A2, we have conducted additional controlled comparisons to better isolate the contributions of action supervision and process-aware training. With the same backbone and training records, action supervision alone reaches 12.931% on AndroidWorld and 5.983% on MobileWorld, compared with 35.345% and 15.385% for full TRAVEC, so the training data alone cannot account for TRAVEC's online improvements. With the SFT initialization, rollout data, and GRPO training schedule also held fixed, incorporating process-aware rewards improves AndroidWorld success from 16.667% to 35.345% and MobileWorld success from 8.547% to 15.385%, providing further evidence for the contribution of process-aware supervision beyond action-only GRPO.


Q3: This paper compares various reward ratios, yet it does not sufficiently explain the rationale behind the selection of these candidate ratios or the internal weighting of individual reward components. Please clarify the design considerations and indicate whether these values were determined based on empirical experience, preliminary experiments, or a systematic parameter search.

A5: The internal weights were fixed by design, while the overall action-to-process ratio was selected through a comparison of five representative configurations. We clarify the rationale for these choices below.

Candidate ratios. Table 4 evaluates five action-to-process reward ratios: 100%:0%, 80%:20%, 50%:50%, 20%:80%, and 0%:100%. These configurations were chosen to cover the two single-objective endpoints, a balanced setting, and two settings dominated by one objective. This provides a coarse comparison of the trade-off between action matching and process supervision rather than an exhaustive parameter search.

Internal weights. Within the action reward, type and argument are weighted 1:7. Since type matching already gates the argument reward (Equation 7), we assign greater weight to argument similarity to emphasize fine-grained action accuracy. The four process rewards receive equal weights to avoid introducing additional preferences among the process supervision signals. Within each process reward, the constituent verifier scores are likewise combined with equal weights and normalized to [0, 1] (Equations 8 and 9). These internal weights were held fixed across all five configurations and were not independently tuned.

Final choice. We adopt the 80%:20% configuration, which achieves the strongest overall performance among the five evaluated settings in Table 4, ranking first on three of the four offline metrics and second on the remaining one. The online benchmarks, AndroidWorld and MobileWorld, were not used for reward-weight selection. We have clarified these design choices and the scope of the weight comparison in the revised manuscript.

Reviewer_KYCoOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TRAVEC, a training framework designed to address state management challenges in long-horizon mobile GUI tasks while maintaining low inference cost and latency. Mobile GUI agents often encounter silent transition failures, state drift, and ineffective recovery during long-horizon execution. These errors can accumulate over time, posing significant challenges to task completion. Existing approaches typically address these issues through online reflection and auxiliary verification, which introduce additional reasoning steps and model calls. TRAVEC addresses these challenges by training a Contract Policy and a Verifier to audit agent-generated trajectories during training. The audited trajectories are then used to optimize the Action Policy through supervised learning and offline GRPO. During deployment, the Action Policy generates executable actions directly, without additional verification or free-form reasoning. The authors construct 550K verifiable supervision items for training TRAVEC and demonstrate strong performance across multiple offline and online benchmarks while maintaining low inference latency.

Critical strengths
  1. The paper presents a valuable framework that unifies transition prediction, outcome verification, and failure recovery as training-time supervision, thereby avoiding additional computational overhead during deployment.
  2. The paper conducts extensive experiments across multiple offline and online benchmarks, providing empirical evidence of the effectiveness of the proposed framework.
Critical weaknesses
  • The paper's primary motivation is to improve the robustness of GUI agents against unexpected state transitions and execution failures in long-horizon tasks. However, the experiments mainly report overall task success rates, without directly evaluating failure recovery. Furthermore, the paper does not independently analyze whether transition failures accumulate as task length increases.
Questions for authors
  • Q1. Can the authors directly evaluate the Action Policy's failure recovery capability?

It would be helpful to report recovery success rates and final task completion rates conditioned on an initial action failure. Such results would provide more direct evidence of the model's ability to recover from execution failures.

  • Q2. Can TRAVEC effectively mitigate error accumulation in long-horizon tasks?

Could the authors group the evaluation results by task length and analyze task success rates, transition failure frequencies, and recovery performance across different task lengths? This analysis would help determine whether TRAVEC effectively mitigates error accumulation as task length increases.

4 comments
AuthorsOpenReview ↗

Response to Reviewer KYCo (1/2)

We sincerely thank Reviewer KYCo for the detailed feedback and constructive suggestions, which have greatly helped improve our work. We have carefully considered every comment and conducted additional analyses of failure recovery and execution length in response. Below, we address each comment individually.


W1: The paper's primary motivation is to improve the robustness of GUI agents against unexpected state transitions and execution failures in long-horizon tasks. However, the experiments mainly report overall task success rates, without directly evaluating failure recovery.

A1: We thank the reviewer for highlighting the importance of directly evaluating failure recovery, which overall task success rates alone do not fully capture. To address this concern, we have conducted an additional evaluation on all 117 MobileWorld tasks, which are not used during training.

For each model, we inspected the complete execution trajectories, including task goals, action sequences, and interface evidence before and after each step. We report the following metrics:

  • Transition failure rate ↓: The proportion of executed steps that fail to produce their intended task-relevant state changes.
  • Recovery rate after first failure ↑: Among trajectories with at least one confirmed transition failure, the proportion in which subsequent actions successfully resolve the first failure, as verified by the resulting interface state.
  • Task success after failure ↑: Among the same trajectories, the proportion that ultimately complete the task successfully.
ModelTask success ↑Transition failure rate ↓Recovery rate after first failure ↑Task success after failure ↑
Qwen3.5-9B5.13% (6/117)76.11%2.63% (3/114)2.63% (3/114)
OpenMobile-8B8.55% (10/117)49.59%13.27% (15/113)5.31% (6/113)
TRAVEC-9B (ours)15.38% (18/117)39.59%35.65% (41/115)13.91% (16/115)

These results offer direct quantitative evidence for the deployed Action Policy's failure recovery capability. Compared with OpenMobile-8B, the strongest baseline on MobileWorld, TRAVEC reduces the transition failure rate by 10.00 percentage points and improves the recovery rate after the first failure by 22.38 percentage points. Its task success rate after failure is 13.91%, compared with 5.31% for OpenMobile-8B. TRAVEC achieves these improvements while retaining an action-only policy without auxiliary verification or reflection at inference time. These results are reported in Section 4.7 and Appendix E of the revised manuscript.


W2: Furthermore, the paper does not independently analyze whether transition failures accumulate as task length increases.

A2: We thank the reviewer for this valuable suggestion. We agree that analyzing transition failures across different execution lengths can provide further insight into the robustness of GUI agents. To address this concern, we have conducted an additional analysis grouping each model's 117 MobileWorld trajectories by the number of steps it executed (1–20 and 21+ steps), using the same evaluation protocol and metrics described in A1.

ModelExecution lengthTrajectoriesTask success ↑Transition failure rate ↓Recovery rate after first failure ↑Task success after failure ↑
Qwen3.5-9B1–20 steps885.68% (5/88)64.93%2.35% (2/85)2.35% (2/85)
Qwen3.5-9B21+ steps293.45% (1/29)93.59%3.45% (1/29)3.45% (1/29)
OpenMobile-8B1–20 steps6811.76% (8/68)42.96%15.63% (10/64)6.25% (4/64)
OpenMobile-8B21+ steps494.08% (2/49)55.54%10.20% (5/49)4.08% (2/49)
TRAVEC-9B (ours)1–20 steps6016.67% (10/60)28.17%36.21% (21/58)13.79% (8/58)
TRAVEC-9B (ours)21+ steps5714.04% (8/57)47.04%35.09% (20/57)14.04% (8/57)

Transition failures are more frequent in longer execution trajectories across all three models. From the 1–20 step group to the 21+ step group, the transition failure rate increases from 64.93% to 93.59% for Qwen3.5-9B, from 42.96% to 55.54% for OpenMobile-8B, and from 28.17% to 47.04% for TRAVEC. Task success rates also decrease across all three models, consistent with the greater challenges of longer execution trajectories discussed in prior work [1][2][3].

AuthorsOpenReview ↗

Response to Reviewer KYCo (2/2)

Importantly, TRAVEC maintains similar recovery rates across the two execution-length groups, achieving 36.21% and 35.09%, respectively. It also achieves the lowest transition failure rate and the highest task success rate in both groups. For trajectories with 21+ steps, TRAVEC achieves 14.04% task success, compared with 4.08% for OpenMobile-8B and 3.45% for Qwen3.5-9B. Although TRAVEC also experiences more transition failures in longer trajectories, these results provide additional evidence of its ability to recover from failures and maintain better task completion performance. This breakdown is reported in Appendix E.3 of the revised manuscript.

References

[1] Bin Kang, Shaoguo Wen, Yifei Bi, et al. LongHorizonUI: A Unified Framework for Robust long-horizon Task Automation of GUI Agent. ICLR, 2026.

[2] Zixiang Chen, Yuheng Lu, Zihao Cheng, et al. JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition. EMNLP, 2026.

[3] Hy Vision Team, Huawen Shen, Zhengyang Tang, et al. HyMobileAgent: Data-Environment Co-Scaling for Efficient GUI Agents. arXiv:2607.14548, 2026.


Q1: Can the authors directly evaluate the Action Policy's failure recovery capability? It would be helpful to report recovery success rates and final task completion rates conditioned on an initial action failure. Such results would provide more direct evidence of the model's ability to recover from execution failures.

A3: We thank the reviewer for this important question. As detailed in A1, we have directly evaluated the deployed Action Policy's recovery capability following the first confirmed transition failure in each trajectory. The Recovery rate after first failure and Task success after failure metrics correspond to the two quantities requested by the reviewer.

TRAVEC successfully recovers from the first failure in 35.65% (41/115) of affected trajectories and ultimately achieves task success in 13.91% (16/115) of them. Compared with the base Qwen3.5-9B model, which shares the same backbone, these represent improvements of 33.02 and 11.28 percentage points, respectively. Compared with OpenMobile-8B, the strongest baseline on MobileWorld, the improvements are 22.38 and 8.60 percentage points, respectively. These results provide direct evidence of failure recovery using only the deployed Action Policy, without auxiliary verification or reflection at inference time.


Q2: Can TRAVEC effectively mitigate error accumulation in long-horizon tasks? Could the authors group the evaluation results by task length and analyze task success rates, transition failure frequencies, and recovery performance across different task lengths? This analysis would help determine whether TRAVEC effectively mitigates error accumulation as task length increases.

A4: We thank the reviewer for this thoughtful question. As detailed in A2, we have analyzed task success, transition failure rates, and recovery performance across two execution-length groups (1–20 and 21+ steps) for each model.

Our results suggest that TRAVEC exhibits greater robustness in longer execution trajectories, although transition failures become more frequent for all three models. TRAVEC's transition failure rate increases from 28.17% to 47.04%, while its recovery rate after the first failure remains similar across the two groups (36.21% and 35.09%). In comparison, OpenMobile-8B's recovery rate decreases from 15.63% to 10.20%. TRAVEC also maintains task success rates of 16.67% and 14.04% in the two groups, respectively, whereas OpenMobile-8B's task success decreases from 11.76% to 4.08%. Taken together, these findings suggest that TRAVEC's improved recovery capability helps mitigate the impact of transition failures on task completion in longer execution trajectories.

I appreciate the detailed response and additional analyses. I'd maintain my positive rating.

Dear Reviewer KYCo,

Thank you for your time and effort in reviewing our paper and reading our responses. We are glad that the additional analyses addressed your concerns, and we truly appreciate your constructive feedback and positive assessment.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Authors’ Global Response and Summary of Revisions

We sincerely thank Reviewers KYCo, 4Dpf, and Ca3K for their thoughtful feedback. We are encouraged that they recognized the practical value of training-time supervision with action-only deployment (KYCo, 4Dpf, Ca3K), the useful design of evidence-grounded transition contracts that organize expected state changes, verification evidence, and recovery supervision (4Dpf, Ca3K), and the broad evaluation across four offline and two online benchmarks (KYCo, 4Dpf, Ca3K).

The reviewers' comments and questions have helped us improve the paper. We have uploaded a revised draft with all changes marked in red. Sections, tables, and appendices cited below refer to this revised draft. We summarize the major updates below.

  • Direct evaluation of failure recovery. We evaluate the deployed action-only policies on all 117 MobileWorld tasks and report transition failure rate, recovery rate after the first failure, task success after failure, and loop rate. (Section 4.7, Table 5; Appendix E.2)
  • Analysis by execution length. We break down task success, transition failures, recovery, and ineffective loops by execution length. (Appendix E.3, Table 11)
  • Matched-backbone training controls. We add an Action-SFT-only baseline and online results for action-only GRPO under a matched backbone, Action-SFT data, and evaluation settings. (Section 4.7; Appendix E.1, Table 10)
  • Training/evaluation separation. We clarify the generation of the 400 AndroidWorld training tasks and their relationship to the 116 evaluation templates. (Section 4.2)
  • Supervision counts and training inputs. We detail the composition of the 550K items and report the training inputs of each policy. (Abstract, Sections 1, 3.3, and 4.2; Appendix B.5, Tables 6 and 7)
  • Reward weights, scoring criteria, and veto threshold. We explain the rationale for the reward weights, the criteria for each score level, and the design of the confidence threshold. (Section 4.6; Appendices A.1, A.3, and B.2)
  • Limitations and future work. We add a discussion of limitations and future directions. (Section 6)

Upon acceptance, we will release the constructed annotations, including the transition contracts, negative and recovery records, preference pairs, and verifier-calibration records, to support reproducibility and future research.

We hope these updates address the reviewers' main concerns. Please see our responses to each comment below.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.