acceptodds
Withdrawn from ICLR 2027

MemGUI-RL: Reinforcement Learning for Proactive Context Management in Long-Horizon Mobile GUI Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Long-horizon mobile GUI agents must decide not only which button to tap but also what to remember: which history to compress, which screen facts to keep, and when. **MemGUI-Agent** casts these decisions as first-class actions through the text-as-ion (**CONACT**) interface, but its supervised policy only imitates demonstrations and never compares alternative memory decisions sampled from itself. We study reinforcement-learning post-training of such policy-managed memory and find that group relative policy optimization (**GRPO**), applied naively, fails in two ways. ***First***, aggregating heterogeneous verifier rewards (format, action type, parameters, folding range) before group normalization lets the component with the largest within-group spread dictate the update, regardless of the intended weights. ***Second***, span-level history folds, the operation that actually compresses context, are only 22.7% of annotated folds, and the majority, step folds, are almost always already correct, so under natural sampling most prompt groups receive identical rewards and the policy collapses to step-only folding (deep-fold rate 17.7% → 0.2%). ***(i)*** We introduce olding-ware eward-Decoupled olicy ptimization (**FARPO**), which standardizes each verifier component within its group before weighting and controls span exposure through a span-to-step ratio ; both mechanisms are characterized analytically. ***(ii)*** The resulting **MemGUI-8B-RL** reaches 48.4% Pass@3 and 39.0% information retention on MemGUI-Bench and 19.7% success on the out-of-distribution MobileWorld, more than doubling the backbone model on every metric, with the gain concentrated on memory-intensive tasks; it is the best open-data 8B model on both benchmarks. ***(iii)*** A seven-ratio sweep shows folding exposure to be a controllable hyper-parameter with a bimodal footprint, and a stepwise ablation attributes the gain to each mechanism. ***An anonymized release of the code, data, trained model, and evaluation logs is publicly available at [https://memgui-rl-anonymous.github.io/](https://memgui-rl-anonymous.github.io/).***

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
mobile GUI agents, reinforcement learning, context management, memory management, long-horizon agents, policy optimization, GRPO

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_umFKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies RL post-training for memory management in long-horizon GUI agents. Building on MemGUI-Agent's ConAct interface and MemGUI-8B-SFT, the authors propose FARPO, which combines GDPO-style per-component reward normalization with resampling of span- and step-folding examples to mitigate differences in reward scale and insufficient training signal for span folding. Training uses fixed annotated states from MemGUI-3K: the current policy samples candidate responses, and a rule-based verifier assigns rewards based on agreement with the annotations, without device interaction during training. The resulting model achieves 48.4% Pass@3 and 39.0% information retention on MemGUI-Bench, and a 19.7% success rate on MobileWorld GUI-only.

Critical strengths
  1. The experimental analysis is reasonably thorough. Stepwise ablations, experiments with seven data mixtures, and measurements of zero-advantage groups help characterize how reward normalization and sample composition affect the policy. In particular, the results reveal substantial changes in folding behavior across training mixtures and a non-monotonic relationship between folding frequency and task performance, providing useful empirical observations for further research.
  2. The paper is easy to follow, with well-presented figures and tables. The anonymous project page is particularly helpful: it clearly summarizes the central ideas and results and makes the paper easier to assess.
Critical weaknesses
  1. The problem formulation is narrow, and the paper does not establish that the identified challenges are shared bottlenecks in the field. The two challenges depend on the authors' four-component reward design and the span/step annotation distribution in MemGUI-3K, respectively. Although reward-scale differences and insufficient training signal may arise more broadly, the experiments cover only the ConAct interface, a single dataset, and one verifier. The contribution therefore appears primarily to refine the post-training procedure of MemGUI-Agent.
  2. The methodological novelty is limited. Reward decoupling directly adopts GDPO's per-component normalization, while folding-aware sampling uses conventional majority-class downsampling. The paper does not sufficiently establish a distinct methodological contribution from combining these components. Moreover, the ablations omit a "scalar GRPO + folding-aware sampling" control, leaving unclear whether resampling alone accounts for most of the gains and whether the two components meaningfully interact.
  3. The rationale for the high span-folding rate remains insufficiently validated. The original ConAct design treats single-step distillation and multi-step consolidation after subtask completion as complementary operations; a low span-folding rate therefore does not necessarily imply inadequate compression. FARPO increases the deployment-time span-folding rate to 91.7%, but the paper does not adequately examine whether these folds occur at appropriate subtask boundaries or reflect a tendency to perform multi-step compression at nearly every step. Higher task scores alone do not establish that the model has learned to select folding timing and granularity according to the current state.
  4. The paper does not evaluate FARPO with larger models or increasing amounts of training data, leaving its generality and scalability untested. Pass@3 on the hardest task group decreases from 34.2% for the SFT model to 28.9%, while the improvement on MobileWorld amounts to only two additional completed tasks. These results provide limited evidence of a substantial improvement on out-of-distribution tasks.
Questions for authors
  1. The PDF is titled "FARPO: Folding-Aware Reward-Decoupled Policy Optimization for Long-Horizon Mobile GUI Agents", whereas the submission title is "MemGUI-RL: Reinforcement Learning for Proactive Context Management in Long-Horizon Mobile GUI Agents." This may simply reflect an unsynchronized title revision, and I would not regard the discrepancy alone as a policy violation. Could the authors clarify the reason for the difference?
  2. Why should the original span-folding rate be regarded as a training bottleneck rather than a reasonable outcome of the teacher's state-dependent decisions? The latter possibility seems central to understanding the underlying problem.
  3. Of the span folds that account for 91.7% of folding operations at deployment, what proportion occurs at clear subtask boundaries, and what proportion repeatedly compresses existing summaries?
Reviewer_cr4ROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Folding-Aware Reward-Decoupled Policy Optimization (FARPO) for post-training mobile GUI agents with policy-managed memory. Starting from MemGUI-8B-SFT, it samples responses on recorded GUI states and evaluates them against annotations using format, action-type, action-parameter, and folding-range rewards. FARPO combines GDPO-style per-component reward normalization with a sampling scheme that increases exposure to span-folding examples. The paper analyzes normalization properties and evaluates the resulting policy on MemGUI-Bench and MobileWorld. Validation includes a stepwise ablation, a sampling-ratio sweep, offline memory metrics, and trajectory analyses. The reported improvements are substantial on MemGUI-Bench but smaller on MobileWorld.

Critical strengths
  1. A practically relevant training approach with meaningful improvements over the appropriate SFT starting point. Training on recorded states avoids the cost of device interaction during RL while permitting downstream evaluation in interactive environments. Relative to MemGUI-8B-SFT, the full method improves MemGUI-Bench Pass@3 from 35.9% to 48.4% and information retention from 30.2% to 39.0%. These comparisons provide more meaningful evidence for the RL contribution than the larger improvements over the original backbone.

  2. Useful diagnostics of how training changes memory behavior. The paper examines zero-advantage groups, span-fold frequency, memory-operation accuracy, and deployment trajectories rather than reporting task success alone. The fixed-100-step, size-matched ablation provides evidence that normalization and sampling each contribute along the tested sequence of configurations. The reported disagreement between folding-range accuracy and downstream performance is also informative: it exposes limitations of annotation-based memory rewards and helps identify where subsequent improvements are needed.

Critical weaknesses
  1. The incremental contribution beyond existing methods is limited and insufficiently isolated. The ConAct interface is inherited unchanged, and the per-component normalization followed by batch whitening is explicitly adopted from GDPO. The distinctive intervention is therefore primarily the sampling prior over span versus step folds. This can be a useful empirical contribution, but stronger controls are needed to establish its significance. Table 5 omits the scalar-normalization-plus-folding-aware-sampling configuration, so it does not fully separate the effects and interaction of the two components. Continued SFT on the same resampled corpus would also help determine how much improvement requires the proposed RL procedure. Furthermore, the ratio sweep varies training duration from 90 to 398 updates, limiting conclusions about the effect of the ratio alone. The analytical propositions clarify estimator properties but do not substantially strengthen the novelty of the underlying normalization method.

  2. The evaluation does not yet establish sufficiently reliable generalization or improvements on the longest tasks. The 40-task subset used to select the sampling ratio is included in the 128-task benchmark used for the headline score. Thus, the reported aggregate is not entirely independent of model selection; results on the remaining 88 tasks are important. On MobileWorld, the improvement over the actual SFT starting point is only two additional solved tasks, from 21/117 to 23/117. The second training seed is evaluated only through offline metrics, which does not establish reproducibility of this downstream gain. Finally, on the Hard tier of MemGUI-Bench, Pass@3 decreases from 34.2% to 28.9%—13 versus 11 solved tasks. The paper acknowledges this limitation, but it materially narrows the evidence for improved long-horizon memory management.

Questions for authors
  1. Evaluation clarification: Can you report the main methods separately on the 88 MemGUI-Bench tasks excluded from the development subset? Please also clarify whether MobileWorld results influenced any configuration or checkpoint choices.

  2. Robustness assessment: Can the second training seed be evaluated on both downstream benchmarks, with paired task-level comparisons and uncertainty estimates? This is particularly important for determining whether the two-task MobileWorld improvement is reproducible.

  3. Decisive ablation: What happens when scalar normalization is combined with the same folding-aware sampling used by FARPO? How does continued SFT on that same resampled corpus perform under a comparable budget? These controls would materially affect my assessment of the contribution beyond normalization and data rebalancing.

  4. Implementation clarification: Equation (6), together with the reported natural span fraction of 22.7%, appears to imply that (\rho=1/4) retains all available step examples and therefore recovers the natural corpus. Why do these two rows in Table 7 require 398 and 264 updates per epoch, respectively? Please identify any additional differences in filtering, corpus construction, or batching, and clarify their implications for the ratio sweep.

Reviewer_kJ4aOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies RL post-training of the ConAct memory-management policy used by MemGUI-Agent. Training samples responses at recorded, annotated GUI states and rewards agreement with reference actions and folding ranges, without executing device rollouts during optimization. FARPO combines per-component reward normalization, adopted from GDPO, with sampling that increases exposure to span folds relative to single-step folds. Starting from MemGUI-8B-SFT, it raises MemGUI-Bench Pass@3 from 35.9% to 48.4% and information retention from 30.2% to 39.0%, while MobileWorld success rises from 21/117 to 23/117 tasks. The paper includes a fixed-step component ablation, a seven-ratio sweep, memory-use diagnostics, and failure analysis.

Critical strengths
  1. The paper identifies a concrete training failure and measures its behavioral consequences. Sections 4.4–4.5 and Tables 7–8 connect the sampling ratio to zero-advantage groups, validation fold behavior, and actual deployment traces. Natural sampling produces almost exclusively step folds, whereas the proposed setting produces span folds in over 90% of cases. This is a useful diagnosis of why applying RL to an existing memory policy can fail even when conventional validation metrics improve.
  2. The comparison against the SFT initialization is informative and fairly transparent. Table 5 holds training steps and corpus size fixed while adding reward decoupling and then sampling. The appendix reports degraded fold-range agreement and the lack of progress on the hardest tasks, rather than hiding these outcomes. The overall gains in task success and information retention give the recipe practical value, and the direct-RL control makes clear how much it relies on the existing ConAct supervision.
Critical weaknesses
  1. The methodological contribution is narrow, and its added value is not fully isolated. FARPO builds on the existing ConAct interface and training corpus, and adopts reward decoupling from GDPO (Sections 2–3). Its main new ingredient is rebalancing span-fold examples. In Table 5, inherited reward decoupling adds 6.2 Pass@3 points, while the sampling change adds 3.1 points and only 0.5 retention points. The missing scalar-GRPO/rho = 9 control leaves unclear how much the combined recipe offers beyond ordinary GRPO with the same data rebalancing. This limits the evidence for a substantial methodological advance.
  2. The reward and evaluation do not establish that FARPO learns better state-dependent memory decisions. The folding reward scores interval agreement but ignores summary content (Section 2; Appendix B.5), so it does not directly distinguish useful compression from losing critical information. Deployment span-fold frequency rises from 13.3% to 91.7% (Table 8), yet Hard-task Pass@3 drops from 34.2% to 28.9% and retention from 32.6% to 29.6% (Table 4). These results leave open whether FARPO learns when and what to compress or mainly a stronger compression preference. The MobileWorld gain is also only 21 to 23 solved tasks out of 117; repeated evaluations are needed to assess whether this exceeds evaluation variability.
Questions for authors
  1. How does scalar GRPO with rho = 9 perform under the same 100-step budget and corpus size? Please compare it with FARPO on task success and retention, including the Hard subset, to isolate the value of reward decoupling once span exposure is matched.
  2. What evidence links the gains to better memory preservation rather than a general preference for compression? A control with a fixed folding schedule at a comparable compression rate, together with an audit of critical information retained in summaries, would help explain the improvement and the regression on Hard tasks.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.