- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper studies how small edge models and large cloud models cooperate on GUI agent tasks. The paper adds two checks: an edge module that tracks multi-step performance drops, and a cloud gate that rejects hopeless tasks early. Experiments show higher success rates while lower cloud step usage.
Critical strengths
The proposed method is intuitive.
Experiments show a performance gain.
Critical weaknesses
-
Although the performance gain is valid, and the proposed method is intuitive, this work is generally an engineering work instead of a research work.
-
The decision modules and LoRA weights are trained on the hidden states and trajectories of a specific GUI agent, so they cannot generalize to new agents or different failure patterns without retraining from scratch.
-
Collecting training labels for each agent requires running 5 rollouts per intermediate state, which is very expensive to repeat whenever the base agent changes.
Questions for authors
Please address the weaknesses.
3 comments
thanks for your response
i will maintain my original rating
Thank you for your feedback. We appreciate your time and consideration.
W1: Research Contributions
We appreciate the reviewer’s recognition of the performance gains and the intuition behind GUI-ECO. We respectfully clarify that our contribution goes beyond integrating existing components: GUI-ECO investigates how task-completion competence can be learned from agents’ internal representations and used to guide edge--cloud collaboration. Specifically, our work makes the following contributions:
1. Formulating collaboration as two distinct decisions. We distinguish whether the edge agent needs assistance from whether the cloud agent is likely to complete the remaining task. GUI-ECO explicitly models both through edge-side handoff and cloud-side acceptance, rather than treating a request for assistance as sufficient justification for cloud execution. This formulation allows us to investigate when cloud intervention is beneficial and when further execution is unlikely to justify its cost.
2. Investigating temporal evidence for handoff decisions. We study whether the evolution of competence scores provides useful information beyond an individual state-wise estimate. This addresses a substantive decision-making problem: a temporary error or low score does not necessarily imply that the agent cannot recover and complete the task.
3. Learning agent-specific task-completion competence. We investigate whether an agent’s internal representations can predict its likelihood of completing the remaining task from the current GUI state. Repeated continuation rollouts provide outcome-grounded supervision for this prediction. Unlike directly predicting routing labels, our approach explicitly estimates the agent’s future task-completion probability from its current state, rather than task difficulty or individual action correctness.
4. Adapting representations for competence estimation. We further investigate whether competence information can be made more accessible through joint head–model training, rather than only extracted from frozen representations. LoRA adapts the representations consumed by the SCE head, BCE anchors predictions to empirical rollout success rates, and pairwise margin enhancement encourages appropriately ordered and separated scores across competence levels.
5. Evaluating these design choices through controlled experiments. Our evaluation across three agent pairs and four desktop/mobile benchmarks examines the task-success–cloud-execution trade-off under different model capabilities and environments. Complementary ablations investigate the roles of joint training, trajectory-level modeling, and cloud-side acceptance, while case studies illustrate how these mechanisms affect individual execution trajectories.
Taken together, these contributions concern problem formulation, competence representation learning, and sequential decision-making, rather than implementation alone.
W2: Agent-Specific Adaptation and Generalization Across Different Patterns
Although our decision modules and LoRA adapters are agent-specific, adapting GUI-ECO to a new agent requires only a modest model-training workload; the data-collection overhead is discussed separately in our response to W3. We collect training data from only two datasets and the additional components are lightweight and efficient to train, with their sizes reported in Table 4 of the original submission. For our largest evaluated agent, GUI-Owl-1.5-32B, LoRA adaptation takes approximately one hour on an NVIDIA A100 GPU. Furthermore, our OOD evaluations on MobileWorld and VisualWebArena show that the trained modules generalize across datasets and tasks without retraining, suggesting that they capture transferable competence and failure patterns rather than merely fitting the training trajectories.
W3: Rollout Data Collection Cost
We acknowledge that collecting five continuation rollouts per sampled state introduces an agent-specific data-collection cost. However, we consider this a manageable offline investment for operator-controlled cloud deployments. Such data collection is performed offline and can be automated and parallelized: for example, RoTS[1], developed by Alibaba Cloud Computing, hosts GUI environments on elastic cloud infrastructure and uses a rollout manager to coordinate asynchronous parallel sampling across distributed environment and agent servers.
Our data collection is restricted to sampled intermediate states from OSWorld and AndroidWorld, without requiring additional supervision for each deployment environment. Even for our largest evaluated agent, GUI-Owl-1.5-32B, collecting the complete rollout dataset required only approximately 100 NVIDIA A100 GPU-hours, representing a relatively modest and manageable offline cost for cloud service providers.
We sincerely hope that our responses can adequately address your concerns.
[1] Tianpeng Bu et al. Recovering Policy-Induced Errors: Benchmarking and Trajectory Synthesis for Robust GUI Agents. ICML, 2026.