acceptodds
Withdrawn from ICLR 2027

Cobra: Co-Training Bi-Level Observation Routing and Action Policies for GUI Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

16 views

Abstract

The information a GUI agent needs changes from one action to the next, but existing agents typically construct the actor's context with a fixed rule. This can omit useful interface structure or introduce irrelevant information. We model observation selection and action generation as a bi-level partially observable Markov decision process and introduce Cobra (Co-trained Bi-level Routing and Action) to train both policies jointly online. At each step, a lightweight router, GUIRouter, chooses between the screenshot and accessibility (a11y) information and selects the relevant nodes. To learn routing from sparse terminal rewards, we introduce CMA (Counterfactual Modality Advantage), which compares terminal outcomes across routing choices at comparable information states without a learned value network. We analyze when a shared adapter update fails to reduce both modality losses and use two modality-specific LoRA adapters on a frozen shared 8B backbone. With the same initialization, training tasks, and rollout budget, Cobra outperforms the strongest of three fixed-observation online-RL controls by 5.5 percentage points on OSWorld-Verified. It also reduces average per-step a11y tokens by 86% compared with fusing visual input and the a11y tree. Compared with its Qwen3-VL-8B-Thinking base actor, Cobra improves OSWorld-Verified success from 33.9% to 46.3%, with consistent gains on AndroidWorld and Windows Agent Arena.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
GUI agents, observation routing, counterfactual credit assignment, reinforcement learning, dual-LoRA adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_fTfjOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Cobra, a framework that jointly trains observation routing and action policies for GUI agents. A router decides whether to supplement screenshots with a pruned accessibility tree, while modality-specific LoRA adapters train the actor for the two observation formats. The method introduces Counterfactual Modality Advantage to provide step-level routing credit from terminal rewards. Experiments on three GUI benchmarks show improved task success, with controlled comparisons and ablations on OSWorld-Verified.

Critical strengths
  1. The paper addresses an important problem for GUI agents: deciding when structural information is useful and which parts should reach the actor. Learning this decision from task outcomes is a practical direction for improving observation use.

  2. The evaluation spans Linux, Android, and Windows environments. The fixed-observation comparisons and component ablations on OSWorld-Verified provide useful evidence for the proposed design.

  3. The paper is generally clear and well organized, with theoretical analysis and implementation details that help explain the method.

Critical weaknesses
  1. The performance degradation from combining vision and accessibility trees needs further analysis. In Table 2, Always-fused Online RL achieves 39.6% success, below the 40.8% of Visual-only Online RL. This is somewhat surprising and directly relevant to the motivation for dynamic routing. The paper discusses redundant information, stale nodes, and modality conflicts, but does not establish which factors cause the degradation. Controlling tree length, filtering irrelevant nodes, and examining failures would help distinguish information quality from context overhead and optimization issues. The shared-adapter gradient analysis does not directly explain why always-fused input underperforms.

  2. The lack of training-free comparisons makes it difficult to isolate the router's value. I would like to see the same frozen actor evaluated under four inference-time settings without additional training: vision only, accessibility tree only, both inputs, and router-selected inputs. Tasks, interaction budgets, and decoding settings should be identical, and the router's training status should be specified. The existing untrained-router ablation still trains the actor and therefore does not replace this comparison. This experiment would clarify whether directly adding structural information is already effective and how much additional benefit routing provides.

  3. The cost analysis excludes router overhead and does not establish an overall efficiency advantage. The reported token savings concern the actor, but the router also processes screenshots, rule-compressed accessibility trees, and history at every step. Some costs may simply be transferred to the router. A fair system-level comparison should include both models' input, output, and computational costs, together with end-to-end latency and average cost per successful task. The current results demonstrate shorter actor context, but whether these savings offset the additional routing overhead remains unclear.

  4. The main experiments lack uncertainty estimates. Several differences are relatively small: the 1.2 percentage-point decrease from always-fused input, the 2.1-point gain from CMA, and the 1.1-point gain over always-fused input on tasks excluded from updates. Although the appendix reports three evaluations per checkpoint, this does not capture training variability. Results across independent training seeds and paired confidence intervals for performance differences would help establish whether these findings are stable enough to support the paper's explanations and generalization claims.

Questions for authors

Refer to the weaknesses.

Reviewer_JBrSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces COBRA, which jointly trains an observation router and a GUI action policy. It uses a 2B router decides whether to supplement the current screenshot with a task-relevant, pruned accessibility tree, while an 8B actor uses separate LoRA adapters for the two observation branches. Router training combines episode-level rewards with Counterfactual Modality Advantage (CMA), estimated by comparing modality-conditioned outcomes within neighborhoods of similar router states. Experiments report 46.3% success on OSWorld-Verified versus 40.8% for the strongest fixed-observation RL control, alongside improvements on AndroidWorld and Windows Agent Arena and reductions in actor-context tokens.

Critical strengths
  1. This paper presents a coherent approach to a practically relevant observation-management problem. The design connects selective admission of accessibility information, task-conditioned pruning, and actor specialization within one learning framework. Retaining screenshots while selectively adding structured evidence is a sensible way to support both visual interpretation and precise control identification. Evaluation across Linux, Android, and Windows establishes practical breadth.

  2. It conducts useful ablations to support the value of learning the router. With the same dual-adapter actor configuration, success increases from 42.4% with an untrained router, to 44.2% with episode-level router RL, to 46.3% with CMA (Table 2). This progression provides meaningful evidence that router optimization and its additional credit signal contribute beyond the initial routing heuristic.

Critical weaknesses
  1. The headline OSWorld evaluation includes all 361 tasks after training on 184 of them. The authors transparently disclose this setup, and the comparison is informative about learning within that pool. On the remaining 177 tasks, however, COBRA achieves 28.3% versus 27.2% for always-fused RL, a 1.1-point advantage (Table 3). This subset is also selected by the base actor’s success/failure behavior, which limits its representativeness. Although Appendix D reports three evaluation runs per checkpoint, uncertainty estimates and independent training-seed results are absent. Consequently, the reliability and generality of the improvement over a simpler fusion strategy remain unclear.

  2. The fixed-observation controls use full-parameter fine-tuning, whereas COBRA freezes the backbone and trains specialized adapters (4.1). Moreover, the always-fused control receives the rule-compressed tree, while COBRA additionally performs learned task-conditioned pruning. An always-admit-pruned-tree control with matched actor adaptation is therefore needed to establish what the learned admission gate contributes beyond pruning and specialization. The adapter comparison also requires clarification: 4.3 describes a shared rank-128 adapter as matching active per-step rank and compute, while Table 4 specifies rank 64 for each routed adapter. These issues weaken the causal attribution of the reported gains.

Questions for authors
  1. Which task instances enter SFT and online RL on each benchmark, particularly AndroidWorld and Windows Agent Arena? Can you provide paired uncertainty estimates for the excluded-task results in Table 3 and clarify whether the gains persist across independent training seeds?

  2. Please reconcile the shared and dual-adapter ranks in §4.3 and Table 4, specifying both total trainable parameters and active per-step parameters. Does COBRA retain an advantage when fixed-observation controls use comparable LoRA adaptation, and when learned pruning is retained but the modality gate always admits the pruned tree?

  3. Equations (4)–(5) and Algorithm 1 appear to apply the combined advantage to the entire router JSON, while the text states that CMA sharpens only the modality gate. Is a modality-specific likelihood or token mask used? Additionally, what evidence supports the reliability of the approximate neighborhoods at (\epsilon=0.1), such as threshold sensitivity or comparison with branching from identical saved environment states?

Reviewer_mbZaOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies the issue of "what to look at" by GUI Agents in each decision-making step. Existing methods usually use one modality fixedly, or unconditionally splice the two, resulting in either a lack of key structural information or the introduction of a large number of redundant tokens (a11y Trees often have tens of thousands of tokens). So a lightweight model was used to decide what model to use, and the lora adapter was trained on the 8B model, which improved the score of the evaluation set.

Critical strengths
  • Ablation experiments: Comparing router training, dual-LoRA vs. single LoRA, and three fixed observation strategies one by one, the contribution of each component is quantified to specific percentage points.
  • Cross-task/cross-platform verification: Not only valid on the subset of OSWorld tasks used for training, but also on tasks that do not participate in training updates and on AndroidWorld and Windows Agent Arena.
Critical weaknesses
  • The method is too tricky and lacks use value. Now the model capability is strong enough, and training two LoRAs does not make much sense
  • I don't understand why CMA is used here, it seems to be a patchwork for innovation
Questions for authors

The only question is why not train both abilities into one model? Is there any evidence that the two abilities conflict?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.