- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes to extend CUA to novel software by training a planner and keep the GUI executor unchanged. The proposed framework has two stages. In stage 1, separate planners are trained for each software using GRPO, and the reward is based on similarities with trajectories approved by the judge. In stage 2, a generalist planner is fine-tuned on trajectories collected from software-specific planners obtained in stage 1. The proposed framework is evaluated on multiple benchmarks including ScienceBoard, TeXstudio, and OSWorld.
Critical strengths
- Given the complexity of different types of software and GUI executors, training the planner only is a promising idea to make the proposed framework generalizable.
- The evaluation protocol is comprehensive. Besides comparisons with baselines, the authors also analyze the effect of reward source, planner backbone, executor, and judge quality. I feel the judge-quality study is particularly informative.
Critical weaknesses
-
Technical novelty is not that significant in my view. Training the planner alone via GRPO is not a new idea and has been proposed in AgentFlow [1]. They key motivation is that training gradient cannot propagate through the executor. Similar ideas have also been explore in harness-related studies, where the harness prevents gradient from propagating back to the agent.
-
The contribution of RL is not well-isolated. Based on my understanding, RL is only used in stage 1, while stage 2 is based on SFT. Therefore, to examine the contribution of RL, the ablation version should be training the generalist planner on trajectories generated by the original Qwen model without any RL. However, this is not the case according to the explanation of variants (iv) and (vi) in Table 8.
[1] In-the-Flow Agentic System Optimization for Effective Planning and Tool Use. ICLR 2026.
Questions for authors
- What is the technical novelty of this work in comparison with AgentFlow?
- What are the differences between variants (iv) and (vi) in Table 8? Is this comparison fair enough to validate the contribution of RL?
2 comments
I thank the authors for their detailed response.
The clarification of Table 8 is helpful and addresses my second question.
However, my major concern about the technical novelty remains. The authors describe AgentFlow as a "short-horizon, largely single-turn" setting, which is inaccurate. Based on my understanding, it optimizes the planner explicitly within a multi-turn interaction loop. Dividing the optimization into each turn does not mean it has a single-turn setting.
While I agree that the pixel-grounding interface introduces new challenges, the methodological advance of the proposed idea beyond existing planner-training studies is still not significant.
I therefore maintain my current score.
We thank the reviewer for the precise, actionable report, and in particular for catching the Table 8 ambiguity — that was our error, and correcting it changed how we describe the RL contribution.
W1/Q1 — Novelty vs. AgentFlow. We accept that planner-only GRPO with frozen downstream modules was proposed there, and the revision now cites and discusses it (Sec. 2, App. I.1). Our claim is not that recipe. The differences that matter: (i) AgentFlow selects among discrete API calls in a short-horizon, largely single-turn setting with terminal rewards, whereas CODA renders the plan into pixel coordinates through a separate 7B grounding model over stateful, long episodes (median 7–9 steps, P90 14–18), and scores the executed action — so the planner must learn to write groundable plans (Fig. 12); (ii) consequently the reward must be dense: under the identical setting an AgentFlow-style terminal objective reaches 18.10 vs 30.17 for ours (Tab. 7b), and 11.21 vs 19.39 on OSWorld; (iii) specialist→generalist consolidation, which yields one planner beating the four-model ensemble (21.04 vs 14.39) with one quarter of the parameters.
W2/Q2 — Table 8. The reviewer is right that it was not readable. Re-checking the manifests, the problem is units, not a confounded control.
#trajand#pairs.The +8.64 gap over the RL-free control decomposes into +5.18 better data from RL (with 17% fewer pairs), +2.45 fresh rollouts, and +1.01 data volume. Note the direction: three of the four steps shrink the corpus while performance rises (24,960 → 5,771 → 4,806 pairs; 9.12 → 12.40 → 17.58). (ii) is a useful sanity check here — it has 4.1× more pairs than (iii) yet scores 3.28 lower, because raw traces include failed rollouts, so the ablation is not measuring data quantity.
We are happy to run the opposite control (top up (iii) to 5,650) if the reviewer prefers; we chose down-sampling because it requires no new rollouts or judge passes, and because (iii) already has marginally more data than (vi).
Revision. (1) Table 8 reports
#trajand#pairsfor every row, with (iv) relabeled as the Stage-1 corpus. (2) Row (vii), the matched-size control, is added. (3) H.2 is rewritten so the +3.46 step is described as fresh data plus consolidation rather than as an isolated RL effect. (4) AgentFlow is cited in Sec. 2 and App. I.1, with Tab. 7b promoted as the terminal-reward baseline. (5) The caption states that (iii) and (vi) are matched to within 2.1%.We thank the reviewer again and would be glad to answer any further questions.