- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Real world web tasks require planning over long sequences of steps. Agents need to infer high level plans, navigate intermediate states, track progress, and perform visual actions. A single agent struggles to handle all these abilities at once. Existing modular multi agent systems separate planning from execution, but they usually lack joint training. To address this problem, this paper proposes JHiMA-WebRL, which explicitly separates a task decomposer from an executor. It first fine tunes both roles using subtask supervision from a teacher model, then jointly optimizes them through hierarchical reinforcement learning. The method explores how joint learning of planning and execution helps with long horizon web navigation.
Critical strengths
- The hierarchical design fits the problem well. Separating task decomposition from action execution allows each role to focus on a specific function and provides a reasonable structural bias for long horizon web tasks.
- The joint optimization method is novel. It extends GRPO through hierarchical rollout grouping and role specific advantage estimation, allowing the planner and executor to train together.
- The construction of subtask supervision has practical value. A strong teacher model generates structured subtasks from multimodal GUI trajectories, providing clear training signals for supervised fine tuning of both roles.
Critical weaknesses
- The baseline scores differ from those in the original paper. Table 1 reports scores of 23.9 and 24.0 for ScaleCUA-7B and ScaleCUA-32B, respectively, while Table 5 in the original ScaleCUA paper reports 37.7 and 44.2. Since the original ScaleCUA scores are close to or higher than those of the proposed method, the authors need to further explain the reasons for this performance gap.
- The evaluation covers only WebArena. This does not show generalization across datasets or effectiveness on different benchmarks. Please include more benchmarks, such as WebVoyager or Mind2Web2, to demonstrate the generality and generalization ability of the method.
- The title emphasizes long horizon tasks, but the experiments do not directly show that the advantage grows as tasks become longer. The interaction limit is 15 steps for both training and evaluation. The paper does not group tasks by the number of steps they require or compare performance, error accumulation, and recovery ability under larger step budgets. This setting differs greatly from current long horizon settings. For example, OSWorld 2.0 uses more than 100 steps.
- It remains unclear whether the framework includes replanning. If the planner works only once at the start, the system lacks a feedback loop from execution to planning. If the planner updates the plan at every step, please clearly emphasize this in the paper, as replanning is an important part of hierarchical planning.
Questions for authors
See above.
General response
We sincerely thank all reviewers for their constructive feedback. We would like to highlight three important points.
1. Broad recognition of methodological novelty. All three reviewers explicitly acknowledged the novelty of our joint hierarchical RL framework. Our central contribution is a new methodology for simultaneously optimizing the decomposer and actor through hierarchical rollouts and role-specific advantage estimation. To our knowledge, prior work has not jointly optimized these components in this manner.
2. Overall performance versus component ablations. We would like to clarify that all configurations in Table 2, including multi-agent SFT and actor-only RL, are our own design variants, not methods from existing papers. The modest 2–4 percentage-point gains reflect the contributions of individual components within our novel framework, rather than incremental improvements over prior research. In Table 1, JHiMA achieves 37.2% versus 24.0% for ScaleCUA-32B RL under the same WebArena training tasks and unified evaluation protocol, despite using fewer parameters.
3. Substantial new experiments during rebuttal. In response to the reviewers' valuable suggestions, we significantly expanded our empirical evaluation:
We believe the independently recognized methodological novelty, competitive overall performance, and substantially expanded experimental evidence together demonstrate a meaningful research contribution. We will clarify the difference between baselines and proposed component ablations in revision.
We respectfully hope the reviewers and AC will consider these contributions and the additional rebuttal evidence holistically in the final assessment.