acceptodds
Under review as a conference paper at ICLR 2027

Live-Browser-Agent: Real-World Centric Fundamental Long-Horizon Browser Agents

Abstract

Training a browser agent for deployment requires assigning credit from a sparse terminal outcome to the few critical steps that determined it. Existing pipelines supervise on successful demonstrations and optimize online with group-relative objectives that assign a uniform trajectory-level advantage to every step. Methods resolving credit below the trajectory level either intervene on the environment or match steps via recurring observations, neither of which is reliable in live browsing sessions. We propose a two-stage training framework to address this. First, Recovery- and UI-specialized Curriculum Supervised Fine-Tuning (RUIC-SFT) treats recovery data as state-conditioned supervision, introducing it only after the conditioning state becomes separable; this significantly improves erroneous-step recovery while reducing redundant actions per task. Second, Divergence-Attributed Online Group Relative Policy Optimization (DAO-GRPO) derives unbiased step-level advantages from prefix-conditioned leave-one-out baselines within each rollout group, requiring no environment resets or per-step model calls. Building on these stages, we develop the Live-Browser-Agent family (4B, 9B, and 27B). Our 27B model attains the highest average Pass@1 among open-weight models on three online browser benchmarks (82.3%), on par with Kimi-k3 and Qwen3.8-Max despite using roughly 90× fewer parameters, and improves its Qwen3.5-27B initialization on three general agentic benchmarks, indicating that browser-centric training does not erode general tool use.

Primary area:
foundation or frontier models, including LLMs
Keywords:
browser agent, online rl, benchmark

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_JxRXOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2(+1)
Soundness
3(+1)
Read review
Paper summary

This paper presents a step-by-step recipe for training a better computer-use / web agent (CUA). It has four parts: a training dataset (live-browser-set, built with a three-part collection recipe DgD_g, DuD_u, DrD_r), a new evaluation benchmark (BrowserBench), and two fine-tuning stages (SFT with general, UI, and recovery portions of data and training schedule, then RL with a proposed GRPO variant). Experiments show gains over baselines on the tested benchmarks, plus studies of the SFT data mixture / schedule and ablations of the GRPO variant.

Critical strengths
  1. Interesting content and large coverage: The paper covers a training set, a benchmark, and two fine-tuning stages, with detailed theoratical description of finetining and demostration of gains over baselines on the tested benchmarks. This is an interesting combined recipe for training CUA agents.
  2. Useful analysis: the study of SFT data mixture / schedule and the ablation of the GRPO variant show practical information for anyone training similar agents.
Critical weaknesses

Some of these suggestions could go in the appendix if the main body becomes too long. I am willing to raise my score if these can be substantially addressed.

Not very easy to understand:

  • Sections 5 and 6 are confusing with all the symbols. It seems some symbols are not properly introduced before use.
  • The introduction is organized in a strange way (BrowserBench is mentioned at the beginning, before the problem formulation, etc.). It probably needs a rewrite.

Ablation results do not seem to report statistical significance

  • Improvement over the baseline is shown on the tested benchmarks, but due to the highly dynamic nature of computer use, it seems hard to find the results convincing without repeated trials.

The main results (Table 2) could benefit from more explanation / model coverage / benchmark coverage

  • No specialized CUA models are included in this table. The explanation for this omission is that frontier models usually outperform specialized CUA models, but this claim is not supported by evidence or citations.
  • Frontier model coverage is also not ideal (Anthropic, Google, etc.). If resources are a constraint, the authors could cite public results. This is especially needed since a new benchmark (BrowserBench) is introduced as well.
  • Some standard CUA benchmarks are not tested: WebArena? Mobile benchmarks if possible? OSWorld if possible?

Would be nice to have:

  • The data collection process needs more coverage in the main text.
  • No code / data release at all, despite being promised in the appendix.

Missing some benchmark / citation coverage

  • Related work on web / CUA / GUI agents? SFT training of other web / CUA / GUI agents? Offline / online RL for web agents?
    • The 4 major components of the paper each need their own discussion of prior work, so readers can better understand their novelty. Perhaps the related work section should be reorganized around them.
    • For example, prior work related to the live-browser-set's data collection recipe (D_g, D_u, D_r) is not discussed at all.
Questions for authors
  1. How many runs are behind Table 2 and the ablations? Could you report mean and standard deviation over repeated trials?
  2. What supports the claim that frontier models outperform specialized CUA models? Could you add a few specialized CUA models and more frontier models to Table 2 (public numbers are fine if compute is limited)?
  3. Based on your setup, is it possible / sensible to add results on standard CUA benchmarks such as WebArena, and confirm whether code and data will be released as the appendix states?
2 comments
Reviewer_JxRXOpenReview ↗

Thank you

Thank you to the authors for addressing my concerns. I am willing to raise my score because most comments were substantially addressed, but I do want to mention 2 lingering concerns.

  1. I understand that due to time constraint, some of the results cannot be collected during the rebuttal phase, but I still believe the paper would benefit from a more rigorous treatment of results, including more comparison with other prior and concurrent CUA-specialized models / agents and e.g. elevate WebArena results to the main chart.
  • For example, when reporting the new WebArena results, you already cited Zhang et al. 2026. That paper is itself a CUA-agent paper with WebArena result, but it is omitted from the results you reported.
  1. The introduction would still benefit from a human touch, including more citations to support claims made there and a better flow of the motivation and the story you want to tell in general. Despite these, I still believe the paper is valuable for the research community and would raise my score to a 3.
AuthorsOpenReview ↗

Thank You for Your Support and Feedback

Dear Reviewer JxRX,

Thank you very much for your insightful guidance throughout the review process and for raising your score to support our work! We also sincerely appreciate your two constructive follow-up suggestions, and we will incorporate both into our final manuscript.

Best regards, The Authors

Reviewer_34bkOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper trains long-horizon browser agents for live websites in two stages:

RUIC-SFT is a data curriculum for supervised fine-tuning. It adds examples of recovering from mistakes only late in training, once a gradient-alignment probe suggests the model can tell when it has gone off track. DAO-GRPO is an online RL objective that gives credit to individual steps. It compares each rollout with sibling rollouts that took the same actions up to that step, so it needs no environment resets and no per-step judge calls. The paper also introduces two resources:

Live-Browser-Set: 2,832 annotated trajectories and about 7,400 grouped rollouts. BrowserBench: 350 live-web tasks in Chinese and English, averaging 37.9 steps. Its 27B model reaches 82.3% average Pass@1 on WebVoyager, Online-Mind2Web and BrowserBench. That's on par with open models about 90× larger.

Critical strengths

Strong results on an important, hard problem. Long tasks on live websites are where browser agents fail today. The 27B model reaches 82.3% average Pass@1 across three live benchmarks, on par with Kimi-k3 and Qwen3.8-Max. It also improves on its base model on three general agentic benchmarks (67.2 to 73.8). Useful resources and a design that fits live pages. BrowserBench fills a real gap: long, bilingual tasks (37.9 steps on average) across 254 live sites. Matching rollouts by their actions instead of by page observations suits live pages, where over 84% of observation-matched groups contain a single rollout. The released rollouts with logged contexts also allow offline re-analysis.

Critical weaknesses

The headline gains may partly reflect training on the evaluation judge.

Same judge for reward and test. BrowserBench is scored by Qwen3.7-Plus, the same judge that produces the RL reward and the progress signal (App. F.3, G). The paper doesn't say which judge scores WebVoyager and Online-Mind2Web. RL against the grader can raise scores without raising true success. The human audit doesn't rule this out. The 96.4% agreement is on 500 trajectories pooled across models. The paper says disagreements weren't concentrated on any model family, but it gives no per-model numbers. In particular, there are none for the trained model before vs. after RL, which is where judge exploitation would show. The benchmark shaped training. The RL task pool was built around the skills that an early BrowserBench run showed were weak (App. F.4). No run-to-run variance is reported for any live-web result. The paper doesn't show that its two methods are why the model is strong.

DAO-GRPO vs. GRPO is one small comparison. It's tested only at 9B on BrowserBench: 42.9% vs. 39.4%. With 350 tasks, one standard error is about 2.6 points. So the total gain, each component's +0.7 to +1.1, and the gaps to GiGPO (39.9%) and HGPO (40.4%) are all within noise. Step-level credit may cover few steps. It only applies where at least two sibling rollouts took the same actions so far. With 6 independent rollouts of up to 50 steps, those groups likely shrink to a single rollout after the first few steps. The paper reports no statistics on group size by step. "Unbiased" isn't what's new. Any leave-one-out baseline that doesn't depend on the current action is unbiased, and Prop. 1 is the standard argument. The claimed benefit is lower variance, which isn't measured. RUIC-SFT mostly shortens trajectories. The curriculum beats the fixed data mixture by 1.4 points in success (38.0 vs. 36.6), which is within noise. The clear effect is fewer steps (22.4 vs. 29.3). The paper defines a measure Δ (Eq. 3) that would show the model recovers only after real mistakes, but never reports it. In short, this is a strong agent with a useful benchmark and dataset. But the evaluation leans on the training judge, and the paper doesn't show that RUIC-SFT and DAO-GRPO specifically explain the gains.

Questions for authors

Clarifications

Which judge scores WebVoyager and Online-Mind2Web? Can you report human-judged success on a subset of tasks for both the RUIC-SFT and final models? How many runs is each result based on, and what's the variance? This matters most for DAO-GRPO vs. GRPO, GiGPO and HGPO in Table 5. At each step depth, what fraction of steps have at least one sibling rollout with the same actions so far? Suggestions

Compare DAO-GRPO with GRPO at 27B and on the other two benchmarks.

3 comments
AuthorsOpenReview ↗

Gentle Follow-up on Our New Experiments and Revisions

Dear Reviewer 34bk,

We hope you are doing well, and we sincerely thank you again for your rigorous and insightful review.

As the discussion period progresses, we wanted to gently check in to see if you have had a chance to look over our responses (Parts A–D) and the revised manuscript. Following your suggestions, we conducted every requested experiment to directly address your concerns:

  • Independent Judge & Per-Checkpoint Human Audit (Part A): Re-evaluating all three benchmarks with an independent cross-family judge (Kimi-k3.1-Pro) and conducting a blinded 100-task human audit before vs. after RL (RUIC-SFT vs. DAO-GRPO) confirmed that the RL gains hold consistently (+5.0 pp human-judged gain at 9B and +11.0 pp at 27B, with 95%–97% judge-human agreement on the RL checkpoints alone).
  • 3-Run Variance, Reward-Matched Baselines & 27B DAO-GRPO vs. GRPO (Parts B & C): We added 3-run Mean ± Std evaluations, trained GiGPO and HGPO with matched RTG + Potential terms (43.0 ± 0.4% for DAO-GRPO vs. 41.1 ± 0.5% for GiGPO and 41.3 ± 0.8% for HGPO), and trained a 27B Online GRPO baseline, which DAO-GRPO-27B outperforms across all three benchmarks (+2.8 pp on WebVoyager, +5.6 pp on Online-Mind2Web, and +6.9 pp on BrowserBench).
  • Step-Depth Cohort Statistics, Gradient Variance & Probe Δ\Delta (Parts C & D): We reported exact K=6K=6 prefix-sharing rates by step depth (71.6% at steps 2–5, 21.4% at steps 6–10), empirical policy-gradient variance (-24.4% reduction over Group LOO), and the Eq. (3) off-path vs. on-path recovery separation Δ\Delta across all SFT variants (+48.0 pp for RUIC-SFT vs. +24.6 pp for fixed mixture).

We would be very grateful to hear if these new results address your concerns, or if there is anything else we can clarify during the discussion period. Thank you very much for your time and consideration!

Best regards, The Authors

Thanks for the thorough response. The independent judge and the blinded human audit resolve my main concern. Human-judged gains match the judge's (+5 points at 9B, +11 at 27B), and judge-human agreement stays at 95–97% after RL. The external benchmark results, the 27B GRPO comparison, and the variance measurement also address W3, W5, and W7. I'm raising my score to 3. A few points remain on the method claims:

The ± values measure evaluation noise, not training variance. Fair point that run-to-run noise on a fixed task set is smaller than the task-sampling SE. But the three passes re-evaluate one checkpoint, so DAO-GRPO's lead of about 2 points over the matched baselines still rests on one training run per method. Please say so in the paper. Prefix sharing covers about 19% of steps, mostly early ones. The other 81% use the Group LOO baseline. Also, the 24.4% variance reduction is measured only on the shared steps. I'd describe DAO-GRPO as a refinement of Group LOO at early branching steps. RUIC-SFT mainly prevents over-correction. Most of the doubling in the Eq. (3) separation comes from fewer false corrections (24.6% to 5.9%). True recovery rises only 4.7 points (49.2% to 53.9%). That's still a useful result, but the paper should frame it that way.

In short, the headline results now hold up. But the method gains are about 2 points at 9B from single training runs, and the paper should present them at that size.

AuthorsOpenReview ↗

Thank You for Your Support and Constructive Suggestions

Dear Reviewer 34bk, thank you very much for reviewing our responses and for raising your score to support our work! We completely agree with your three presentation points and will explicitly state in the final manuscript that the 9B ablation ± values reflect evaluation variance on a single training run, frame DAO-GRPO as a variance-reducing refinement of Group LOO at early branching steps, and highlight that RUIC-SFT primarily suppresses false on-path over-corrections alongside its +4.7 pp true recovery gain.

Reviewer_3BLBOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents Live-Browser-Agent, an end-to-end training framework for browser-use agents that combines recovery-oriented curriculum SFT, prefix-conditioned online RL, and bilingual long-horizon evaluation. Although the individual algorithmic and data contributions are not highly novel, the work systematically brings together the components needed for a training-ready browser-use pipeline, with substantial attention to data verification, error recovery, context management, and reliable execution. Experiments report strong browser performance across multiple model sizes.

Critical strengths

1.The paper addresses practical challenges in real-web agent training, particularly error recovery and sparse feedback under non-resettable, dynamic environments. 2.The recovery dataset includes localized errors and execution-verified corrections, providing useful supervision beyond successful demonstrations. The bilingual benchmark also broadens evaluation coverage. 3.The empirical results are promising, particularly for the 27B model on external browser benchmarks. Using a shared harness across evaluated models improves comparability.

Critical weaknesses
  1. The paper does not evaluate whether advantages identify annotated decisive actions. It also omits cohort-size statistics under the actual training setting of (K=6), leaving unclear how often prefix conditioning remains useful at deeper steps.
  2. SFT on only 2,003 general-task trajectories plus 829 UI and recovery trajectories yields substantial gains, reaching 15–25 percentage points on several benchmarks, with large improvements even for the weaker 4B model. While promising, gains of this magnitude warrant clearer documentation of the training data’s provenance. Could the authors explain how the tasks and templates were constructed and provide an overlap audit against the evaluation benchmarks, including task instructions, templates, and websites, to establish training–evaluation separation?
  3. GiGPO and HGPO do not receive the same RTG and potential components as DAO-GRPO. Prefix LOO itself adds only 1.1 percentage points in Table 5, without repeated runs or uncertainty estimates to establish robustness.
  4. Recovery supervision relies on human-written reflections and corrections, yet the dataset contains only 417 recovery trajectories. How many annotators were involved, what was the annotation cost, and how were the diversity and realism of injected errors assessed? Can you provide evidence that the pipeline scales beyond the reported collection cycle with manageable human effort? Is it possible to consider the proposed data generation/annotation pipeline scalible?
  5. In Eq. (7), the advantage numerator changes across steps through the potential, prefix cohort, and shrinkage weight, while the denominator remains the standard deviation of initial returns (G_{j,1}). How do you ensure that this mismatch does not produce excessively large advantages for some steps or disproportionately weight tasks with small initial-return variance?
  6. Shared-prefix steps remain in the loss and may receive opposing advantages depending on subsequent outcomes. Although these updates can cancel under identical contexts and an unclipped objective, matching action prefixes does not guarantee identical observations or responses, and clipping may disrupt cancellation. Can you quantify the resulting gradient variance and compare against masking shared-prefix steps to demonstrate that the method concentrates learning on consequential decisions?
Questions for authors

See weaknesses.

2 comments
AuthorsOpenReview ↗

Gentle Follow-up on Our Response and Additional Analyses

Dear Reviewer 3BLB,

We hope you are having a good week, and we thank you again for your positive and thoughtful review of our work.

We wanted to gently check in to see if our responses (Parts A–C) and the revised manuscript have addressed your questions. In brief, we have added all of the requested empirical analyses and clarifications:

  • Decisive-Action Identification & K=6K=6 Cohort Statistics (Part A): We evaluated advantage alignment on the 256 annotated error-recovery instances (DAO-GRPO reaches 90.0% sign accuracy and 78.5% Top-3 recall vs. 69.5% and 41.4% for Group LOO) and reported step-depth cohort statistics under K=6K=6.
  • Data Provenance, Overlap Audit & Reward-Matched 3-Run Ablations (Part B): We detailed how the 2,832 trajectories provide 21,183 step-level training instances, confirmed 0% instruction/template/website overlap with BrowserBench, and reported 3-run Mean ± Std results comparing DAO-GRPO (43.0 ± 0.4%) against reward-matched GiGPO (41.1 ± 0.5%) and HGPO (41.3 ± 0.8%).
  • Recovery Annotation Scalability, Advantage Bounds & Shared-Prefix Masking (Part C): We documented the annotation cost (~15 min/trajectory across 6 annotators with automated triage and verification), explained how bounded returns and ϵ=0.05\epsilon=0.05 prevent small-denominator spikes, and quantified policy-gradient variance (-24.4% for DAO-GRPO, and -25.8% when masking unbranched shared steps).

Please let us know if you have any further questions or if there is anything else we can clarify during the discussion period. Thank you once again for your valuable feedback!

Best regards, The Authors

I thank the authors for the detailed response and for the substantial new empirical work in Parts A–C and the revised manuscript. I have reviewed the added analyses, and my overall assessment remains positive. I am keeping my rating.

Reviewer_GY5eOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper shows how to train web browser AI agents for long, multi-step tasks on real websites. Training these agents is hard because live sites don't let you undo an action or reset to try something else from the exact same point, and the agent only learns whether it succeeded at the end of all the steps. To address this, the authors combine two key ideas: first, they teach the model normal successful paths before introducing mistakes and recovery steps (RUIC-SFT). Second, they use RL to compare attempts that share the same starting steps (DAO-GRPO) to identify which action caused the problem. The authors tested this approach across three model sizes on standard benchmarks and a new bilingual set called BrowserBench, showing good improvements over other models.

Critical strengths
  • I find the design of DAO-GRPO very well suited for live web environments. Since live site sessions cannot be rewound and page states almost never repeat, comparing attempts that share the same action sequence is a smart way to assign credit. I also appreciate how the method cleanly handles single attempts by falling back to a group-level baseline empirically. This is a simple yet effective idea that could apply beyond web browsing tasks.
  • The results for the RL phase are strong. I noticed consistent performance gains across all three model sizes and benchmarks.
  • Building a long-horizon benchmark is another strength of the paper, especially in a bilingual setting.
Critical weaknesses
  • My main concern is with the evaluation. The paper uses the same Qwen3.7-Plus judge to provide the RL reward and to score the final results on BrowserBench, without checking whether the gains hold up under an independent evaluator. The reported 96.4% agreement with human annotators helps, but that number is pooled over all models, right? The results are also reported as Pass@1 with only one rollout per task, without repeated runs or uncertainty estimates. On top of that, some claims go further than the numbers show, like "matches or surpasses proprietary frontier models...", but understandably GPT-5.5 is ahead on all benchmarks.

  • I am also not sure the gains actually come from the proposed credit assignment. The prefix-conditioned baseline only contributes when at least two of the six attempts share the same action prefix, but the paper never reports how often this happens during training. If I am not mistaken, the examples in Figure 8 use 16 to 20 attempts instead of the actual training group size of six. Also, it seems the baseline methods are run without the same reward terms in Table 5, so the comparison does not isolate the contribution of action matching. As it stands, it is hard to tell how much of the gain comes from the core idea of prefix-based credit assignment.

Questions for authors
  1. Which evaluator was used for WebVoyager and Online-Mind2Web? It would also help to see the human agreement for the RL-trained model on its own, and the threshold that turns a judge score into a pass.
  2. How large are the shared-action groups at different steps with the attempts, and for what share of steps does the prefix-conditioned baseline actually contribute? Which sampling temperature did you use for the rollouts?
  3. Would it be possible to repeat the runs in Tables 4 and 5 and report the mean and standard deviation, and to run GiGPO/HGPO with the same reward terms as row 3 of Table 5?
  4. Is Φ\Phi only a baseline, or also part of the reward, as I noticed in Appendix G suggested? And how is it ensured that the judge’s mark at step tt does not use information from later steps?
  5. Are the tuning tasks kept separate from BrowserBench? An anonymous link to the benchmark, data and judge prompts would also help.
  • Minor points and suggestions: 'Recov.' in Tables 4 and 5 is not defined, and it is not clear how 'Steps' is averaged. Fig. 4 seems to be missing. Also, if I read it correctly, the WebArena average step count is 10 in Fig. 1, but different in Table 7. I also suggest reporting active parameters next to the claim and mentioning the BrowserBench gap to Kimi-k3 in the abstract.
1 comment

Dear Reviewer GY5e,

We hope this message finds you well, and we sincerely appreciate your supportive review and constructive questions.

We wanted to gently follow up to see if you have had a chance to review our responses (Parts A–C) and the revised manuscript. To briefly recap, we have addressed all of your questions with new experiments and revisions:

  • Independent Evaluator & RL-Only Human Audit (Part A): We re-scored all three benchmarks with an independent cross-family evaluator (Kimi-k3.1-Pro), added a blinded human audit on RUIC-SFT vs. DAO-GRPO alone (confirming 95%–97% human agreement and +5.0 pp / +11.0 pp human-judged RL gains), clarified the binary pass threshold, and tempered our headline claims in the Abstract.
  • Prefix Cohort Statistics (K=6K=6) & Reward-Matched 3-Run Ablations (Part B): We reported the exact step-depth cohort sizes under K=6K=6 training (T=1.0T=1.0), added 3-run Mean ± Std evaluations for Tables 5–6, and ran GiGPO and HGPO with the exact same RTG + Potential terms as Row 3.
  • Baseline Clarification, Task Separation & Presentation Fixes (Part C): We unified the state-dependent baseline description in Appendix G, confirmed strict tuning/test separation, and fixed all table definitions (Recov., Steps), figure numbering, WebArena step counts, and parameter comparisons.

Please let us know if our responses and revisions satisfactorily resolve your concerns, or if we can provide any further details. Thank you again for your time and helpful feedback!

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.