- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2(+1)
Read review
Paper summary
This paper introduces Weblica, a framework for generating scalable and reproducible web environments for training visual web agents. By constructing synthetic interactive websites, Weblica enables large-scale reinforcement learning without relying on live websites. Experiments demonstrate improved performance across multiple web navigation benchmarks.
Critical strengths
S1. Scalability
Weblica provides a scalable approach to generating diverse interactive web environments. The framework could be useful to the community if made publicy available.
Critical weaknesses
W1. Limited Scientific Novelty
While Weblica demonstrates scalability, its scientific novelty relative to prior work, such as VeriEnv, InfiniteWeb, and AutoWebWorld, appears limited. The contributions primarily lie in scaling up environment generation and RL training rather than introducing fundamentally new methodological insights. As a result, the work appears to be primarily an engineering contribution, with limited scientific novelty beyond existing approaches.
W2. Insufficient Ablation of Capability-Driven Synthesis
The paper lacks a critical ablation study to validate its primary methodological distinction: capability-driven synthesis. In particular, it does not compare environment generation with and without capability conditioning under matched environment counts and RL training budgets. Consequently, it remains unclear whether capability-driven synthesis itself contributes to the reported performance gains or whether these gains are primarily driven by the scale of the generated training data.
Questions for authors
- What are the key scientific contributions of Weblica beyond scaling, particularly compared to VeriEnv, InfiniteWeb, and AutoWebWorld?
- Can the authors provide a controlled ablation to demonstrate the effectiveness of capability-driven synthesis?
- How does the reliability of the LLM-based judge compare to programmatic verification, and how does this choice affect RL training?
5 comments
Response to Reviewer q5m8 (2/3) – Capability-driven synthesis and LLM judge
Capability-driven synthesis vs. scale (W2, Q2)
We would like to point out that capability conditioning is what allows synthesis to scale in the first place. It is a common observation that generative models produce highly similar outputs when prompted repeatedly with the same request [1, 2]. As a result, the quantity of synthetic data scales easily, but its diversity does not [3]. The common remedy is to condition each request on attributes sampled in advance [1, 4]. This yields more diverse data and better downstream models than a plain prompt [1], and sampling the attributes from a taxonomy of topics or capabilities further increases diversity and coverage [2, 5]. We observed the same when generating websites, as repeated requests for the same capability converged to similar domains and styles (App. B.3). Category and style seeds address this for content and appearance. The capability seed, drawn from an inventory extracted from agent trajectories including failed ones (App. B.2), decides which interactions a site exercises, so we obtain thousands of sites that exercise different interactions instead of variants of the same site. For us, scale and capability conditioning are therefore intertwined.
Regarding scale, please also note that simply scaling up is not enough. To demonstrate this, we cache real websites from the InSTA (v3) task pool and train on the same number of tasks (10K) with the same recipe. Synthesized environments still perform better overall (Sec. 5.4).
| pass@1, 8B, same recipe and 10K RL tasks | OM2W | DeepShop | WTB | Avg. |
|---|---|---|---|---|
| Base (Qwen3-VL-8B) | 28.6 | 24.1 | 21.8 | 24.8 |
| RL on cached real websites | 35.3 | 35.8 | 30.2 | 33.8 |
| RL on synthesized websites (Weblica-8B) | 39.2 | 34.2 | 33.5 | 35.6 |
LLM judge vs. programmatic verification (Q3)
Programmatic verification needs a checker for each task, written into or alongside the site's code, and it works only when success shows up in a stored state or an exact answer string. Prior work restricts its tasks accordingly. VeriEnv admits only "tasks with a single, well-defined answer", InfiniteWeb checks flags it inserts into localStorage, and AutoWebWorld does not score the content of answers.
An LLM judge, in contrast, scores any task from its natural-language criteria and the screenshots of the episode. It needs no per-task code and also accepts free-text answers, e.g., a Weblica-val task asks what an error code describes and which equipment it affects (App. E.2, Fig. 11). This is why we use it as the RL reward (Sec. 4). As reported in App. F.5, our judge agrees with human labels on 88% of 100 training tasks and reaches 91.0% accuracy on 300 Online-Mind2Web trajectories, with open-weight judges on par. Its remaining errors act as reward noise, yet the trained policy improves its base model by 10.8 points on average on live-web benchmarks scored by their own official judges.
Please also note that LLM judges are an established choice for web agents. Live-web benchmarks such as Online-Mind2Web, DeepShop and WebTailBench score agents with LLM judges rather than programmatic checks, given the diversity of real-web tasks, and LLM judges are increasingly used as rewards for training web and GUI agents, e.g., in DigiRL [6] and PAE [7]. All our live-web results are measured with the benchmarks' official judges.
[1] Yu et al., Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias, NeurIPS 2023.
[2] Davidson et al., Reasoning-Driven Synthetic Data Generation and Evaluation, TMLR 2026.
[3] Ge et al., Scaling Synthetic Data Creation with 1,000,000,000 Personas, arXiv 2024.
[4] Samvelyan et al., Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts, NeurIPS 2024.
[5] Li et al., Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models, arXiv 2024.
[6] Bai et al., DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning, NeurIPS 2024.
[7] Zhou et al., Proposer-Agent-Evaluator (PAE): Autonomous Skill Discovery for Foundation Model Internet Agents, ICML 2025.
Response to Reviewer q5m8 (3/3) – Public availability
Public availability (S1)
The paper gives full details for reproducing the pipeline, as summarized in our Reproducibility Statement. Appendix B covers capability extraction, the seed pools, the coding-agent harness and its tools, task and criteria generation, all prompts, per-site statistics and generation costs. Appendix C details how the cache is constructed. Section 4 and Appendix F describe the training recipe, hyperparameters, compute, action space and judge prompt. The environments are static HTML/CSS/JS, and open-weight judges perform on par with GPT-4o (App. F.5), so training on them does not require a proprietary model. Generation is a one-time step that other capable coding agents could also perform, and the rest of the pipeline builds on open-weight models (Qwen3-VL), open-source tools (Playwright and veRL) and a public task dataset (InSTA (v3)).
We hope these answers clarify our contributions, and we are glad to discuss any remaining questions.
Response to Reviewer q5m8 (1/3) – Scientific contributions
Thank you for the constructive review and for recognizing that Weblica offers a scalable approach to generating diverse interactive web environments, which could be useful to the community. We address your concerns below.
Scientific contributions (W1, Q1)
We respectfully disagree that the contribution is mainly engineering. Weblica differs from the works you mention both conceptually and in focus. Conceptually, two of the three do not train with online RL, and all three train at a much smaller scale. AutoWebWorld uses offline single-step RL on 29 websites, and VeriEnv uses rejection-sampling fine-tuning on 97 websites and leaves RL to future work. InfiniteWeb does use online GRPO, but on 600 tasks and with rewards limited to what its generated code checks. In contrast, Weblica trains online multi-turn RL on thousands of environments and tasks. Importantly, Weblica rewards each task with an LLM judge that reads natural-language criteria and the screenshots of the episode, so no task needs a hand-written checker, and Weblica can reward a much broader set of open-ended tasks, including free-text answers (see Q3).
Weblica also achieves strong results. In a system-level comparison on Online-Mind2Web, the only benchmark all four works report, Weblica-8B reaches 39.2 pass@1 on all 300 tasks, 10.6 points above its base model. The other three report 28.7 (InfiniteWeb), 24.6 (VeriEnv) and 14.0 (AutoWebWorld). Weblica-8B also improves substantially on the other two live-web benchmarks, DeepShop and WebTailBench (Table 1).
Please also note that our focus goes beyond environment synthesis. Besides synthesizing websites, we build a second reproducible alternative to live-web training, HTTP-level caches of real websites, and compare the two under the same recipe (Sec. 5.4). Both improve substantially over the base model, and each comes with its own trade-off. Caching keeps real content but supports only the recorded interactions and the tasks the recording agent could solve. Synthesis can target the interactions agents need, and it performs better overall (see the table in our answer to W2 below). We believe this comparison of environment choices is useful to the community beyond our specific pipeline, and Reviewer HPbX also found it "worth studying".
In summary, Weblica makes three scientific contributions beyond scaling.