- The novelty of the core method and how it differs from prior work remain unclear.
Environment synthesis, task-driven development, testing and iterative repair, state-supported verification, and process rewards have all been explored in prior research. For example, InfiniteWeb [1], which the paper already cites, includes unified specifications, task-driven testing and iterative repair, and automatically generated task evaluators. CUA-Gym [2] likewise combines the generation of tasks, environment states, and rewards, along with browser-based verification and repair workflows.
Echoverse’s most promising point of differentiation is its use of execution failures by the policy during training to continuously co-adjust the environment, tasks, and verifiers. However, Section 4 and Appendix B mainly describe how the builder, verifier, triage, and fixer work together; they do not yet adequately explain the specific methodological advance this mechanism offers over existing test-and-repair workflows. In particular, reliably distinguishing between insufficient policy capability and environment defects, and deciding what to repair and in which direction to evolve, still rely largely on model judgments and manual QA, with no independent quality measurement.
-
The experiments on environment depth and co-evolution are confounded and do not yet provide sufficient support for mechanism-level attribution.
In the comparison between shallow and deep environments, the authors changed both frontend fidelity and functional support. Appendix D.3 also notes that the training datasets for the two groups were only similar in size and were not matched at the trajectory level. Therefore, the gains on real websites may stem from improvements in visual similarity, functional coverage, task composition, and demonstration quality alike. The current results support the idea that higher-quality synthetic environments can aid training, but they do not make it possible to determine how much of the gain comes from the notion of depth as defined in this paper.
The ECHOSTAY experiment has a similar issue. After the fixes, task feasibility increased from 48% to 78%, and model accuracy rose from 16.2% to 38.5%. It is reasonable that the authors compared the two models under the same test conditions. However, the original environment had severe control defects, and the distribution of feasible tasks and demonstrations available after the fixes changed as well. As a result, this experiment does not distinguish the benefits of rollout-driven co-evolution from those of ordinary software fixes and an increase in effective training data.
A minor issue: A total of 37 tasks cannot yield a value of 48.0%: 17/37 ≈ 45.9%, while 18/37 ≈ 48.6%. Please clarify the actual number of evaluation tasks and how the figure was calculated.
-
The verifier is a cornerstone of the entire framework, yet its reliability has not been quantified.
The paper emphasizes “database grounding,” but the final judgment is still made by a plain-text GPT-4.1 judge (Section 3.1, Appendix G). The paper reports no agreement rate between the judge and human annotators, nor any false-positive or false-negative rates. The prompts in Appendix G are fairly permissive: the write task explicitly accepts “additional modifications,” while the read task accepts answers containing “more information”; counts and IDs only need to be “close.” In RL, these rules could directly become reward loopholes—for example, by listing multiple candidate answers or appending write operations with side effects. The authors acknowledge this in Section 3.1, but do not conduct any reward-hacking analysis. Also, Appendix D.6 mentions that after EchoChat’s scorer was realigned, the proportion of scorable tasks rose from 34% to 99%. This suggests that the scorer may have been seriously misaligned before the fix, further underscoring the need to audit residual scoring errors in other worlds.
There is also a minor issue: the scoring instructions in the read-task prompt include “0 = no required changes present…,” which looks like it was copied from the write-task prompt.
-
The novelty of the RL component is limited, and the main results are based on in-distribution evaluation.
The recipe itself combines GRPO with outcome-based rewards and LLM-judged step-by-step process rewards. Related work cited in the paper, such as PRO-CUA and VAGEN, has already explored step-by-step or process-based feedback. The paper compares only against a “no step-by-step reward” baseline, without evaluating other process-reward approaches, such as learned process reward models.
The validation and training sets are drawn from the same six environments and use the same task-generation pipeline. Although the task IDs and normalized targets do not overlap, this is still an in-distribution evaluation. Most of the improvement in mean@4 occurs within the first 20 updates, after which performance plateaus. The step-by-step rewards are not centered, and their weight of 0.5 is on the same scale as the trajectory reward, so the policy is optimizing GPT-4.1’s preferences to a considerable extent. The authors should clarify whether this introduces a preference for longer responses or other biases—for example, how trajectory length changes during synthetic RL training.
-
The gains in the synthetic environment are substantial, but strong evidence that they transfer to accuracy on real-world tasks is still lacking.
Synthetic RL raises mean@4 from 39.05% to 60.34%, demonstrating that the model learns in the constructed environment. However, across 281 matched Online-Mind2Web tasks, the number of successes increases only from 154 to 159, or from 54.80% to 56.58%. Appendix F.5 reports a 95% confidence interval of −4.6 to 8.1 percentage points. The authors explicitly acknowledge this, and I appreciate their transparency; at this stage, the results are not enough to establish a reliable gain in real-world task accuracy.
The clearest positive changes are a drop in the non-termination rate from 14.9% to 3.6% and a reduction in the average number of actions from 36.2 to 31.5. However, the average number of actions on successful trajectories is essentially unchanged; the overall reduction is mainly due to shorter long-tail failure trajectories. Thus, this evidence more strongly supports improved termination behavior, while any improvement in successful task execution still needs to be verified. Continued training on real websites is also preliminary: the number of successes on a fixed set of 71 questions rises from 16 to 21, but the success rate across all scorable tasks remains roughly 21.5%, and evaluation coverage varies across checkpoints. Given the distributional differences between the training tasks and the public benchmark, I would not interpret the nonsignificant results as evidence that the method is ineffective; however, the paper’s strongest evidence of gains currently remains confined to the authors’ own environment.
Thank you for the careful review, and for recognizing that environment and verifier defects are an important source of unreliable training signal. We appreciate the questions on co-evolution and verification and address each one below.
Co-evolution is supported by three controlled comparisons, spanning SFT and RL and covering eight domains:
Each controlled ablation requires rebuilding environments, recollecting trajectories, and retraining, so we could not run one per world. We observed the same pattern across all worlds during development, and Appendix D.6 documents rollout-driven repairs beyond EchoStay. One EchoForum task set goes from 0 to 36 of 37 tasks solved, and EchoChat's gradable tasks go from 34% to 99%. Without these repairs, the affected tasks would have produced wrong training signal.
Yes, at two levels:
A world ships only when 95% of claims and 95% of tasks pass, with no hard blockers, and in these checks errors were rare.
Extra database writes are allowed deliberately. Many applications log updates even when the agent only browses, so rejecting all extra writes would penalize correct behavior. When co-evolution surfaces false positives from a class of unintended changes, we add explicit constraints to the reference to disallow that class. The verifier itself improves through the same loop.
Data volume is controlled in every ablation, so the gains are not from more data:
The better trajectories are the mechanism, not a confound. A repaired environment makes more complex interactions feasible, so the same budget yields better data. This is exactly what co-evolution is designed to do.
The recipe generalizes to 290 held-out validation tasks that are disjoint from training:
The step-reward ablation isolates the recipe's contribution. Without step rewards, mean@4 at update 20 is 47.19%, versus 59.91% with them, a 12.73 pp gap under identical settings.
Online-Mind2Web is an out-of-distribution test by design. Our tasks are write-heavy workflows in login-gated applications, which live-web benchmarks cannot support, while OM2W is read-only research on public sites. RL also starts from an SFT checkpoint trained on 2,000 live-site tasks, which already captures most of the live-web gain. Despite both factors, RL still improves success (54.80% → 56.58%) and cuts non-termination by 4× (14.9% → 3.6%), with no loss in live-web accuracy.
Every concern raised is answered by controlled results already in the submission:
The only addition needed is the exact SFT counts. We do not believe a weak reject reflects this evidence, and we ask the reviewer to raise their score.