- Rating
- 1(−1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper asks when the repair of a failed WebAgent episode should count as a persistent capability rather than a one-off fix. The authors propose BoundaryLoop. An auxiliary CodeAgent turns failure evidence into executable workflows that are replayed in a real browser. Repaired workflows from several instances are then abstracted into a parameterized family procedure, which enters a Registry only after passing transfer, regression, no-match and browser-boundary tests. On WebArena Shopping Admin, static repair passes the evaluator in 59/84 runs but carries a stale parameter in all of them, while promoted family workflows complete 84/84 runs cleanly. Stored workflows also pass 33/33 later runs against 0/33 for the base agent. I do not work on web agents, so my comments focus on the framing and the experimental evidence.
Critical strengths
- The motivation is clear and the question is well chosen. Separating "the evaluator accepts the answer" from "the procedure is actually bound to the new task" is useful, and parameter pollution is a concrete failure mode. The formulation is also clean. Instance repair versus family workflow, the acceptance contract in Eq. 1, and the split between failure frontier and capability boundary make the claims testable. The 59/84 versus 84/84 contrast in Table 1 illustrates the point well.
- Ablation shows that source-only or held-out-only rules would admit a candidate that fails at the browser boundary. The experiments support the claims as stated, and the authors are upfront about cohort denominators and limitations in the appendix.
Critical weaknesses
- Strong baselines from this area are missing. The main comparisons are the base agent, a parameter-closure control and the authors' own static repair.
- The results sit almost entirely at the two ends, and the scope is narrow. The base agent gets 2/84, task-only repair gets 0/24, the family workflow gets 84/84, and stored workflows get 33/33 versus 0/33. Numbers like these say little about how the method behaves in between. A base agent that almost never succeeds suggests a weak setup (the 84-run study uses a Qwen3.6-35B-A3B model with an 8 step budget), and a workflow that always succeeds is close to a script with parameter slots. Nearly everything is also on one site with about ten task families, and the second-site probe only checks public URL contracts with no private oracle. I would like to see a stronger base agent with a fair budget, harder or more varied families where the method does not saturate, graded results such as success versus the number of repair rounds, and at least one more site or benchmark.
Questions for authors
- If the base agent is stronger, does the gap in Table 1 and Table 4 shrink, and by how much?
Overall I find the problem framing worthwhile and the paper well written, but the experiments are not yet convincing. I lean toward weak reject for now. If the authors add strong baselines under matched budgets and results in settings that do not saturate, I am happy to raise my score.
I will decrease my score as a clear reject.