acceptodds
Under review as a conference paper at ICLR 2027

BoundaryLoop: From Failure Evidence to Persistent Web Capabilities

Abstract

An episodic WebAgent repair becomes a reusable capability when its executable behavior survives new task parameters and later runs. We study how failure evidence supports that transition. BoundaryLoop turns failed WebAgent episodes into browser-tested instance workflows, compares repairs to extract parameterized family procedures, and admits those procedures to executable memory after transfer and boundary validation. The WebAgent supplies task, action, DOM, and browser-state evidence; an auxiliary CodeAgent synthesizes and tests the correction. In a frozen WebArena Shopping Admin comparison with 84 runs per condition, the Base WebAgent records 2/84 execution successes and 0/84 clean completions. Static repair passes the evaluator in 59/84 runs but has parameter pollution in 84/84, whereas promoted family workflows complete 84/84 runs without recorded pollution or drift. Compact aligned evidence matches the 6/24 accepted-repair yield of full raw evidence with 12.7% fewer input tokens. A frozen family procedure transfers to 3/3 original instances excluded from abstraction and 5/5 post-freeze generated instances. In a separate later-runtime study, stored workflows pass 33/33 runs versus 0/33 for the matched BaseAgent. These results locate capability acquisition in cross-instance invariance and validated reuse.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Web agents, failure-conditioned repair, executable workflows, cross-instance generalization, persistent capabilities, agent self-improvement

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_BWTJOpenReview ↗
Rating
1(−1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper asks when the repair of a failed WebAgent episode should count as a persistent capability rather than a one-off fix. The authors propose BoundaryLoop. An auxiliary CodeAgent turns failure evidence into executable workflows that are replayed in a real browser. Repaired workflows from several instances are then abstracted into a parameterized family procedure, which enters a Registry only after passing transfer, regression, no-match and browser-boundary tests. On WebArena Shopping Admin, static repair passes the evaluator in 59/84 runs but carries a stale parameter in all of them, while promoted family workflows complete 84/84 runs cleanly. Stored workflows also pass 33/33 later runs against 0/33 for the base agent. I do not work on web agents, so my comments focus on the framing and the experimental evidence.

Critical strengths
  1. The motivation is clear and the question is well chosen. Separating "the evaluator accepts the answer" from "the procedure is actually bound to the new task" is useful, and parameter pollution is a concrete failure mode. The formulation is also clean. Instance repair versus family workflow, the acceptance contract in Eq. 1, and the split between failure frontier and capability boundary make the claims testable. The 59/84 versus 84/84 contrast in Table 1 illustrates the point well.
  2. Ablation shows that source-only or held-out-only rules would admit a candidate that fails at the browser boundary. The experiments support the claims as stated, and the authors are upfront about cohort denominators and limitations in the appendix.
Critical weaknesses
  1. Strong baselines from this area are missing. The main comparisons are the base agent, a parameter-closure control and the authors' own static repair.
  2. The results sit almost entirely at the two ends, and the scope is narrow. The base agent gets 2/84, task-only repair gets 0/24, the family workflow gets 84/84, and stored workflows get 33/33 versus 0/33. Numbers like these say little about how the method behaves in between. A base agent that almost never succeeds suggests a weak setup (the 84-run study uses a Qwen3.6-35B-A3B model with an 8 step budget), and a workflow that always succeeds is close to a script with parameter slots. Nearly everything is also on one site with about ten task families, and the second-site probe only checks public URL contracts with no private oracle. I would like to see a stronger base agent with a fair budget, harder or more varied families where the method does not saturate, graded results such as success versus the number of repair rounds, and at least one more site or benchmark.
Questions for authors
  1. If the base agent is stronger, does the gap in Table 1 and Table 4 shrink, and by how much?

Overall I find the problem framing worthwhile and the paper well written, but the experiments are not yet convincing. I lean toward weak reject for now. If the authors add strong baselines under matched budgets and results in settings that do not saturate, I am happy to raise my score.

1 comment

I will decrease my score as a clear reject.

Reviewer_5m8hOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper asks when repairing a failed WebAgent episode amounts to acquiring a reusable capability. BoundaryLoop records a failed episode as cross-view evidence (parameters, action trace, DOM-grounded controls, browser state), has an OpenCode-based CodeAgent synthesize an executable instance workflow that is replayed in a real browser and revised from counterexamples, abstracts same-family workflows into a parameterized family workflow, and admits it to a versioned registry only after promotion gates (source replay, transfer, regression, non-target/no-match, browser boundary). On WebArena Shopping Admin it reports that static repairs pass the evaluator in 59/84 runs while carrying stale parameters in 84/84, whereas promoted family workflows complete 84/84 cleanly (Table 1); failure-conditioned evidence repairs 41/168 replays vs. 0/24 for task-only input (Table 2); a family-368 chain transfers 3/3 and 5/5 (Table 3); stored workflows pass 33/33 later runs vs. 0/33 for the base agent (Table 4). No code, tasks, evaluators or workflow artifacts are provided.

Critical strengths

The central distinction is sharp and the evidence for it is real. Table 1's "59/84 evaluator passes with 84/84 parameter pollution" exposes a failure mode that success-rate reporting hides, and the evidence-view study (Table 2) honestly reports that trajectory-only views match or beat aligned ones. All table arithmetic reconciles (Tables 2, 4, 8, 9; A.3 inventory 89+58+35 = 182). This is why I think the idea deserves a corrected paper; it is not enough to carry the current one.

Critical weaknesses
  1. The headline results use workflows that the paper's own appendix attributes to engineering outside the proposed pipeline. Table 1 covers families 288/366/367; Table 4 covers 250/270/367. Sec. 6.4 (p.8) says the 366/367 repairs were rejected at the gates and "neither enters this lineage's persistent memory"; Table 7 (p.18) lists 366 as "Strict lineage rejected; later mixed operational version admitted" and 250 as "Operational v1 reuse; separate strict v2 rejected"; App. A.1 (p.17) calls these "mixed engineering lineages … not additional strict CodeAgent-provenance promotions"; App. C (p.20): "The abstraction step is deterministic engineering rather than direct CodeAgent output." What a "mixed operational version" is or how it was produced is never stated. As written, 84/84 and 33/33 show that a parameterized script of unspecified origin runs reliably — not that the proposed loop produces such scripts. The only end-to-end CodeAgent chain is family 368 (Table 3, two source tasks). Compounding this, the promotion gates — the first stated contribution — are enumerated as 5, 6, 7 and 5 gates in Fig. 1 (p.2), Fig. 2 (p.4), Sec. 4.4 (p.5) and Fig. 3 (p.9), with no thresholds or construction procedure for non-target/no-match tasks, so the admission criterion cannot be reproduced.

  2. The base-agent condition anchoring the "capability gap" is not comparable. Table 5 (p.16) gives the Table 1 base cohort "Custom GUI executor … 8 steps," whereas later cohorts use AgentLab with 20–30 steps; the base agent averages 662/84 = 7.9 actions per episode (budget nearly always exhausted) and logs 743 online calls > 84 × 8 = 672 ("online call" undefined). Tasks are author-generated with private evaluators (App. A: "28 generated tasks"), although the abstract frames a "WebArena Shopping Admin comparison"; no standard WebArena success rate is reported, while App. A.3 lists 89/182 historical strict successes on the original tasks without agent or budget. The undefined "Parameter closure" arm is nearly identical to Base (743 vs 743 calls, 49/84 vs 49/84 drift, 83/84 vs 83/84 evidence), and "Execution" is undefined (Static repair: Execution 0/84 but Evaluator 59/84). The 0/84 clean base result therefore cannot be separated from the budget and executor choice.

Questions for authors

Clarification (answers would change my assessment):

  1. For each family in Tables 1 and 4, what is the "mixed operational" / "later operational" workflow, how was it produced, and how does it differ from the rejected strict CodeAgent lineage? Is the "deterministic abstraction step" a general algorithm (pseudocode?) or per-family code?
  2. Please give one authoritative gate list with thresholds and ordering, and the construction of non-target/no-match tasks.
  3. Why 8 steps on a custom executor for the Table 1 base? What does the base achieve on the 28 generated tasks under the 20–30-step AgentLab setting, and which agent/budget produced A.3's 89/182?
  4. Define Execution / Clean / Evaluator / Evidence / Pollution / Drift in terms of Eq. 1, and "Parameter closure." Were Base and Parameter-closure independent runs?
  5. Were the three "original transfer" tasks (Table 3) excluded from the demonstrations given to the CodeAgent during repair? Are Table 3's five generated tasks the same five on which Base already scores 5/5 in Table 8?

Suggestions: release the audit script, tasks, evaluators, prompts and at least one complete family-workflow artifact; quantify offline synthesis cost; make the abstract's evidence-compression claim match Table 2 (trajectory-only 7/24 at 18.3k tokens beats aligned-minimal 6/24 at 33.1k); remove the internal archive path in App. C and the creator-tool metadata in embedded figure PDFs.

Reviewer_93pjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents BoundaryLoop, a framework for turning failed WebAgent episodes into reusable executable workflows. It first collects task, action, DOM, and browser-state evidence. A CodeAgent then synthesizes and refines repairs through browser replay. Successful instance workflows are abstracted into parameterized family workflows. These workflows are admitted to persistent memory after validation. The experiments report improved task completion and reduced online model calls for stored workflows, including transfer to new task instances.

Critical strengths
  1. The paper addresses a practical and important problem: a repair that succeeds on one instance may still contain stale parameters and fail on similar tasks.
  2. It distinguishes evaluator acceptance from task-grounded execution, including whether the requested parameter was actually applied in the browser.
  3. Browser replay and checks of visible control state provide stronger validation than relying solely on generated code or model self-evaluation.
Critical weaknesses
  1. The strongest results involve workflows with mixed engineering provenance, making it difficult to attribute the gains to the full BoundaryLoop pipeline.
  2. The paper lacks a matched-budget baseline that directly generates parameterized workflows with a browser-enabled CodeAgent.
  3. The paper does not clearly articulate its motivation or explain where its novelty lies relative to existing memory and task-management approaches.
Questions for authors

Can the authors report full lifecycle costs, including synthesis and validation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.