acceptodds
Under review as a conference paper at ICLR 2027

From Pages to Tools: Generating and Integrating Page-Coupled Tools for Web Agents

Abstract

Structured tools offer web agents an alternative to repeated low-level browser interactions, but exposing tools alone does not guarantee effective use. The design problem spans both tool construction and agent integration: which application capabilities should be exposed, when they should be available, and how their descriptions and execution feedback should enter the agent loop. We introduce PACT (PAge-Coupled Tools), a framework for generating and integrating page-coupled tools. PACT derives reusable, single-function capabilities from page and application evidence, independently verifies their semantics, scope, execution, and agent-facing selection, and dynamically registers tools according to the live page state. Tool execution is bounded from higher-level agent reasoning, leaving tool selection, composition, and subsequent decisions to the calling agent. On 812 WebArena tasks, PACT improves success across all 16 agent-model configurations and reduces token usage in most of them. Tool-set and integration studies further show that tool quality and information exposure materially affect both performance and resource use. Additional evaluations across model sizes, web-agent fine-tuned models, WebArena Verified, and our mock websites further support the generality of PACT while revealing model- and configuration-dependent tradeoffs.

Primary area:
foundation or frontier models, including LLMs
Keywords:
web agents, tool generation, page-coupled tools, agent-tool integration, WebMCP

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_e8HgOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper proposes PACT, a pipeline that generates page-coupled tools for web agents following the WebMCP draft. A coding agent collects task-independent page states by BFS from a site’s homepage, discovers page-native capabilities, specifies each as a single-function tool with a page-state registration condition, implements it as a JavaScript callback or annotated HTML form, and a separate agent verifies semantics, scope, execution and selection; execution terminates at any full-page navigation so composition stays with the calling agent. Integration is treated as a design variable (full vs. lightweight page × full definitions vs. catalog). On 812 WebArena tasks, four agents × four models all improve in success with the default integration, and 14/16 configurations use fewer tokens. Additional studies cover a naive and an agent-optimised tool set, integration and screenshot ablations, Qwen-3.5 model sizes, fine-tuned web agents, WebArena Verified, and four self-built mock sites.

Critical strengths
  • Clear motivation with design choices that follow from it. The Introduction states four concrete limitations of WALT-style learned tools (L1 trajectory-bound coverage, L2 bundled navigation/agentic steps, L3 state-independent exposure, L4 unexamined integration), and each PACT component answers one: task-free evidence-based generation (L1), the single-function boundary at page navigation (L2), the registration condition Ct(s)C_t(s) (L3), and the observation × exposure design space (L4).
  • The improvement from PACT is consistent and robust. Experimental results show that the performance gains from PACT upon the base agents do not decrease in any of the 16 agent × model configurations on 812 WebArena tasks and rises by up to +23.4 pp, with tokens down in 14/16.
  • The integration study is informative. In section 3.3, the authors turn tool–agent integration into a 2×2 design space (full vs. lightweight page × full tool definitions vs. catalog) and Section 4.1.3 measures it on ColorBrowserAgent with two models over the same 812 tasks. The finding that the same tool set yields different accuracy and cost depending on what the agent sees is interesting.
Critical weaknesses
  1. Positioning and novelty are overstated and two directly relevant works are uncited. SkillWeaver (Zheng et al., 2025) synthesises skills as APIs from self-exploration with a test-and-debug stage; WebXSkill (Wang et al., 2026) already exposes skills by page via a URL-pattern graph (the substance of L3) and studies deployment modes with model-dependent preferences (the substance of L4). The genuinely new elements are the in-page WebMCP execution substrate and trajectory-free generation; the single-function boundary and state-conditioned exposure are refinements of existing ideas.

  2. No quantitative comparison with any existing tool- or skill-extraction method. Every quantitative comparison is Original vs. PACT or PACT-variant vs. PACT-variant; the only reference to an existing method is the qualitative Table 8.

  3. A single benchmark, evaluated under a modified grader. Only one established benchmark, WebArena, is used; there are no results on WebVoyager, Online-Mind2Web, or live sites. The two supplementary evaluations do not change this: WebArena Verified (Table 4) is the same 812 tasks with corrected evaluators, run for one agent–model pair, and the mock Chinese sites (A.12, Table 20) are generated by Kimi-K3 with 90 author-written tasks, again for one pair. Section 4 further replaces the grader’s LLM with Qwen3.7-max, so the reported numbers are not comparable with published WebArena results, making it impossible to verify the effectiveness of PACT relative to other methods.

Questions for authors

Please see weaknesses.

4 comments
AuthorsOpenReview ↗

Author response for Reviewer e8Hg - 1

Thank you for your careful review and for your positive assessment of the motivation, evaluation, and integration study. We appreciate your suggestions on related work, comparisons with existing methods, and evaluation beyond WebArena.

“SkillWeaver … synthesises skills as APIs from self-exploration with a test-and-debug stage; WebXSkill … exposes skills by page via a URL-pattern graph.”

We thank the reviewer for highlighting these closely related approaches. We agree that reusable skill generation, verification, and context-dependent availability have been explored in prior work. We have expanded Section 2 to discuss SkillWeaver and WebXSkill and clarify our distinctions.

Unlike approaches that construct reusable agent skills, PACT focuses on automatically generating WebMCP-compatible capabilities grounded in existing website functionality. Specifically, PACT derives tools from page states, DOM structures, and application logic, registers them according to live page conditions rather than URL patterns alone, and bounds their execution within page-native operations. These tools preserve existing website workflows and human-facing interactions, while leaving navigation, reasoning, and tool composition to the calling agent.

Our contribution therefore lies in the integrated generation, verification, and agent integration of page-coupled tools, rather than claiming novelty for the individual mechanisms.

“No quantitative comparison with any existing tool- or skill-extraction method.”

We thank the reviewer for this suggestion. WALT provides both its own tool designs and a dedicated agent architecture. To ensure a more controlled comparison, we integrated WALT tools into CUGA and evaluated both WALT and PACT tools using the same agent (CUGA), reasoning model (Qwen-3.7-Plus), and 812 WebArena tasks.

Comparison with WALT. Results are reported in Appendix A.15 (Table 23).

ConditionSR (%)Tokens (all)Tokens (succ.)Output tokens (all)Output tokens (succ.)
CUGA: Original agent28.8360.7k179.9k5.0k3.1k
WALT agent + WALT tools37.2146.5k123.9k4.1k3.4k
CUGA + WALT tools40.4271.5k217.8k2.8k1.9k
CUGA + PACT tools44.3206.5k199.1k3.8k2.8k

Under the same CUGA agent, PACT outperforms WALT tools by 3.9 percentage points while reducing mean token usage by 23.9%. We additionally report WALT with its original agent architecture for reference, although differences in agent design prevent a direct attribution of performance differences to the tools alone. These results are summarized in Section 4.1.2.

“There are no results on WebVoyager, Online-Mind2Web, or live sites.”

We additionally evaluated PACT on live websites using Online-Mind2Web (Appendix A.16). The results are summarized below.

Online-Mind2Web. AgentOccam with Qwen-3.7-Plus is evaluated on 210 tasks after excluding 90 task pairs affected by external errors.

ConditionSR (%)Tokens (all)Tokens (succ.)Output tokens (all)Output tokens (succ.)Steps (all)Steps (succ.)
Original agent29.5153.7k114.0k8.5k4.9k13.09.0
Full Page + Full PACT39.0 (+9.5)177.3k127.1k8.7k5.3k14.09.8

PACT improves task success by 9.5 percentage points, while increasing mean token usage by 15.4% and steps by 7.7%. Unlike the locally hosted WebArena environments, live websites can introduce additional interaction overhead, and available tools may not cover every required operation. Tool selection, invocation, and subsequent browser interactions may therefore increase resource consumption. The higher success rate may also reflect additional progress on tasks that would otherwise terminate unsuccessfully, although we have not isolated these effects.

These findings demonstrate that PACT can improve task success on diverse live websites, while highlighting a trade-off between task completion and interaction cost. We do not claim exhaustive page-state coverage, and improving exploration efficiency and tool coverage for large dynamic websites remains an important direction for future work.

“The two supplementary evaluations do not change this: WebArena Verified … is the same 812 tasks with corrected evaluators.”

The revised Section 4.1.6 distinguishes WebArena Verified from the live-web evaluation, while Section 4 introduces the separate mock-site extension. WebArena Verified provides a check on revised task descriptions and evaluation within the WebArena task family. The mock-site extension examines generated websites, while the new Online-Mind2Web experiment evaluates public live websites.

AuthorsOpenReview ↗

Author response for Reviewer e8Hg - 2

“Section 4 further replaces the grader's LLM with Qwen3.7-max.”

Thank you for raising this concern. We would like to clarify our motivation for using a different grading model. Our primary goal is to evaluate the effectiveness of PACT under a consistent evaluation protocol across different agents and tool configurations.

The WebArena codebase specifies gpt-4-1106-preview as its LLM grader. However, OpenAI announced on April 22, 2026, that this model would be shut down on October 23, 2026, after our submission deadline but before the rebuttal period. Therefore, relying on this model would prevent us from maintaining the same grading configuration throughout the evaluation and additional rebuttal experiments. We consequently adopted Qwen3.7-Max for all LLM-based grading checks, ensuring that all agent and tool configurations are evaluated consistently.

Evaluator agreement. To assess the potential impact of this change, we conducted an additional evaluator-agreement analysis using GPT-4o and GPT-5.6-Sol, which are the replacements for gpt-4-1106-preview, according to OpenAI. The results are reported in Appendix A.14 (Table 22).

ConditionGPT-4o AgreementGPT-4o κGPT-5.6-Sol AgreementGPT-5.6-Sol κ
Main experiment (16 agent-model pairs)98.7%0.80099.0%0.844
AgentOccam (Qwen-3.7-Plus) + Verified Naive98.5%0.76499.0%0.840
WALT agent + WALT tools97.7%0.61897.9%0.636
CUGA + WALT tools98.5%0.75399.0%0.830

For the main experiments, GPT-4o and GPT-5.6-Sol achieve 98.7% and 99.0% agreement with Qwen3.7-Max, respectively, with Cohen's κ of 0.800 and 0.844. The additional evaluations also show high agreement across different tool configurations.

These results support the consistency of our evaluation under different grading models. Although changing the grader limits direct numerical comparison with previously published WebArena results, our conclusions are based on matched comparisons under the same evaluation protocol. We further address cross-method comparability by evaluating WALT and PACT tools using the same CUGA agent, reasoning model, tasks, and grader (Appendix A.15).

Thank you for the candid response and the additional experiments. The controlled WALT comparison under the same CUGA agent and the Online-Mind2Web evaluation on live sites address my main concerns about missing baselines and single-benchmark evaluation, and I am raising my score accordingly. Some concerns remain:

  1. Novelty. As the authors acknowledge, the contribution lies in the integrated pipeline rather than in its individual mechanisms. The main remaining distinction from prior skill-based approaches is registration based on live page conditions rather than URL patterns, which is a modest increment.
  2. WALT comparison. The +3.9 pp gain comes from a single agent–model pair and a single run, without a significance test. The token advantage holds only relative to CUGA + WALT tools; WALT with its own agent uses fewer tokens (146.5k vs. 206.5k). It is also unclear whether the integration settings were tuned equally for both tool sets, and tool-generation costs are not compared. A quantitative comparison with SkillWeaver is still missing.
  3. Efficiency on live sites. On Online-Mind2Web, PACT increases tokens by 15.4% and steps by 7.7%. This differs from the efficiency gains reported on WebArena and suggests that the cost benefits may not carry over to live websites.

I encourage the authors to include these analyses and caveats in the revision.

AuthorsOpenReview ↗

Response to Reviewer e8Hg

Thank you for the updated assessment and the higher score. We would like to share a few additional details:

“Novelty ... the contribution lies in the integrated pipeline ... Page-conditioned registration is a modest increment.”

We would like to clarify that PACT differs from prior skill-based approaches in its objective, generation process, and execution semantics, beyond tool registration.

While SkillWeaver and WebXSkill acquire reusable agent skills from exploration experiences or execution trajectories, PACT automatically transforms existing website functionality into WebMCP-compatible tools. It derives tools from page states, DOM structures, and application logic without requiring task-execution trajectories or prior agent-side experience, making generation independent of specific agents.

Moreover, PACT exposes bounded, page-native operations rather than multi-step agent behaviors, leaving navigation, reasoning, and tool composition to the calling agent. Each tool is independently verified and dynamically registered based on live page conditions.

PACT's primary contribution is thus a trajectory-independent framework for exposing website functionality through WebMCP, while separating website operations from agent-level decision-making.

“Significance test.”

We conducted a paired analysis of 812 WebArena tasks using CUGA with Qwen-3.7-Plus. PACT achieved 44.3% success versus 40.4% for WALT, solving 99 tasks missed by WALT, while WALT solved 67 missed by PACT. The paired improvement was 3.94 percentage points (95% paired-bootstrap CI: 0.86–7.02; two-sided exact McNemar p = 0.0159), indicating a statistically significant improvement.

“The token advantage holds only relative to CUGA + WALT tools; WALT with its own agent uses fewer tokens (146.5k vs. 206.5k).”

We thank the reviewer for this observation. WALT with its own agent uses fewer tokens (146.5k vs. 206.5k), but also achieves a lower success rate (37.2% vs. 44.3%).

PACT aims to generate agent-independent tools that generalize across Web agents, rather than optimize for a specific agent. We therefore compare PACT and WALT under the same CUGA agent to isolate tool effectiveness. In this controlled setting, PACT improves the success rate from 40.4% to 44.3% while reducing token consumption by 23.9%.

We additionally evaluated PACT tools with the WALT agent. PACT achieves 40.3% success, compared with 37.2% using WALT tools. Mean recorded agent token usage decreases from 146.5k to 125.2k per task. Table 23 reports the corresponding resource statistics.

ConditionSR (%)Tokens (all)Tokens (succ.)Output tokens (all)Output tokens (succ.)Cache hit % (all)Cache hit % (succ.)
WALT agent + WALT tools37.2 (+8.4 pp)146.5k123.9k4.1k3.4k45.948.0
WALT agent + PACT tools40.3 (+11.5 pp)125.2k95.6k4.0k3.0k40.638.9

“Equally tuned for both tool sets, and tool-generation costs.”

Neither integration was tuned on WebArena. Both followed CUGA configurations with Qwen-3.7-Plus. PACT tools were registered based on live page states, while WALT's released workflows and tool catalog were adapted to CUGA's interface. Both operated within CUGA's browser context, with CUGA handling planning and action selection.

Appendix A.17 reports 1,381.0 million tokens for generating and verifying the 184 PACT tools, including page-state collection and revision but excluding the task-independence audit. Since we used WALT's released workflows, its original construction cost is unavailable. Table 23 compares execution costs only, excluding tool construction.

“A quantitative comparison with SkillWeaver is still missing.”

While SkillWeaver's implementation is publicly available, its pre-generated WebArena skills do not appear to be released. Unlike WALT, whose tools can be directly adopted, evaluating SkillWeaver requires reproducing its skill generation.

This process is ongoing but has been delayed by slow API responses and the large number of required calls. We plan to include the comparison in the revised manuscript once completed.

“On Online-Mind2Web, higher token usage is observed.”

We thank the reviewer for highlighting this trade-off. On Online-Mind2Web, PACT improves the success rate from 29.5% to 39.0% (+9.5 percentage points), while increasing token consumption by 15.4% and execution steps by 7.7%.

Although PACT does not reduce absolute execution costs on this benchmark, the relative improvement in success rate (+32.2%) substantially exceeds the increase in token consumption (+15.4%), indicating a more favorable success-to-token trade-off. These results demonstrate that PACT's efficiency benefits vary across benchmarks, while its improvements in task completion extend to live websites. We have clarified this trade-off in the revised paper.

Reviewer_HH5oOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes PACT, a page-coupled tool, binding named function of web operations to a specific webpage. It uses skills to guide a coding agent to explore website states. A key observation is that these page-coupled tools are made available according to the current page state. These tools are provided to coding agent according to the current state, and the coding agent reasons on the tools and current page status. The ablations study on what is the right amount of information needs to be provided to the agent is also studied in the paper. Overall, according to the benchmark on the WebArena across 812 tasks, PACT improves task success rate in all configurations and reduces average token usage.

Critical strengths

S1. The evaluation is solid in that main eval covers all webarena tasks and 16 agent model combinations, showing the improvement in terms of success rate and mean token suage rate.

Critical weaknesses

W1. The motivation for choosing this problem is grounded in four limitations of existing approaches. However, for L1, PACT also requires exploration. What is special about PACT in this phase? Does it reduce token and latency overhead? How does PACT ensure coverage? For L2, based on the WebMCP definition, a tool is a named function of web operations. Once these interfaces are provided, the agent can combine the tools and reason at the tool level instead of using low-level actions. It seems that PACT is nothing to do with this limitation. For L3, the paper argues that providing all tool interfaces upfront is wasteful. However, given a user task on a website, the landing page should also expose all the interfaces upfront so that the agent can reason about them. PACT only reduces the interfaces exposed within a specific website. For L4, although I appreciate the ablation experiments, the results show that even with PACT, providing the full page performs best. Overall, these limitations are either not convincing (at least to me) or difficult to connect to PACT.

W2. Regarding the methodology itself, PACT ties tools to a webpage, but this may reduce flexibility. For example, if a user wants to perform a task using a tool that is not tied to the current webpage, how can this be done? Does the agent simply fail? If there is no good fallback when the user's task does not match the current webpage, it is difficult to justify that PACT's benefits outweigh the flexibility it loses.

W3. The baseline comparison only includes AgentOccam, but there are multiple other methods, including WALT and ActionEngine.

W4. ActionEngine is also not mentioned in the related work

W5. Another concern is coverage. Blindly asking an agent to explore and generate named interfaces could miss some operations needed to complete a task. What is the coverage? Which tasks are covered by the named functions generated by PACT, and which tasks actually benefit from them?

Questions for authors

Q1. PACT also needs to explore the website. what PACTs add in addition to the existing methods? how do the authors check whether the generated tools cover the operations needed for the tasks?

Q2. I am not sure why tools should only be available on the current page. If a user needs a tool from another page, what happens? Does this introduce extra navigation, and how much token or latency overhead does it have?

Q3. Why is there no experimental comparison with WALT and other methods? The current results show that adding PACT helps existing agents, but how does it compare with another method that also generates tools?

5 comments
AuthorsOpenReview ↗

Author response for Reviewer HH5o - 1

Thank you for your detailed comments and for recognizing the breadth of the main evaluation. Your questions about page context, navigation, and coverage are helpful for clarifying what PACT provides and how an agent uses it.

“For L1, PACT also requires exploration. What is special about PACT in this phase?”

PACT uses exploration to collect page states, then derives capabilities from the corresponding DOM structure, application logic, and visible behavior. This allows generation to reuse the website's existing workflows and interaction design. The resulting tools include both the operation and the conditions under which it applies. The revised L1 in the introduction and the generation procedure in Section 3.2 explain this distinction. Section 4 now points to the stage-level construction costs in Appendix A.17 (Table 25).

“For L2 … the agent can combine the tools and reason at the tool level instead of using low-level actions.”

We agree that tool composition belongs to the agent. PACT makes this division explicit in the generated interface: tools implement individual page-native capabilities, while the calling agent selects and combines them with its predefined actions. A tool can coordinate several low-level operations, such as filling and submitting a form, while subsequent planning remains in the agent's interaction loop. The revised introduction presents this as a design choice serving the shared browsing environment, and Section 3.2.2 defines the tool execution boundary.

“For L3 … the landing page should also expose all the interfaces upfront so that the agent can reason about them.”

Our design starts from a web environment that people and agents can both browse (following WebMCP protocol). The current page provides the shared context, and its registered tools expose capabilities available in that context. The website's navigation, information architecture, and interaction guidance remain available to help the agent reach other capabilities, just as they support human browsing. We have clarified this motivation in the introduction. Section 3.3 explains how tool availability follows the page state and now explicitly describes navigation to other pages.

“The results show that even with PACT, providing the full page performs best.”

The full-page results fit this design: page observations provide context for planning and checking outcomes, while tools provide structured operations and returns. The Original agent already receives its standard page observation and retains its predefined actions; Full Page + Full PACT adds the registered tools to that interface. Section 4 now makes this baseline explicit, and Table 3 uses “Original agent (Full Page)” in the integration comparison. The Lightweight Page and PACT Catalog conditions examine the consequences of changing how this information is supplied.

“If a user wants to perform a task using a tool that is not tied to the current webpage, how can this be done? Does the agent simply fail?”

The agent can navigate using its predefined browser actions, observe the destination, and invoke the tools available there. For example, an agent can open a product from a listing using a normal click and then call the product page's add-to-cart tool. Appendix A.4 illustrates this combination of tools and predefined actions. Verification also tests individual capabilities through requests grounded in the current page, exercising the collaborative-browsing case in which a user delegates a specific operation to the agent. Sections 3.3 and 3.2.2 now make the cross-page behavior and this verification use case explicit, respectively.

AuthorsOpenReview ↗

Author response for Reviewer HH5o - 2

“The baseline comparison only includes AgentOccam, but there are multiple other methods, including WALT and ActionEngine.”

We thank the reviewer for this suggestion. WALT provides both its own tool designs and a dedicated agent architecture. To ensure a more controlled comparison, we integrated WALT tools into CUGA and evaluated both WALT and PACT tools using the same agent (CUGA), reasoning model (Qwen-3.7-Plus), and 812 WebArena tasks.

Comparison with WALT. Results are reported in Appendix A.15 (Table 23).

ConditionSR (%)Tokens (all)Tokens (succ.)Output tokens (all)Output tokens (succ.)
CUGA: Original agent28.8360.7k179.9k5.0k3.1k
WALT agent + WALT tools37.2146.5k123.9k4.1k3.4k
CUGA + WALT tools40.4271.5k217.8k2.8k1.9k
CUGA + PACT tools44.3206.5k199.1k3.8k2.8k

Under the same CUGA agent, PACT outperforms WALT tools by 3.9 percentage points while reducing mean token usage by 23.9%. We additionally report WALT with its original agent architecture for reference, although differences in agent design prevent a direct attribution of performance differences to the tools alone. These results are summarized in Section 4.1.2.

“ActionEngine is also not mentioned in the related work.”

Thank you for pointing us to ActionEngine. Its application-state memory and synthesis of executable task programs are relevant to our goal of reusing website functionality. PACT exposes page-native capabilities within an existing agent's observation–action loop, where the agent continues to choose and compose operations. Section 2 now includes ActionEngine in the discussion of reusable web skills and tools.

“Which tasks are covered by the named functions generated by PACT, and which tasks actually benefit from them?”

A task can combine PACT tools with predefined actions and benefit from tools for only part of its execution. Our task-paired analysis examines those benefits: for AgentOccam with Qwen-3.7-Plus, 136 baseline failures become successes and 45 baseline successes become failures. The analysis groups these changes by action efficiency, action selection, parameter handling, information access, state handling, and prior knowledge. Section 4.1.1 reports these task outcomes in Table 1 and refers to Appendix A.8.2 for the category definitions and counting procedure. Appendix A.3 describes the capabilities exposed by the tool set.

AuthorsOpenReview ↗

Follow-up on Rebuttal and Remaining Concerns

Dear Reviewer HH5o,

Thank you for your time and consideration of our submission. As the discussion period is coming to an end, we would like to kindly follow up on our rebuttal and ask whether our responses have adequately addressed your concerns.

If you have any remaining questions or concerns, we would be happy to provide further clarification before the discussion period closes.

Thank you again for your valuable feedback.

Best regards,

The Authors

Reviewer_u3kuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces PACT, a framework for generating and integrating page-coupled tools for web agents using WebMCP. Instead of learning tools from agent trajectories, PACT explores website states and generates tools from page and application evidence. Each tool is designed to perform a single, bounded function and is checked by an independent verification agent for semantics, registration scope, execution, and selection (Sec. 3.2, Fig. 1).

PACT also dynamically registers tools based on the current page state and explores different ways of exposing page observations and tool descriptions to agents (Sec. 3.3, Fig. 2).

The authors evaluate PACT on 812 WebArena tasks across four agent frameworks and four LLM backends. PACT improves success rates in all 16 configurations and reduces token usage in 14 of them (Fig. 3). Additional experiments study tool quality, integration strategies, model sizes, fine-tuned agents, and WebArena Verified.

Critical strengths
  1. A clear and practical tool design.

I like the idea of keeping tools tied to the current page state and limiting each tool to a specific function.

Compared with WALT, which can bundle multiple page interactions into a single tool, PACT stops execution at full-document navigation boundaries and leaves subsequent decisions to the agent (Appendix A.5). I think this is a reasonable design, as it gives the agent more opportunities to observe the environment and adjust its actions.

The independent verification process is also useful. Checking tool semantics, scope, execution, and selection provides a systematic way to catch potential problems before deployment.

  1. Broad evaluation and useful empirical findings.

The experimental evaluation is fairly extensive. The authors test 16 agent-model combinations on 812 WebArena tasks, and some improvements are substantial. For example, CUGA with Gemini-3.7-Flash improves from 31.3% to 54.7%, while AgentOccam with Doubao-Seed-2.1-Turbo improves from 41.4% to 60.7% (Tables 10 and 12).

I also appreciate that the authors include results where PACT is less effective. The model-size, screenshot, and fine-tuning experiments provide a more balanced picture of when these tools help and when they introduce overhead.

Critical weaknesses
  1. The contribution is largely a composition of existing design ideas, and the experiments do not isolate which of them is responsible for the gains. PACT’s main ingredients already exist: page-state-dependent registration is a WebMCP mechanism (the paper says so in Sec. 1), generating validated reusable tools from website functionality is WALT’s pipeline, and the generation itself is carried out by a coding agent following procedural prompts (App. A.1). The paper positions itself against WALT through L1–L4, but these distinctions are thinner than they appear. L3 is addressed by WebMCP itself rather than by PACT. For L1, both WALT and PACT rely on an agent exploring the site; the difference is what gets recorded (execution traces vs. page states and source), and PACT’s coverage is equally determined by which states its BFS reaches (Sec. 3.2.1). I do not think a new learning algorithm is required for this kind of work. But if the contribution is a set of design choices, those choices need to be isolated empirically, and they are not. There is no ablation of dynamic vs. static registration, of bounded tools vs. tools that span full-page navigations, or of generation with vs. without the independent verification step. The Naive tool set (Table 2) changes all of these at once (see Weakness 2). The paper also reports no statistics on verification (how many candidates were rejected, how many revision rounds, what each of the four checks caught), so even the contribution of verification is invisible. This is consequential because the claimed novelty rests on these design choices, and the current experiments show only that the overall framework works, not that its specific design choices are why.
  2. The baseline comparison and the headline claim are weaker than the paper states.

(a) The Naive baseline is confounded. Naive and PACT differ in tool count (126 vs. 184), functional mix (read-only 76 vs. 44, interaction 17 vs. 76; App. Tables 14 and 5), declarative share (0 vs. 21), and possibly the generating model: PACT was generated by GPT-5.5 (Sec. 4), while App. A.8.1 does not state what generated the Naive set. The paper does offer evidence that tool quality matters (Naive executes more tools, 6.4 vs. 4.2, yet gains only 1.0 pp, and information access is its largest regression category in Table 16), but this is consistent with several explanations. A coverage-matched baseline generated by the same model is needed to attribute the gap to the PACT generation procedure. (b) WALT, the paper’s main point of reference (L1–L4), is compared only qualitatively (App. A.5, Table 8). Since WALT is published, running its tools inside the same agents on the same tasks appears feasible, and without it the central claim—that page-coupled, bounded, verified tools are better than WALT-style tools—remains untested. (c) “Improves success across all 16 configurations” (abstract, conclusion) is not supported at the stated strength by the paper’s own Table 13: WebOperator/Qwen-3.7-Plus gains +0.4 pp with a 95% CI of [−2.4, 3.2] and p = .863. ColorBrowserAgent/GPT-5.6-Luna gains +1.0 pp with no test reported, while tokens rise 8% and steps go from 12.4 to 15.5 (Table 11). Significance is reported for only 4 of the 16 configurations. The same paired test should be applied to the other small gains (A3-Qwen3.5-9B +0.5 pp in Table 19, which the paper itself calls descriptive; Naive +1.0 pp in Table 2). This is consequential because the universality of the improvement is the paper’s headline result, and because the comparison that would most directly support the design argument (vs. WALT) is missing.

Questions for authors
  1. Novelty and ablations: Could the authors provide more direct ablations of dynamic registration and bounded execution? I think this would help clarify which parts of PACT are actually responsible for the improvements.

  2. Baseline comparison: How much of the improvement over the Naive tool set comes from having more tools and broader functional coverage? Would it be possible to compare against WALT or a coverage-matched baseline?

  3. Scalability: How does the BFS page-state collection process scale to large dynamic websites, such as SPAs with infinite scrolling or many possible UI states?

  4. Fine-tuned agents: Why does PACT provide such a small additional gain on A3-Qwen3.5-9B while increasing token usage? Is this mainly due to extra tool-selection overhead, or do the tools interfere with the agent's existing action policy?

  5. Maintenance: How much does the tool-generation and verification pipeline cost in practice? If a website changes its DOM structure or application logic, how much work is needed to update the tools?

4 comments
AuthorsOpenReview ↗

Author response for Reviewer u3ku - 1

Thank you for your thoughtful review. We appreciate your positive assessment of the bounded tool design, independent verification, and broad evaluation. Your suggestions have helped us strengthen both the empirical comparisons and the explanation of the design.

“The contribution is largely a composition of existing design ideas, and the experiments do not isolate which of them is responsible for the gains.”

We agree that WebMCP provides the underlying mechanisms for page-state-dependent tool registration, and we do not claim these mechanisms as our contribution. Rather, WebMCP defines how developers can expose structured website capabilities to agents, but does not provide an automated method for discovering, implementing, and verifying such tools on existing websites.

PACT addresses this gap by automating the construction of WebMCP-compatible, page-coupled tools from existing website evidence. Specifically, it discovers capabilities from page states, DOM structures, and application logic; generates single-function tools with explicit applicability conditions; and independently verifies their semantics, registration scope, execution, and agent-facing selection.

Unlike WALT, which generates tools from demonstrated website functionality and execution traces, PACT derives tools directly from page and application evidence without requiring demonstration traces as generation inputs. Both approaches involve website exploration, but they differ in how reusable capabilities are identified and constructed.

Our goal is to provide an automated, application-grounded framework that connects tool construction, verification, and integration while preserving existing website workflows and human interaction. We have clarified this positioning in the revised Introduction and Section 3.

“Could the authors provide more direct ablations of dynamic registration and bounded execution?”

Bounded execution. PACT is designed to preserve the website's existing interaction structure for both human users and AI agents. Each tool implements a page-native capability within its applicable context, while subsequent navigation and tool composition remain under the calling agent's control. Allowing tools to continue execution across full-document navigation boundaries would require additional logic to establish and manage page states, effectively changing the scope and semantics of the generated capabilities. Sections 3.2.2 and 3.3 clarify this design choice and explain how agents navigate to capabilities on other pages.

Dynamic registration. We conducted an additional ablation comparing dynamic and static tool registration while keeping the generated tool set and agent configuration unchanged. The results show that static registration achieves comparable task success but incurs slightly higher token usage. This suggests that dynamic registration primarily improves the efficiency of tool exposure by restricting the available interfaces to those relevant to the current page state, rather than directly improving task success.

AuthorsOpenReview ↗

Author response for Reviewer u3ku - 2

“The Naive baseline is confounded.”

“App. A.8.1 does not state what generated the Naive set.”

“How much of the improvement over the Naive tool set comes from having more tools and broader functional coverage?”

“The paper also reports no statistics on verification.”

Thank you for raising these concerns. We have clarified the baseline setup and conducted an additional experiment to better distinguish the effects of verification and the complete generation pipeline.

Baseline setup and tool coverage. Both the Naive and PACT tool sets were generated using GPT-5.5, as now clarified in Section 4.1.2 and Appendix A.8.1. The Naive set was generated using WebMCP documentation without our generation skills, while PACT explicitly explores website states, discovers page-native capabilities, and generates and verifies tools. The resulting difference in tool count and functional coverage (126 Naive tools vs. 184 PACT tools) is therefore partly an intended outcome of our generation pipeline, rather than an uncontrolled difference in the generating model.

Effect of verification. To better isolate the role of verification, we conducted an additional experiment applying one round of verification to the existing Naive tool set. This process filters out 14 of the 126 tools without modifying the retained tools. The results are summarized below, with further details in Appendix A.13 (Table 21).

Verified Naive tool set. AgentOccam with Qwen-3.7-Plus is evaluated over 812 WebArena tasks.

ConditionSR (%)Tokens (all)Tokens (succ.)Output tokens (all)Output tokens (succ.)Steps (all)Steps (succ.)
Original agent42.1273.4k105.2k18.2k6.7k17.08.0
Naive tool set (126 tools)43.1 (+1.0)243.0k121.8k14.6k7.4k16.08.4
Verified Naive (112 tools)49.6 (+7.5)168.9k110.6k9.2k6.2k10.47.4
PACT (184 tools)53.3 (+11.2)151.2k57.8k9.7k3.7k9.94.8

Verification improves the Naive baseline's success rate from 43.1% to 49.6%, while reducing mean token usage from 243.0k to 168.9k and mean steps from 16.0 to 10.4. Importantly, this improvement is achieved by reducing the number of available tools, indicating that tool quality matters beyond tool quantity.

Effect of the complete pipeline. PACT further improves success to 53.3%, outperforming Verified Naive by 3.7 percentage points with lower token usage and fewer steps. Unlike verification-based filtering alone, our complete pipeline also includes systematic page-state exploration and capability discovery, which are designed to provide broader functional coverage. While the comparison does not fully disentangle the contributions of coverage and individual generation choices, it demonstrates that verification-based filtering alone does not account for the performance achieved by the complete PACT pipeline.

Verification statistics. We have also clarified in Section 4 that verification and revision are capped at five rounds per website, with early stopping when no further issues are detected. The verification procedure covers semantics, registration scope, execution, and agent-facing selection. The additional Verified Naive experiment provides quantitative evidence of how verification-based filtering affects downstream agent performance.

“Running [WALT's] tools inside the same agents on the same tasks appears feasible.”

We thank the reviewer for this suggestion. WALT provides both its own tool designs and a dedicated agent architecture. To ensure a more controlled comparison, we integrated WALT tools into CUGA and evaluated both WALT and PACT tools using the same agent (CUGA), reasoning model (Qwen-3.7-Plus), and 812 WebArena tasks.

Comparison with WALT. Results are reported in Appendix A.15 (Table 23).

ConditionSR (%)Tokens (all)Tokens (succ.)Output tokens (all)Output tokens (succ.)
CUGA: Original agent28.8360.7k179.9k5.0k3.1k
WALT agent + WALT tools37.2146.5k123.9k4.1k3.4k
CUGA + WALT tools40.4271.5k217.8k2.8k1.9k
CUGA + PACT tools44.3206.5k199.1k3.8k2.8k

Under the same CUGA agent, PACT outperforms WALT tools by 3.9 percentage points while reducing mean token usage by 23.9%. We additionally report WALT with its original agent architecture for reference, although differences in agent design prevent a direct attribution of performance differences to the tools alone. These results are summarized in Section 4.1.2.

AuthorsOpenReview ↗

Author response for Reviewer u3ku - 3

“‘Improves success across all 16 configurations’ … is not supported at the stated strength.”

Thank you for highlighting the importance of distinguishing observed improvements from statistical significance. The abstract, main results, and conclusion now describe the result as higher observed success rates across all 16 configurations. The paired Qwen-3.7-Plus results in Table 13 show how the effect varies: WebOperator gains 0.4 percentage points with a 95% confidence interval of [−2.4, 3.2], while CUGA and AgentOccam gain 15.5 and 11.2 points, respectively. Section 4.1.1 now points directly to this paired analysis and discusses success together with token and step usage.

“How does the BFS page-state collection process scale to large dynamic websites?”

PACT explores reachable states through page-native interactions and normalizes collected states by route patterns, UI conditions, and application context to avoid repeatedly processing equivalent states. This allows tool generation to focus on distinct website capabilities rather than enumerating every possible interaction trajectory. Nevertheless, we acknowledge that exhaustive exploration is difficult for large dynamic websites, particularly those with continuously changing content or extensive UI states.

To further examine applicability to larger and more diverse websites, we extended our evaluation to Online-Mind2Web. We generated 2,684 PACT tools across 144 hostnames, with coverage constrained by website-specific WebMCP registration limits. Importantly, PACT tools complement rather than replace predefined browser actions, allowing agents to continue interacting with websites when suitable tools are unavailable.

We additionally evaluated PACT on live websites using Online-Mind2Web (Appendix A.16). The results are summarized below.

Online-Mind2Web. AgentOccam with Qwen-3.7-Plus is evaluated on 210 tasks after excluding 90 task pairs affected by external errors.

ConditionSR (%)Tokens (all)Tokens (succ.)Output tokens (all)Output tokens (succ.)Steps (all)Steps (succ.)
Original agent29.5153.7k114.0k8.5k4.9k13.09.0
Full Page + Full PACT39.0 (+9.5)177.3k127.1k8.7k5.3k14.09.8

PACT improves task success by 9.5 percentage points, while increasing mean token usage by 15.4% and steps by 7.7%. Unlike the locally hosted WebArena environments, live websites can introduce additional interaction overhead, and available tools may not cover every required operation. Tool selection, invocation, and subsequent browser interactions may therefore increase resource consumption. The higher success rate may also reflect additional progress on tasks that would otherwise terminate unsuccessfully, although we have not isolated these effects.

These findings demonstrate that PACT can improve task success on diverse live websites, while highlighting a trade-off between task completion and interaction cost. We do not claim exhaustive page-state coverage, and improving exploration efficiency and tool coverage for large dynamic websites remains an important direction for future work.

“How much does the tool-generation and verification pipeline cost in practice?”

We have added construction-cost statistics in Appendix A.17. Generating the main 184-tool WebArena set required 1,381.0 million tokens, including page-state collection, tool generation, independent verification, and revision.

StageTokens (M)
Page-state collection101.7
Tool generation387.9
Independent verification764.4
Revision and re-verification127.0
Total1,381.0

These costs represent an upfront investment, as the generated tools can be reused across agents, models, and browsing tasks without regeneration. When a website changes, affected tools and registration conditions may require updating and re-verification. Our current experiments quantify initial construction costs, while maintenance costs remain to be systematically evaluated.

“Why does PACT provide such a small additional gain on A3-Qwen3.5-9B while increasing token usage?”

Our results suggest that web-agent fine-tuning reduces the incremental benefit of PACT. For the base Qwen3.5-9B model, PACT improves success by 4.8 percentage points while reducing token usage by 10%. However, for the fine-tuned A3-Qwen3.5-9B, the improvement decreases to 0.5 points, with token usage increasing by 9%.

One possible explanation is that fine-tuning already equips the model with effective browser interaction capabilities, reducing the additional benefit of structured tools, while tool selection and invocation introduce extra overhead. However, our experiments do not isolate these factors. We clarify this trade-off in Section 4.1.5 and explicitly describe the small A3 improvement as descriptive rather than statistically established.

AuthorsOpenReview ↗

Follow-up on Rebuttal and Remaining Concerns

Dear Reviewer u3ku,

Thank you for your time and consideration of our submission. As the discussion period is coming to an end, we would like to kindly follow up on our rebuttal and ask whether our responses have adequately addressed your concerns.

If you have any remaining questions or concerns, we would be happy to provide further clarification before the discussion period closes.

Thank you again for your valuable feedback.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.