acceptodds
Under review as a conference paper at ICLR 2027

Web$hell: A Unified Interface for Agentic Web Access

15 views

Abstract

Large language model (LLM) agents are becoming direct users of the open web. As they take on increasingly complex and information-intensive tasks, effective web access is becoming a core capability for agentic problem solving. Existing approaches have advanced agentic web access through different , including search-augmented agents, browser-based agents, and, more recently, filesystem-based agents. Nevertheless, as users expect agents to complete long-horizon, end-to-end tasks, they increasingly need to conduct broad information gathering, perform browser actions, and revisit acquired evidence within a single trajectory. Yet our preliminary analyses reveal that existing access models struggle to meet these demands. They primarily support either information gathering or browser interaction, while combining their capabilities can involve substantial costs or performance trade-offs. To this end, we present Web\\text{\\} textitobjects text $ text $$hell opening new possibilities for more capable and efficient LLM agents on the web.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Agentic Web, LLM Agents, Agent Interfaces, Web Agents

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_o8nVOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Web$hell, a framework that provides a unified interface for web agents to search for information, interact with webpages, and revisit stored evidence. Preliminary analysis shows that previous web search and navigation methods do not generalize well across web tasks, showing limitations of using only certain types of access model. To mitigate the this problem, the proposed approach represents content and controls as addressable objects, uses a deterministic compiler to preserve their relationships, and caches page states for selective access. The proposed approach is reported to obtain the best performance among the evaluated methods on multiple benchmarks, generally with fewer steps and comparable or lower LLM inference costs.

Critical strengths
  • This paper aims to address a practical problem that web agents usually need both information gathering and browser interaction.
  • The system design is well-motivated and coherent, and the evaluation on multiple benchmarks shows the effectiveness of the proposed approach.
Critical weaknesses
  • It seems that the main underlying LLM is gpt-5.6-luna, and it will be better if more models can be tested to see how well the proposed framework can fit different models, and especially whether some components is dependent on specific model choices.
  • There is a lack of ablation studies on the components of the framework, and the source of improvement is not fully isolated since it changes too many aspects at the same time.
Questions for authors
  • Does the cost mesurement includes all the aspects, such as fetching and compilation, if not, what are the end-to-end latency and total resource costs, including all different aspects?
  • At many times, the screenshot of the web pages can contain helpful information, would it be feasible to include such information in the proposed framework?
  • I'm wondering whether there will be errors or exceptions when processing and compiling the web pages. Since real-world pages can be noisy and contain errors themselves, how to ensure the processing can handle different kinds of cases?
Reviewer_N3h8OpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper addresses how LLM agents can combine broad information gathering, webpage interaction, and evidence reuse within a single web interface. In a preliminary analysis, the authors show that existing search-augmented, browser-based, and filesystem-based agents struggle to generalize across both search and navigation tasks. Simply combining search and browser observations (HybridSBHybrid_{SB}) sharply increases per-call input tokens, while adding a filesystem with shell operations (HybridSBFHybrid_{SBF}) hurts navigation because selective reading often misses the context needed to operate page controls. To address these issues, the paper proposes WebShell, a unified interface built around three components: (1) WebContract, which models webpage information and interactive controls as addressable objects with an address, a value, and contextual associations, organized under a standard file layout and accessed through shared operations; (2) WebCompiler, which uses deterministic rules based on HTML semantics and ARIA attributes (without LLM inference) to compile live webpages into structured files while linking content to related actions; and (3) WebCache, which stores compiled page snapshots so agents can interact with live pages while still inspecting earlier page states as read-only records. Across six benchmarks covering web search (WideSearch, SealQA), web navigation (WebArena-Verified, WebWalkerQA), and deep research (DeepResearchBench, DeepWeb-Bench), WebShell achieves the highest task scores among the compared baselines in the main table, while using fewer agent turns and competitive inference cost.

Critical strengths

The paper's main strengths are its clear problem formulation and practical interface design. First, the preliminary analysis in Section 3 gives a clear quantitative motivation for a unified interface, showing that naive hybrid access increases per-call input tokens by 290% to 626% and that selective grep inspection can separate page text from the controls needed to act on it. Second, while individual components such as persistent workspaces (Fetch-then-Explore), rule-based DOM restructuring (AgentOccam), and filesystem-style browser shells (CLI-Anything / DOMShell) already exist in prior work, WebShell integrates them into a coherent object and operation schema. In particular, keeping contextual associations and action references attached to selectively read content (including table cells and grep matches), alongside maintaining both read-only historical snapshots and live browser states, directly addresses the gap between reading and acting. Third, the empirical evaluation covers six diverse benchmarks, controls for temporal drift through concurrent runs, blocks HuggingFace reference pages to prevent data leakage, and shows stable selective-access cost when up to 400 distractor pages are added to the cache.

Critical weaknesses

The main limitations lie in isolating the exact source of the performance gains and clarifying several experimental controls and scope conditions:

1、Confounding interface design gains with additional page-loaded data: As shown in Figure 4 and Appendix C.2, WebCompiler extracts not only rendered DOM text and controls, but also embedded structured data and same-origin text responses captured during page loading, compiling them into searchable tables or fact records. Because the baselines mainly operate on cleaned page text or AXTrees, the reported improvements in task accuracy and evidence coverage (Figure 5) appear to mix two factors, namely exposing extra underlying data from network and page-load responses versus organizing the same visible page content into a better representation. Without an ablation that disables this extra data extraction or provides the same captured data to baselines, it is hard to tell how much of the gain comes purely from the unified interface representation.

2、Omitting the strong HybridSBHybrid_{SB} baseline from the main navigation evaluation:

In the preliminary analysis (Section 3.2) on 100 sampled WebArena-Verified tasks, HybridSBHybrid_{SB} without filesystem filtering achieves a 74.0% success rate. However, the main evaluation (Table 3) only includes HybridSBFHybrid_{SBF} (59.0%), which suffers from navigation degradation, while leaving out HybridSBHybrid_{SB}. Including HybridSBHybrid_{SB} on the full test set is necessary to directly evaluate the quality and cost trade-off of WebShell (67.9%) against the strongest hybrid navigation baseline.

3、Confounded turn reductions and cross-reference ablations:

First, Appendix C.2 notes that WebCompiler merges related form fields and their submission button into a single callable action, as illustrated by the single-step search form call in Figure 4. Thus, the reduction in agent turns partly reflects coarser, bundled action granularity rather than reduced page re-interpretation alone. Second, the ablation in Figure 6 removes only cross-references between information and action objects rather than the full Association field, and simultaneously changes how action IDs are obtained and invalidated, since it requires a separate control listing request whose IDs expire upon page changes or new listing requests according to Appendix B.2. As a result, the 6.0 to 10.1 percentage point drop mixes the loss of inline action links with extra listing steps and changed handle lifecycles.

4、Conditions on cost savings and unverified generalization boundaries:

A closer look at Appendix Table 8 shows that token efficiency varies by task category. While WebShell reduces both input and output tokens in Deep Research, it actually uses 21.6% more total input tokens than Fetch-then-Explore in Web Search. Its similar dollar cost in Web Search relies on the provider's 90% discount for cached prompt inputs and slightly fewer output tokens, which may not transfer to serving setups without discounted prefix caching. Furthermore, some benchmark margins are narrow without reported variance across runs. In addition, the main agent is only tested with a single backbone, and the paper does not quantify how often deterministic HTML and ARIA rules fail to identify or contextualize custom controls or elements that only appear after hover or scroll events.

Questions for authors

1、Separating representation gains from extra page-loaded data: To isolate the benefit of the unified interface from extra data sources, how do WebShell's task performance and evidence coverage on WideSearch, SealQA, and DeepWeb-Bench change if you disable the extraction of embedded structured data and same-origin network responses, or alternatively expose those same captured records to the baselines?

2、Comparison with HybridSBHybrid_{SB} and impact of bundled actions: Since HybridSBHybrid_{SB} achieved 74.0% on the preliminary WebArena-Verified subset, what are its success rate and token cost on the full 789-task WebArena-Verified set and on WebWalkerQA? Also, on average, how many low-level browser actions (such as typing into inputs plus clicking submit) are bundled into a single execute call in WebShell?

3、Cleaner ablation of information-to-action cross-references: In Figure 6, removing cross-references also forces the agent to request a separate, non-persistent control listing. If actions.tsv remains directly accessible in the page directory with the same ID stability as in the full system, and only the inline action pointers inside information objects are removed, how much do navigation accuracy and cost change?

4、Rule coverage on real webpages and broader applicability: Across the evaluated webpages, what proportion of interactive controls fail to be recognized as action objects or lose their contextual labels due to missing HTML and ARIA semantics? How does WebShell handle controls that are dynamically injected into the DOM only after hover or scroll events? Finally, how well do smaller or open-weight models follow the 8-operation schema and addressing syntax without producing invalid addresses or tool-call errors?

Note to Authors: If you can satisfactorily address these four questions during the rebuttal phase with empirical evidence or clarifying experiments, I would be willing to raise my score to a 4.

Reviewer_bUjWOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper presents a common interface for web agents that converts web pages into information and action objects thru WebContract and WebCompiler. it stores page content and history in WebCache and provides shared operations. it shows this env gives models best results across six benchmarks at similar/lower cost.

Critical strengths

Design is simple and clear. Ablation tests the effect of linking information and action objects. I checked the reported numbers. Token totals in Table 3 match Table 8, and accuracies in Table 3 correspond to exact counts for n=254 and n=420.

Critical weaknesses
  1. This paper seems to have missing baselines? HybridSB has 74.0% on WebArena verified sample, which is the highest, but table 3 only has HybridSBF 57.8%. BrowserUse and AgentOccam are also missing from the navigation comparisons.

  2. All results come from single runs. This likely confounds the observed results. Please add repeated runs to show reliable improvements.

  3. Fig 6 reports 56.9% accuracy on 100 WebWalkerQA questions (App. B.2). This is not possible for one run of 100 binary-scored questions. Please explain how you got the accuracy.

  4. The search-oriented average of 65.8 in fig1 matches the mean of FtR, Search-o1, and WebThinker in fig 5 but it excludes FtE, yet sec3 includes FtE in this group.

  5. The costs in Table 8 cant be reproduced using fixed price for each token type. The implied cached-to-uncached price ratio is about 0.06-0.09, but the paper states 10%.

  6. LLM suggestion checking against relevant papers- the paper misses three relevant approaches published before the July 17 cutoff: Beyond Browsing: API-Based Web Agents (ACL Findings 2025), Internal APIs Are All You Need (arXiv:2604.00694), and element-reference accessibility snapshots in Playwright MCP.

  7. You only tested one model family

Questions for authors

Why is HybridSB missing from Table 3? Can the authors report HybridSB and BrowserUse on full WebArena-Verified and WebWalkerQA?

Can the authors report at least three runs with confidence intervals for Table 3?

How many questions and runs were used for the 56.9% result in Fig. 6?

Which agents and queries were used to calculate the average in Fig. 1?

What prices were used for Table 8? Why are the costs in Fig. 6 different from Table 3?

Do the results hold with another model family? - Can you test 3?

Happy to raise the score if you properly address or rebut these questions.

1 comment

Thanks for the detailed response. The repeated runs, the full 420 question ablation and the extra model families address most of my concerns. the cost explanation (cache-write pricing) makes sense now. Raising my score.

One thing left. On full WebArena-Verified HybridSB is 0.649 vs 0.671 for Web$hell, so the margin there is about 2 pts, not the 7-11 pts vs HybridSBF in your table. Please put HybridSB in Table 3 and adjust the claim. The cost difference ($0.017 vs $0.066) is the stronger point here anyway.

Also the extra model family results are 100 tasks and one run each, so some margins are within noise. would be good to say so in the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.