acceptodds
Desk rejected at ICLR 2027

VERA: Verification and Environmental Risk Awareness for Functional Model Induction through Open-Ended Web Exploration

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Open-ended GUI exploration offers a way to acquire application knowledge before downstream tasks are specified. However, successful tool execution does not establish that a functional claim is supported, and the interactions used to test unknown functionality may involve environmental risks. We introduce VERA, a new framework for constructing functional models through evidence-based knowledge admission and contextual risk awareness. Each function is represented as a hypothesis with an observable outcome specified before execution. VERA compares subsequent interface observations against this expectation, admits supported claims, and retains unresolved or unsuccessful attempts for inspection and further exploration. Persistent hypothesis pools and context recovery sustain evidence collection, while risk annotations associate selected actions with their acquisition context. In controlled web exploration studies, evidence-based admission raises knowledge precision from 82.5% to 97.0% while retaining 32 of 33 human-supported functions. Full exploration attains greater functional coverage at additional cost, while visual context improves risk categorization on an external context challenge. These results demonstrate VERA's effectiveness in acquiring reliable functional knowledge while making the environmental risks of its acquisition explicit and traceable. The source code is available at https://anonymous.4open.science/r/AI-Web-explorer-4516.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
web agents, open-ended web exploration, GUI agents, functional model induction, evidence-grounded verification, knowledge admission, environmental risk awareness

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_fBP4OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

VERA acquires functional application knowledge through open-ended web exploration. It specifies expected outcomes before execution, verifies them using before/after screenshots, and admits supported claims into persistent knowledge. Recovery mechanisms support continued exploration, while contextual risk annotations document potentially sensitive operations.

Critical strengths

1.The paper addresses a meaningful reliability problem: successful tool execution does not necessarily justify retaining a functional claim. The overall workflow is understandable and appropriately separates outcome verification from risk assessment.

2.The controlled admission comparison on identical traces is useful. Precision improves from 82.5% to 97.0% while retaining almost all human-supported functions, providing encouraging evidence for evidence-based admission.

Critical weaknesses

1.The specific methodological contribution is not sufficiently isolated. The admission baselines compare verification against accepting proposals or executor success. They do not establish the value of fixing expectations before execution over ordinary screenshot-based outcome verification. A matched verifier without precommitted expectations would help clarify the incremental contribution.

2.The evaluation is narrow and does not establish downstream usefulness. The core admission study contains only 45 functions from two applications. Moreover, Appendix H demonstrates symbolic compatibility rather than actual downstream task execution. Consequently, both generalization and the practical benefit of the induced functional knowledge remain insufficiently demonstrated.

3.The exploration gains are not evaluated under comparable total cost. On SauceDemo, Full uses 138 high-level calls versus 34 for Linear. Although these costs are transparently reported, higher attained coverage under substantially more interaction does not establish a cost-effective advantage for the recovery mechanism.

Questions for authors

How does VERA compare with an otherwise identical outcome verifier without frozen expectations?

1 comment

Thank you for your comments on the contribution, evaluation scope, and exploration cost.

W1 and Q1: Expected outcomes and knowledge verification

VERA's functional hypothesis specifies a location, action, possible preconditions, and an expected observable outcome. Fixing the expectation before execution gives the verifier a criterion for assessing the original claim against subsequent observations.

Following your suggestion, we compared VERA with a verifier without a preset expected outcome using the same actions and screenshot pairs:

VerifierAdmission evidence supportSupported-attempt retention
VERA88.2%96.3%
No preset expectation84.3%93.6%

Admission evidence support is the fraction of admitted attempts that are supported; supported-attempt retention is the fraction of supported attempts admitted. The comparison covers the two shopping sites and RealWorld, including 166 attempts with complete screenshots. VERA's two metrics are higher by 3.9 and 2.7 percentage points, respectively. These results support knowledge admission based on a preset functional claim: the expected outcome provides a clear object of verification while retaining attempts supported by observational evidence.

W2: Evaluation scope and downstream use

We chose sites whose functions and outcomes can be checked individually, and added three runs on the non-shopping RealWorld content community. Initial attempt-level evidence assessments yield:

SiteDomainRunsAttemptsAdmission evidence supportSupported-attempt retention
SauceDemoShopping348100.0%100.0%
Practice ShoppingShopping35574.3%86.7%
RealWorldContent community36790.4%100.0%

Four Practice Shopping attempts with incomplete screenshots are excluded from the proportions. In RealWorld, VERA verified article browsing, filtering, editing, and publishing. Its admission evidence support rate was 90.4%, and all clearly supported attempts in the initial review were retained. The system also recorded contextual risks for evidence-gathering actions, illustrating this knowledge-acquisition process in a non-shopping setting. The RealWorld candidate scan included an additional instruction to check visible top navigation.

For downstream use, the controlled admission study on identical traces increases knowledge precision from 82.50% to 96.97% while retaining 32/33 supported functions. Appendix H shows that all three exported models support symbolic planning: two yield plans to order completion, while the third lacks a necessary transition and cannot reach that goal. This illustrates how knowledge admission affects symbolic reachability.

W3: Exploration mechanism and cost

Recovery provides further opportunities to collect evidence when the current interface's candidates are exhausted or verification does not support a claim. All conditions use the same ordinary-attempt cap; extra recovery calls are counted in the cost.

On SauceDemo, mean functional coverage across three runs was 59.1% for Full and 39.4% for Linear. Their respective totals were 138 and 34 high-level calls. Full's total comprised 62 ordinary attempts and 76 recovery/reset calls. These figures show the observed coverage gain alongside its interaction cost.

Reviewer_jMeqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VERA, a knowledge acquisition framework for inducing functional models through open-ended web exploration. VERA generates candidate functional hypotheses, executes the corresponding interactions and uses visual evidence to verify their outcomes before admitting them into the knowledge base. A separate annotator assesses the potential risks of actions in their current context. Experiments on two shopping applications evaluate the precision of admitted functional models and the coverage achieved through exploration.

Critical strengths
  • VERA retains failed and uncertain attempts, making the acquired knowledge more inspectable

  • The paper provides useful experimental details, including er-run coverage and interaction costs, confusion matrices, and concrete false-admission and false-rejection cases

Critical weaknesses
  • Incremental methodology

Predicting an expected effect before acting and verifying it afterwards is explored by VeriGUI, world-model guardrails, UI-KOBE, ActionEngine. The statement ``knowledge admission as an explicit objective'' is largely a relabeling of verified exploration. The risk annotator also resembles WebGuard-style classification with a five-category taxonomy. This paper lacks either detail technical comparison or experimental comparison against these related work.

  • Unclear benefit of combining verification and risk assessment.

This paper brings together outcome verification and contextual risk assessment, but these two components operate separately and are evaluated separately. Whether the combination of these two components are beneficial is unknown.

The practical value of the acquired functional models also remains insufficiently evaluated. The main experiments focus on knowledge precision and functional coverage. While Appendix H provides a preliminary application through symbolic planning, it demonstrates interface compatibility and symbolic reachability rather than successful execution of the generated plans.

  • Limited and biased evaluation

The core knowledge acquisition experiments cover only two applications, both in the shopping domain. This narrow scope leaves the framework's applicability to other types of applications uncertain.

More importantly, the knowledge reference labels are initialized by an AI model and then checked by a single human reviewer. Relying on one reviewer without independent annotation or cross-checking leaves potential anchoring effects and subjective judgments insufficiently assessed.

Questions for authors
  • How does VERA perform against existing work such as VeriGUI, world-model guardrails, UI-KONE, Action Engine, WebGuard, SafePred, and OS-Sentinel?

  • What benefit does integrating outcome verification with risk assessment provide?

  • Does using the acquired functional models improve downstream task success or execution efficiency?

  • How does VERA perform on applications outside the shopping domain?

2 comments

Thank you for your comments on the contribution, evaluation scope, and practical use of VERA.

W1: Incremental methodology

VERA starts from knowledge acquisition in open-ended exploration: when website functionality is unknown, the system must assess both whether observations support the acquired functional knowledge and what consequences the actions used to obtain those observations may have.

VERA uses a functional claim throughout the process: before execution, it specifies the function to verify and its expected outcome, and assesses the potential risk of the evidence-gathering action; after execution, it uses interface observations to decide whether to admit the claim to the functional model, retaining the rationale and acquisition context.

VERA's contribution is to establish and evaluate this process, allowing the evidence for functional knowledge and the risks of obtaining it to be examined together. We further discuss its relationship to the cited work in Q1.

W2: Benefit of combining verification and risk assessment

VERA links pre-action risk assessment and post-action outcome verification to the same functional claim. The functional model thus retains both whether the claim is supported by observation and the risk context of the evidence-gathering action, facilitating review of knowledge acquisition.

We examine the two judgments through separate knowledge-admission and risk-assessment experiments: the former measures knowledge precision and supported-function retention, while the latter evaluates the effect of interface context on risk judgments. Linked records allow these judgments to be associated with the same exploration attempt.

W3: Evaluation scope and reference labels

To broaden the application domain, we conducted three additional runs on RealWorld, a content community with article browsing, filtering, editing, publishing, comments, and account settings; see Q4.

To address concerns about AI initialization and single-person review of reference labels, two researchers independently assessed 103 attempts from SauceDemo and Practice Shopping using a common guideline, without access to the original reference labels or system decisions. Their annotations agreed on 94/103 attempts (91.26%), with Cohen's κ = 0.856, demonstrating high inter-annotator agreement and providing further support for the reliability of the evidence assessments.

Q1: Relation to prior work

VeriGUI uses action-effect verification for failure detection and correction; UI-KOBE and ActionEngine organize exploration knowledge for later use; and WebGuard, SafePred, and OS-Sentinel study interaction risks.

VERA focuses on knowledge acquisition within these stages: what evidence admits an explored functional claim into a persistent model, and what potential risk was incurred to obtain that evidence. It organizes functional hypotheses into testable claims, uses observations to decide admission, and links the acquisition context to each claim.

To evaluate knowledge admission, we compare admission rules on the same candidates and interaction traces; the risk experiment examines the effect of interface context on risk judgments. These results show VERA's performance in knowledge admission and assessment of evidence-gathering risk. The cited methods focus on execution correction, knowledge organization, or interaction risk; direct comparison with complete systems would require aligning their inputs, outputs, and evaluation goals.

Q2: Combining outcome verification and risk assessment

In real-world open-ended exploration, an agent may interact with a site to test unknown functionality, while the evidence-gathering action itself may modify data, initiate a transaction, or have other consequences. An attempt therefore raises two questions: whether the interface observation supports the proposed functional claim, and what potential risk the action used to obtain it carries.

The first concerns the evidential validity of the knowledge; the second concerns the potential consequences of obtaining evidence. Together, they characterize the knowledge-acquisition process. VERA organizes both judgments around the same claim, allowing later review to inspect both the basis of the knowledge and its acquisition context.

Q3: Downstream use

VERA organizes functions, semantic locations, functional relations, and supporting evidence into a structured model for downstream planning. In Appendix H, all three Practice Shopping runs exported symbolic models; two yielded a plan to an order-completion goal. The third could plan only to cart-related goals because a necessary transition was not admitted. This illustrates how admission affects symbolic reachability.

On identical traces, VERA increased knowledge precision from 82.50% to 96.97% over executor-success admission, reduced unsupported admissions from 7 to 1, and retained 32/33 supported functions. This directly measures the quality of the functional knowledge on which downstream planning depends.

The experiments therefore show both the evidential reliability and symbolic-planning utility of VERA's knowledge: the former evaluates which functions enter the model with support, and the latter illustrates how those functions support planning for concrete goals.

Q4: A non-shopping website

We conducted three RealWorld runs. The table uses attempts as the unit: admission evidence support is the proportion of admitted attempts supported by the reference assessment; supported-attempt retention is the proportion of supported attempts admitted.

SiteDomainRunsAttemptsAdmission evidence supportSupported-attempt retention
SauceDemoShopping348100.0%100.0%
Practice ShoppingShopping35574.3%86.7%
RealWorldContent community36790.4%100.0%

On RealWorld, VERA verified functions including article browsing, filtering, editing, and publishing. Its admission evidence support rate was 90.4%, and all clearly supported attempts in this analysis were retained. The system also recorded contextual risks for evidence-gathering actions, illustrating this knowledge-acquisition process in a non-shopping setting.

Reviewer_ArsDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper combines known ideas to create a framework for for constructing functional models proposing an agent workflow allegdly managing hypothesis in novel ways, e.g. through risk annotation. There is no theoretical contribution and evaluation lies on an ablation study without comparison to other approaches.

Critical strengths
  • Open-ended exploration is a timely and important research direction
  • Some of the high level ideas are known but the paper provides some interesting thoughts (see also next point) though their novelty is not well-established and overall their integration appears rather incremental.
  • The contribution relies on combining known things in a way that makes innovation questionable, e.g., the separation of concerns (discover vs reuse) sounds interesting, also assessing risks is a worthy problem to tackle. But the mechanism is simply a workflow combining existing ideas (correct me if you disagree). But as agents constantly reflect on usage and outcomes with very rich literature on these ideas and on defining workflows (see the paper's own related works and some more below), claims must be more concise better embedded into existing literature clearly showing how each of these elements differs from prior work (if so) and benefits must be clearly shown beyond an ablation study.
  • The paper shows through an ablation study and two benchmarks that their approach leads to gains.
  • The paper does not compare against any other works, thus, it is unclear if the paper is really beneficial. Consider your own cited works (e.g. the 4 below) and maybe also those under the next point. I understand that some of these might not have all elements you have and not all of them solve the exact same problem, but you need to make clear that they and others cannot be applied to your problem if you don't compare.
  • GUI-explorer: * UI-KOBE: * ActionEngine: * Don't Act Blindly: Robust GUI Automation via Action-Effect Verification and Self-Correction
  • There are concerns regarding novelty, please discuss differences to: * PropGen, * OSExpert * SkillWeaver
Critical weaknesses

see above

Questions for authors

see above

2 comments

Thank you for your comments on VERA's contribution and evaluation.

1. Contribution

Open-ended exploration builds reusable application knowledge before the website's functionality is known. Testing a functional hypothesis raises two related questions: whether the observation supports the resulting knowledge, and what consequences the evidence-gathering action may have.

VERA uses a functional claim to connect these questions. Before execution, it specifies the expected observable outcome and assesses the potential risk of the selected action. After execution, it uses interface observations to decide whether to admit the claim to the functional model, linking the evidence and acquisition context to the same record. The contribution is an evaluated knowledge-acquisition process in which support for a claim and the risk of obtaining its evidence can be examined together.

2. Relation to prior work

GUI exploration includes candidate discovery, action execution, and the organization of experience for later tasks. GUI-explorer emphasizes function-oriented exploration; UI-KOBE and ActionEngine organize exploration knowledge for reuse; PropGen verifies functional hypotheses to generate test properties; SkillWeaver and OSExpert turn experience into reusable skills; and VeriGUI uses expected effects to detect execution failures and guide correction.

VERA focuses on the reliability and risk of knowledge acquisition. It distinguishes executor success from observed support for a functional claim, and uses the latter to govern admission. It also links the contextual risk of evidence collection to the claim and attempt record, making both the basis of the knowledge and its acquisition context inspectable.

3. Comparison and evidence

The cited systems address exploration, skill learning, action correction, or property generation. Comparing complete systems would also introduce differences in exploration, execution, and knowledge use. To isolate knowledge admission, we compare admission rules on the same candidates and interaction traces; a separate study evaluates risk assessment.

Relative to admitting functions based on executor success, evidence-grounded admission increases knowledge precision from 82.50% to 96.97%, reduces unsupported admissions from 7 to 1, and retains 32 of 33 supported functions. This shows how the admission rule selects evidence-supported functional knowledge from the same exploration traces.

Thanks, what was asked by me and also reviewer jMeq is to compare against approaches. In the paper it is neither visible how you compared exactly against mechanisms from other papers nor full systems (which you claim makes limited sense).

There are many candidates and its the authors job to investigate them carefully and compare also during evaluation - just to give one more pointer as the authors deny entire systems evaluations, there is also work dealing with reliability etc., e.g., Reliable Knowledge Acquisition for Continual Learning Agents (RIKA) (2026).

Reviewer_LmsgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper looks at when an observation from GUI exploration is reliable enough to become reusable application knowledge. VERA writes down the expected outcome before an action, checks the before/after screenshots against that expectation, and decides whether to admit the claim. It keeps failed and uncertain attempts for later inspection, uses persistent candidate pools and recovery to continue exploring, and attaches contextual risk labels to the interactions. These labels are recorded without changing execution or admission.

Critical strengths

1.In general I like the core idea here. Separating "the tool didn't crash" from "the action actually worked" by locking in expectations before clicking is a smart way to stop agents from hallucinating success. 2.I appreciated how openly the paper reports its limitations. The recovery study includes the extra interaction cost, and the risk study acknowledges the uncertainty.

Critical weaknesses
  1. Limited scale: Core exploration is tested on only two simplistic apps with a 25-attempt cap. Scalability to complex, dynamic real-world web environments remains unproven.
  2. The whole system relies heavily on the VLM perfectly judging the before/after screenshots. If the locked-in expectation is slightly too strict, it wrongly tosses out valid functions, and there's no way to recover from that.
Questions for authors

1.I would like to see how VERA performs against a standard outcome verifier without pre-execution expectations, using the same model and observations.

2.Could the authors report attempt-level admission precision? I am particularly interested in whether each admitted claim is supported by evidence from that same attempt.

3.It would be helpful to see the precision and retention results for individual runs, rather than only the pooled results, to understand how much aggregation affects the findings.

1 comment

Thank you for your comments on the experimental design, knowledge admission, and verification mechanism.

W1: Experimental scale and research focus

GUI exploration spans candidate discovery, interaction verification, and the organization and use of acquired knowledge. VERA focuses on knowledge acquisition: when a functional claim has sufficient evidence to enter a reusable model, and what potential risks accompany evidence collection.

Large-scale performance also depends on the exploration strategy and executor. A comparison of complete systems would reflect multiple components and would not isolate knowledge admission. We therefore selected websites whose functions and outcomes can be checked individually, allowing us to assess admission precision and supported-function retention against explicit references.

The cap of 25 attempts applies to ordinary high-level attempts: each tests one functional hypothesis and may involve multiple low-level browser interactions. The same ordinary-attempt cap applies across exploration conditions; additional recovery calls are counted separately.

W2: VLM judgments and overly narrow expectations

VERA's VLM verifier assesses whether observations support a functional claim using the action, the expected outcome fixed before execution, and before/after screenshots. Admission depends both on the VLM's interpretation and on the expectation. VERA retains hypotheses and attempt records, allowing bounded within-run retries and cross-run aggregation to provide further opportunities for verification.

For example, Appendix H.1 describes a cart operation rejected because its expectation was too narrow; a later attempt supported the same function and admitted it to the cumulative model. Thus, one false rejection does not mean that the function is permanently lost. We further analyze attempt-level admission and function retention in Q2/Q3.

Q1: Comparison with a verifier without a preset expectation

VERA aims to build reusable web functional models. Each functional hypothesis includes an operation location, an action, possible preconditions, and an expected observable outcome. The expected outcome is a natural part of the hypothesis: fixing it before execution provides a criterion for assessing the original claim against subsequent observations.

Following your suggestion, we compared VERA with a verifier without a preset expected outcome on the same actions and screenshot pairs:

VerifierAdmission evidence supportSupported-attempt retention
VERA88.2%96.3%
No preset expectation84.3%93.6%

Admission evidence support is the fraction of admitted attempts supported by the reference assessment; supported-attempt retention is the fraction of supported attempts admitted. The pooled comparison includes 166 attempts with complete screenshots across the two shopping sites and RealWorld. VERA's two metrics are higher by 3.9 and 2.7 percentage points, respectively. The results support knowledge admission based on a preset functional claim: the expected outcome provides a clear object of verification while retaining attempts supported by observational evidence.

Q2–Q3: Attempt-level evidence and aggregation

Across 103 attempts in six runs on the two original websites, 58 of 67 admitted attempts (86.6%) had support in their own before/after screenshots; the remaining nine were indeterminate under this assessment.

MetricFirst attempt¹After within-run retries¹Cross-run aggregation²
Admission yield72.3%81.5%82.5%
Supported-function retention90.4%94.8%96.9%
Admission precision93.0%87.1%93.9%

¹ Mean over six runs, using run-level function keys. ² Function keys pooled across runs within each site.

Site and runAdmission precisionSupported-function retention
SauceDemo 1100.0%100.0%
SauceDemo 2100.0%100.0%
SauceDemo 3100.0%100.0%
Practice Shopping 175.0%90.0%
Practice Shopping 275.0%90.0%
Practice Shopping 372.7%88.9%
Cross-run aggregation¹93.9%96.9%

¹ Cross-run results use deduplicated functions; individual-run results use functions in the respective run.

Both tables are based on attempt-level evidence; the paper uses the original function-level reference labels. Indeterminate admissions are not counted as supported. Within-run retries increase mean supported-function retention from 90.4% to 94.8%; cross-run aggregation retains 96.9% of supported functions. These results show that a single false rejection need not cause a function to disappear from the cumulative model.

Comments on the paper

2 comments
Area_Chair_voyTOpenReview ↗

Reviewers responses to authors

Dear reviewers,

Author-reviewer discussions end on November 18th. If possible, please respond to author comments by that time so that authors have opportunity for any additional responses.

Appreciate reviewers who already engaged in the discussion.

Thank you.

Dear Area Chair and Reviewers,

We sincerely thank all reviewers for their valuable feedback. As the discussion period draws to a close, we would like to briefly summarize our main contributions and the work completed during the discussion.

Paper Summary

We are pleased to see the reviewers’ recognition of our work in open-ended web exploration, an important research direction (Reviewer ArsD): VERA acquires reliable functional knowledge supported by observational evidence through outcome verification (Reviewers Lmsg and fBP4), identifies potential risks that evidence-gathering actions may pose to the application environment through contextual risk assessment (Reviewer ArsD), and retains records that make both the acquired knowledge and the process by which it was obtained traceable (Reviewer jMeq).

Discussion Period Summary

During the discussion period, we responded to the questions raised by all four reviewers and provided the following additional experiments, analyses, and clarifications in our rebuttal.

  • Methodological contribution and relationship to prior work (Reviewer ArsD, Reviewer jMeq). We clarified how VERA relates to prior work and explained why it connects outcome verification and the risks of evidence collection to the same functional claim.
  • Role of preset outcomes in verification (Reviewer Lmsg, Reviewer fBP4). We compared VERA with a verifier without preset outcomes on 166 attempts across three websites, using the same actions and before-and-after screenshots. The proportion of admitted attempts supported by evidence was 88.2% for VERA and 84.3% for the comparison verifier.
  • Evaluation scope and reliability of reference labels (Reviewer Lmsg, Reviewer jMeq, Reviewer fBP4). We conducted three additional exploration runs on RealWorld, a non-shopping content community, and two researchers independently annotated 103 attempts from the original two websites.
  • Retention of supported functions and recovery costs (Reviewer Lmsg, Reviewer fBP4). We added attempt-level evidence checks and per-run analyses to show how retries and cross-run aggregation affect the retention of supported functions. We also reported the coverage gains from recovery and the associated interaction costs.
  • Downstream use of functional models (Reviewer jMeq, Reviewer fBP4). Using the symbolic planning example in Appendix H, we explained how knowledge admission affects which planning goals are reachable.

Detailed results and analyses are available in the corresponding review threads. Thank you again for your time and consideration.

Sincerely,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.