acceptodds
Under review as a conference paper at ICLR 2027

CAVEAT: Towards Robust Computer-Use Agents in Incentive-Misaligned Environments

Abstract

Computer-use agents (CUAs) increasingly act on behalf of users online. What happens when the environments they operate in have incentives of their own? Online marketplaces, for example, may favor some products over others, steering agents away from the user's objective. Existing CUA benchmarks cover cooperative settings or explicit attacks, but do not test whether agents preserve user objectives when the environment itself has a stake in the outcome. We introduce CAVEAT, a controlled benchmark spanning nine marketplace environments and a taxonomy of eight common steering mechanisms. Across five model families, agents purchase the user-optimal product in 78.6% of matched-control episodes but only 17.3% when steering mechanisms are enabled. Larger models and more reasoning improve robustness, but substantial failures persist. Trajectory analysis and targeted ablations identify three weaknesses in how agents decide: they (1) prematurely narrow the set of alternatives they consider, (2) impose priorities the user never stated, and (3) commit before resolving decision-relevant evidence. Guided by this diagnosis, we develop CAVEAT-Harness, which targets these failures and raises the optimal purchase rate by up to 80.0 percentage points, and show that targeted post-training further improves a smaller open model. These results establish incentive robustness as a distinct challenge for delegated agents, diagnose failure modes, and show how targeted interventions can substantially improve robustness.

Primary area:
datasets and benchmarks
Keywords:
Computer-use agents, Incentive misalignment, Agent robustness, Benchmarking

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_f3V9OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CAVEAT studies whether CUAs preserve user objectives when truthful marketplace presentation favors other products. Nine simulated marketplaces implement eight steering mechanisms, with matched controls preserving catalogs, product facts, and the unique user-optimal purchase. Across five model families on 52 tasks, optimal purchasing drops from 78.6% to 17.3% under steering. Single-mechanism experiments and trajectory analyses examine search closure, preference interpretation, and unresolved evidence. A task-specification and verification harness substantially improves selected models, including an 80-percentage-point gain on five large-catalog tasks. Post-training a 27B model also improves performance on held-out product types.

Critical strengths
  • Matched controls preserve the decision problem while varying presentation, and purchase outcomes are programmatically scored. The large degradation across five model families, plus increased purchases of the same promoted products, convincingly demonstrates steering in this setting, and that the problem is real and imoprtant.

  • Single-mechanism and targeted ablations move beyond aggregate failure rates. The prompting baseline, harness-component ablations, and held-out product-type evaluation make the mitigation results more informative than an unstructured prompt-only comparison.

  • I think it's also an interesting failure mode that previous works have not explored.

Critical weaknesses
  • The verification tool responsible for most of the Hard-set improvement is not specified at the implementation level. Appendix H describes its purpose and the agent’s coverage protocol, but not the tool’s full input schema or how it checks submitted coverage/evidence and approves a candidate.

  • Mitigation evidence is preliminary, as the authors acknowledge: Hard has five tasks, post-training transfer uses 16 non-laptop Shop tasks, and the cross-environment check has four Stay tasks. These results demonstrate promising interventions but establish their generality less strongly than the broader benchmark findings.

Questions for authors
  • The paper mentions Fara, but I don't see it in Figure 3. Also, it would be better to evaluate more model families, like at least one from Anthropic (Given that they are often good at CUA tasks, and they are usually safer compared to other providers. For example, they are shown to be less "goal-directed" compared to other model families in Shayegani et al. 2026, "Just Do It!? Computer-Use Agents Exhibit Blind Goal-Directedness"). There are also specifically trained open CUA models out there, like UI-TARS, OpenCUA, etc., that are worth being evaluated.

  • The paper mentions small models, but then refers to 27B. There are much smaller models available like the ones I mentioned in the previous points, so 27B does not really count as small, and I think the claims of the paper need to be revised a bit, or actuall smaller models like 4B-7B-8B need to be tested.

  • Please specify how the verification tool checks candidate evidence and coverage, beyond the protocol in Appendix H. Is validation deterministic or model-based, and can an agent purchase after a failed or omitted check? Section 6.1 already states that the harness has no hidden catalog access.

  • Which tasks or trajectories informed harness design and prompt tuning? Please provide per-task mitigation outcomes and clarify the task-clustered uncertainty for the five-task Hard and four-task Stay evaluations.

  • In the preference-order experiment, the global optimum is never found. Among inspected nondominated alternatives, what user-specified rule determines the correct trade-off? How do you separate unsupported priority claims in rationales from demonstrably worse purchase choices?

  • I could not find any details about the computational resources used, especially for the training of the 27B model. Can you please specify the resources for reproduction? Also, can you provide some insights into the resources needed for smaller models of sizes 4B to 8B?

Reviewer_NYF8OpenReview ↗
Rating
4(+2)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces CAVEAT, a benchmark to investigate how much an incentive-misaligned environment can influence decisions of computer use agents in online marketplace environments. CAVEAT therefore does not consider explicit attacks but the impact of an ordinarily working environment that favors certain products. Across nine self-hosted market places inspired from real platforms eight steering mechanisms used in commercial practice such as sponsored placement and preferential ranking are applied. Each task has both hard requirements (such as less than $1000) and comparative requirements that require the agent to search through the displayed results to find the unambiguous optimal product. Every evaluation is run twice, once in the incentive-misaligned environment with steering activated and once as a clean control with the steering deactivated. The paper finds a significant reduction in the purchase rate of the optimal product, dropping to zero on a large-scale shopping suite with 2,112 products. Evaluations are extensive, covering several model families from open-source to frontier. The paper identifies three different failure modes: the agent stops searching too early, adds priorities the user never stated, and decides with unresolved evidence. The paper constructs a harness that addresses exactly these shortcomings. This largely restores performance, usage of the harness can be taught to small open-source models via fine-tuning. The paper is written clearly and well-understandable, figures are clear and informative. The evaluation and environments are realistic. However, some questions remain and cannot be resolved from the paper.

Critical strengths
  • Steering mechanisms are taken from real-world documented practices and each mechanism is ablated individually.
  • Evaluations are extensive: main evaluation across different platforms and models from open-source to frontier, highly-realistic full scale evaluation examples, scaling with model generations and size, and individually ablated steering mechanisms
  • The paper contributes a custom harness, optimized for tasks in web shopping/booking that require comparing of products across a large corpus by (i) checking whether the search looked at all examples by comparing visited and total listings, (ii) asking to submit full listing information, and (iii) deterministically returning the best option from the submitted list given strict constraints; the usage taught via fine-tuning to a small open-source model.
Critical weaknesses
  • CAVEAT-hard is underspecified: information about the explicit setup and how the stated results on CAVEAT-hard could be obtained are not clear (see Q1-Q3).
  • What counts as failure mixes different things: The model actually falling for one of the promoted products that violates a hard requirement (as shown in Fig. 2), the model purchasing one of the promoted products that fulfills all hard requirements but is not the optimal product, and the model abstaining from a purchase and not making a decision. E.g., all 50 CAVEAT-hard failures satisfy all hard requirements. Therefore, Fig. 2b is misleading as the chosen product might not violate the hard requirements.
  • Sorting and filtering for the relevant metrics the agent needs to compare, is not always available. While real marketplace platforms for e.g. laptops allow filtering for disk size, weight, …, on CAVEAT the agent cannot filter for the relevant metrics nor sort beyond price/rating/popularity. That means, buried under sponsored results, the model has no other choice than clicking one-by-one through the shown results.
Questions for authors
  1. CAVEAT-Hard control (90%): The paper describes the agent as seeing a screenshot + interactive elements and §5 and §6 describe the agents are browsing page by page. For CAVEAT-hard, where does the optimal product sit under control and misaligned environments? What is the default order? For e.g. laptops weight and battery appear only on the individual product pages. This means, with a step budget of 250 steps, page-by-page inspection cannot find the optimal product within hundreds of candidates that fulfill the hard requirements if the optimal product appears on later pages (potentially after page 1, as the authors write, the model often stops searching on page 1). And if the optimal product appeared on the first page during the control task, the control does not check that the task was correctly solved as not all suitable products have been inspected but just that the model stopped being lucky that it has already seen the optimal product. How can the control set be solved with 90% accuracy?
  2. In the BrowserUse environment, the agent has a JavaScript evaluate action. What fraction of CAVEAT-Hard episodes solved the task by using the JavaScript tool rather than by browsing? (+ difference between base and harness?)
  3. Using the CAVEAT-Harness performance rises to 80% on CAVEAT-hard? Given that “the harness uses only the user request and information obtained through ordinary browsing; it receives no benchmark answers, hidden catalog state, or incentive metadata“, how does the task that involves comparing hundreds of products that fulfill the hard requirements, become feasible within 250 steps? The harness asserts that the agent looks at all products, how is this feasible within the given step count? Example trajectories would be useful in general, especially within the CAVEAT-harness.
  4. Are promoted products declared as such? Are all steering mechanisms disclosed and marked as required by law? As steering mechanisms are sourced from real practices their implementation should follow legal requirements, such as labeling sponsored products.
  5. Please report the split of episodes that did not purchase the optimal product into: (i) bought a promoted product that violates the hard requirements, (ii) bought a promoted product that fulfills the hard requirements, (iii) bought a non-promoted product that fulfills the hard requirements but is suboptimal, (iv) bought a non-promoted suboptimal product that sits on the first page, and (v) bought nothing. Can the agent even abstain or articulate that it cannot search through the full multi-page database? (iv) is relevant especially for the clean control, to answer whether it was really the promotion that made the model buy that product or whether the model tends to end on early products anyways. On CAVEAT-hard, how many products are promoted, how many of the promoted products / all products fulfill the hard requirements. Taking the cheapest product in the catalog with the most sales, excellent reviews, and fulfilling every stated requirement, which appears on top of the page, might overstate the mechanistic claim that agents fall for steering.
  6. In the incentive-misaligned environment, can the agent bring the user-optimal product on the first page of results? So is it even possible for the agent to find the optimal product when it stops on page 1, just by using filters and sorting?
  7. Does the agent cheat using the CAVEAT-harness? Did the model submit a number of looked-at products equal total products without having looked at everything? How is ensured, the model is not guessing the hidden information from the listing site?
Reviewer_tjFkOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates the user-alignment of browser agents in online marketplace environments when the environment contains various mechanisms (8 kinds) that promote items that are not optimal for the user's goal. Given the same harness, five model families all exhibit sharp drop in success rate at finding the user-optimal item. The paper investigates the impact of each of the 8 kinds of environment steering mechanisms, and point out three failure modes when agents get misdirected by the environment. The paper proposes a simple yet effective harness that can raise success rate for closed models, and show that post-training is somewhat effective for Qwen3.5-27B.

Critical strengths

The problem, benchmark, investigation, and results are all novel and timely, as computer/browser agents get increasing adoption by consumers. The experimental setup and results are rigorous and clearly written, including the the taxonomy of common marketplace steering mechanisms, per-mechanism impact on failure rate, the agent failure mode diagnosis, effect of model reasoning/size/generation. The two harness methods of structured task specification and providing a verification tool are simple yet effective.

Critical weaknesses

The mitigation methods (structured task specs and verification tool) worked well on GPT-5.6 Terra and Sol, but not well on Qwen3.5. This disparity deserves explanation, which is currently lacking. It's odd that Qwen failed to abide by the task specs or use the verification tool, when it can get 69% on the base task already.

Questions for authors

Could you choose a more precise term than "robustness"? Robustness is a general term, e.g. robustness against environment stochasticity (popups) or nonstationarity. You should not claim the entire word purely for your purpose of investigating agent success against marketplace environment steering. Could you make "incentive misalignment" (line 124) more precise? For benchmark design, being able to evaluate the severity of environment steering would be helpful. E.g. something like the overlap between optimal set for objective V and the optimal set for U? Could you make the conceptual difference between incentive-misalignment and adversarial attack more precise? Suppose a shopping site is really designed to make agents buy the highest-price product. Surely that counts as adversarial against an agent acting on behalf of an average buyer who usually purchases the lowest price item.

Comments on the paper

1 comment

Dear AC and Reviewers,

We thank all reviewers for their thoughtful and constructive engagement during the discussion. All three reviewers now recommend acceptance: Reviewer NYF8 with a score of 4, and Reviewers tjFk and f3V9 with 3.

We are encouraged that the reviews recognize the following strengths.

  • A new and timely problem. Reviewer tjFk finds the work "novel and timely". Reviewer f3V9 finds the problem real and important and the failure mode one that "previous works have not explored."
  • Realistic environments and a clean causal design. Reviewer NYF8 finds the environments realistic, with steering mechanisms drawn from documented practice. Reviewer f3V9 notes that the matched controls "convincingly demonstrate steering."
  • Extensive, rigorous evaluation and effective mitigation. Reviewers NYF8 and tjFk find the evaluation extensive and rigorous, and Reviewer tjFk calls the harness "simple yet effective."
  • Clear presentation. Reviewers tjFk and NYF8 find the paper clear and easy to follow.

We have uploaded a revised PDF with all changes highlighted in blue.

  1. More models. We added Claude Opus 5, plus one more dedicated CUA models UI-TARS-1.5-7B. The finding holds across all of them.
  2. A failure decomposition, showing that steering redirects purchases to promoted products.
  3. A fuller CAVEAT-Hard specification, covering default ordering, search controls, evaluate usage and search effort.
  4. An implementation-level specification of the verification tool.
  5. Per-task mitigation results, with exact task-level tests and compute details.
  6. Sharper concepts and scope. This covers incentive robustness, misalignment severity, the distinction from adversarial attacks, the role of position, and scoped claims for smaller models.

CAVEAT gives the community a controlled way to measure whether agents stay faithful to users when the environment has its own incentives, and shows that the resulting failures can be diagnosed and substantially reduced. We thank the reviewers and the AC again for their time and engagement. Their careful comments have made the paper stronger.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.