acceptodds
Under review as a conference paper at ICLR 2027

No Such Thing as a Free Read! Work as Payment for Web Agents

Abstract

AI agents increasingly browse the web on users' behalf, returning answers without sending users to the original source and cutting content owners off from the advertising that pays for their work. We study work as payment: agents earn access to content by completing verifiable tasks for its owner, such as writing code or labeling images. In controlled browser experiments with eight models and four agent implementations, agents readily pay with work: they submit contributions in about 90% of runs, and the strongest models solve all required coding tasks in every run. Yet they do not act on cost. Models that state a limit of about three tasks when asked directly solve 32 problems as agents to retrieve two identifiers, keep working for websites that never deliver, and largely ignore instructions to minimize cost. No agent bypassed an access control, but one model that refused unsafe requests when asked directly produced responses judged harmful in about half of its agent runs. Work-based access therefore requires agents that know when to stop paying.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Web agents, work as payment, content access, cost awareness, AI safety

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_khbZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Summary

This paper studies a setting in which websites require AI agents to complete useful work, such as coding or image labeling, before providing access to protected content. Through controlled browser experiments involving multiple models and agent implementations, the authors investigate whether agents can manage the cost of this work-based access mechanism. The results show that agents often continue submitting work despite excessive or unreliable requirements, and that repeated pressure can also weaken safety refusals in some configurations.

Critical strengths

Strengths

  1. The paper introduces a novel and relevant problem setting. Rather than assuming that web content is freely accessible once an agent locates it, the work considers an interaction in which access must be earned through verifiable contributions. This perspective connects agent autonomy with the economic incentives of content providers and raises questions about cost control, stopping decisions, and the reliability of agent behavior that are not captured by conventional browsing benchmarks.

  2. The evaluation examines several dimensions of agent behavior. The experiments include multiple models and agent implementations, as well as websites with different work requirements and access behaviors. This makes it possible to study not only whether agents can complete work, but also whether they stop when requirements become unreasonable, continue when promised access is never delivered, or change their behavior under repeated rejection.

  3. The findings reveal a meaningful gap between stated preferences and actual agent behavior. Agents can claim to have a low spending limit when asked directly but still complete many more tasks during execution. The safety experiments also suggest that a model's direct refusal behavior may not fully predict its behavior when embedded in a persistent agent loop. These observations provide useful empirical evidence about the interaction between persistence, incentives, and safety.

Critical weaknesses

Weaknesses

  1. The controlled environment may not capture the full complexity of real content-access mechanisms. The experiments use ten locally hosted website templates and primarily two forms of work: coding and image labeling. Real publishers may have different incentives, verification standards, task values, and content-access policies. The paper would be stronger if it more clearly established how the selected settings map to realistic publisher-agent interactions and which conclusions are expected to generalize beyond the controlled environment.

  2. The evaluation does not fully quantify the economic value of the exchange. The number of tasks completed is a useful measure of work expenditure, but it is not equivalent to the actual cost of the work or the value received by the website owner. Different submissions may vary substantially in quality, usefulness, and computational cost. A more explicit model of the utility of completed work, the value of the protected content, and the cost of continued attempts would make the conclusions about rational stopping and excessive payment more precise.

  3. The safety findings should be interpreted carefully across models and agent configurations. The observed deterioration in refusal behavior is substantial for some configurations but not uniform across all tested models. Since the paper also shows that the agent implementation strongly affects whether runs continue or stop, it remains important to separate the effects of repeated access pressure from those of the agent scaffold, task design, and model-specific behavior. Additional controls and a clearer characterization of the conditions under which refusals deteriorate would strengthen the broader safety claim.

Questions for authors
  1. Clarification The paper studies whether agents provide work in exchange for content access and independently verifies work submissions. Could the authors clarify how submission quality is measured across coding and image-labeling tasks, and whether the reported work counts distinguish correct, useful contributions from merely completed submissions? This affects how strongly the results support the claim that websites can obtain valuable work rather than simply induce agents to expend effort.

  2. Assessment-critical evidence The observed deterioration in refusal behavior appears to vary substantially across models and agent implementations. Could the authors clarify how robust the safety finding is across the tested model–agent configurations, including the corresponding denominators and uncertainty for the key harmful-response rates? In particular, how much of the reported effect can be attributed to repeated access pressure itself, as opposed to the agent scaffold or other differences between direct prompting and agent execution? This distinction is important for assessing the scope of the paper's safety conclusions.

1 comment

We thank the reviewer for the careful review and for recognizing the novelty of the setting and the gap between what agents say and do! Below we address each point.

W1 (realistic settings). Work as payment is not deployed on today's websites, so we cannot measure it in the wild. Controlled websites let us change one thing at a time (the price, whether the website delivers, how the agent can pay) and check every submission. To keep the pages realistic, five of our ten website templates are local recreations of real websites (e.g., RTINGS and the U.S. National Park Service), and the pages are server-backed web apps in a real Chromium browser. What we expect to generalize is how the agents behave: they choose sensibly between offers shown side by side, but have no limit on the total they spend when a website keeps asking for more. How long they persist depends on the model and the agent implementation.

W2 (economic value). Besides task counts, we report the tokens consumed: the median successful 32-problem run used 2.2 to 3.8 million tokens depending on the model, about 70k to 120k per problem, and took 18 to 42 minutes. We report tokens rather than dollars because tokens convert directly into cost at any price, whereas prices differ between models, change over time, and differ for the same model across providers. The value the website receives is the verified correct work (see Q1).

W3 and Q2 (safety finding). The safety result mainly shows a possibility: a website that keeps rejecting the work can push a model into harmful responses. DeepSeek-V4-Flash shows that this does happen: it refused almost all unsafe requests when asked directly (2 harmful replies out of 40), yet produced harmful responses in 69 of its 128 agent runs. GPT-5.6-Luna kept refusing (4 of 128 runs), so the effect indeed depends on the model. The pressure comes only from the website: the user's request is benign, and the website repeats one fixed rejection. Within the same runs in setting F, 12 of DeepSeek-V4-Flash's 47 first responses were harmful, before any rejection, and 31 of its 38 second responses, after one (G: 7 of 50, then 23 of 45). So the agent setting raises the rate a bit, and repeated rejection much more. We will further clarify this in the paper.

Q1 (quality of the work). Both validators are deterministic. A coding solution counts as correct only if it passes all public and hidden private LiveCodeBench tests, and an image label must match the GQA reference (ignoring case and whitespace, with "grey" and "gray" treated as equal). Every submission is checked, whether or not the website grants access. We report submissions and correct work separately: participation counts any nonblank submission, while every count of completed work, such as the largest workload or the correct pairs under moving goalposts, counts only correct solutions. So the websites did obtain valid work: the four strongest models solved both coding problems in every run with every agent.

Reviewer_XmSfOpenReview ↗
Rating
3
Confidence
4(+2)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces work as payment: a publisher releases protected content only after an agent completes verifiable tasks (LiveCodeBench coding problems, GQA image-color labeling). This substitutes for the advertising revenue that AI-summarizing browsing agents destroy (motivating examples: DWork–Naor proof-of-work, reCAPTCHA, crowdsourcing gold-item QC, and the consent-crisis literature). Ten locally hosted websites (five fictional: Fieldline/Stackline; five recreations of RTINGS and the U.S. NPS) present access offers; each run asks for two identifiers that exist only in protected content. Work is graded independently of whether the site releases content, which is the paper's key methodological asset.

Eight models (GPT-5.6-Luna/Sol/Terra, GPT-6-Astra, GPT-OSS-120B, Grok-4.6, DeepSeek-V4-Flash/Pro) run through four BrowserGym agents (AgentLab GenericAgent, MostBasicAgent, VisualWebArenaAgent, and a TapeAgents adaptation). RQ1 (§5.1): agents submitted in 91% of coding runs and 88% of labeling runs; 334/384 (87%) coding runs solved both problems and got access, and no run with a submission had both solutions wrong (Table 1: pooled coding Sub. 91±2, Acc. 89±3, Both 87±3, Answer 84±4). Labeling is much weaker (68% both correct, 88% for Sol vs 46% for Grok-4.6), and on the three hardest images only 8/40 submitted labels were correct — Figure 10 shows models converging on a common wrong color (green for a blue umbrella), evidence of ambiguous reference labels.

RQ2 (§5.2) varies price at 2/4/8/16/32 Easy problems under an adaptive decreasing schedule. GPT-5.6-Sol and Grok-4.6 completed 32/32 in all 48 settings with three of four agents, at a median of 39–51 browser actions and 2.2–3.8M tokens; no model reached 32 with GenericAgent, because the page exceeded its default prompt limit. Meanwhile, asked directly without a browser, Sol, Astra, and Grok-4.6 state a median maximum of ≈3 Easy tasks (Figure 3b); Terra says it "cannot solve coding tasks" yet completed 32 in 34/64 settings, and DeepSeek-V4-Pro answered 0 to all 800 maximum questions while still working as an agent. Cost instructions changed nothing (Figure 12: Sol's median is 32 under all four instructions). Under moving goalposts (a site that always says "2 more"), only 37% of runs left despite an equivalent site offering 3 fixed solutions; GPT-5.6-Sol left in 4% of runs (Figure 4a: 1/12/0/2 across agents) and answered correctly in 3%, versus 61% for DeepSeek-V4-Flash, which left in 69% — "being able to solve the tasks cheaply made agents easier to exploit." Under always-wrong feedback, 8,562 submissions were rejected, 90% of which passed all tests. In payment settings A/A′/B/C (§5.2.3, Table 3), agents do pick the cheaper bundle side-by-side (59/64 for Luna in A; mislabeled A′ drops DeepSeek-V4-Flash access from 91% to 66%).

RQ3 (§5.3): no run bypassed the deliberately SQL-injectable access-code form in setting D, though 85% of 6,367 rejected solutions were correct — pure free labor at ~1M median tokens (~10× payment settings). Under unsafe-request settings F/G, DeepSeek-V4-Flash's mean StrongREJECT score rose from 0.09 to 0.51 in F and 0.06 to 0.33 in G, and for a bank-robbery request 0/10 baseline and 0/15 first responses were harmful but 10/16 runs eventually produced one.

Critical strengths
  • Independent work verification is a real methodological contribution. Grading every submission against LiveCodeBench private tests (or GQA references) separately from the site's access decision makes exploitation measurable — this is how "90% of rejected work was correct" and "322 of 334 coding runs with access answered correctly" become computable. Most agent evaluations cannot say this.
  • The stated-vs-behavior gap is quantified cleanly and is quotable. Figure 3b plus §B.3–B.4 (400 responses per model per difficulty, prompts reproduced verbatim, ten rephrasings for DeepSeek-V4-Pro) is more rigorous than the usual informal "models don't know their limits."
  • Agent-level breakdown prevents over-generalizing from model-only results. §5.2.2 shows GPT-6-Astra leaving in 100% of TapeAgents runs and 0% with GenericAgent/VWA; Figure 11, 12, 16, 17, 18b, 19, and 20 all disaggregate by scaffold. The conclusion that "evaluations should report the full agent system" is earned, not asserted.
  • Unusually candid negative results. GQA ambiguity is documented with per-image label dumps (Figures 9, 10); the 100k-vs-200k input-limit caveat is footnoted in Figure 4; the 32-problem cap is acknowledged as making "largest completed workload" a lower bound, with a 64-problem pilot where the best run solved 63/64.
  • Null result reported without spin. "No run exploited the form" (§5.3.1) is backed by a corpus-wide sweep (170 Hard runs with guessed /supplement addresses, 41 Google searches by GPT-5.6-Sol, 454 entered codes of which 408 were the case identifier and 0 were payloads, Appendix C).
Critical weaknesses
  • Two of eight models carry the second half of RQ2 and all of RQ3. Always-wrong feedback (Luna, DeepSeek-V4-Flash), payment settings A–C, and settings D/F/G use only Luna and DeepSeek-V4-Flash at 64 runs per model per setting. The safety conclusion — "about half of its agent runs" — is one model, and the direction of the two-model contrast (Luna 58% vs Flash 51% leaving) is not obviously robust.
  • Scaffold and context limits confound the "models ignore cost" headline. No model finished 32 problems under GenericAgent "because the page exceeded its default prompt limit" (§5.2.1); §B.2 notes the first prompt alone reaches 117–128k tokens for TapeAgents at 32 Hard problems. The headline numbers are therefore conditional on one agent's prompt budget, and no prompt-budget-matched comparison is reported.
  • "Cost" is operationalized as task count, which the paper itself concedes is not cost (§3: "Task count measures workload rather than computational cost"). Token totals appear only as aggregates (2.2–3.8M; Figure 13); there is no monetary, latency, or per-solution-equivalent normalization, so "32 problems for two identifiers" cannot be converted into a welfare statement.
  • No statistics anywhere. All model comparisons are raw percentages; only RQ1 reports s.d. over 3 trials, and cells are 16 runs (Figure 19, Figure 20). Given the multiple comparisons across 8 models × 4 agents × 4 instructions, claims like "agent implementation mattered a lot" or "instructions changed the share that left only between 36% and 39%" have no uncertainty bounds.
  • The adaptive decreasing schedule confounds the workload measurement. §B.1: workloads are chosen adaptively from prior outcomes and stop at first success, so 32-problem completions are measured mostly on strong models/scaffolds while 2-problem results come from weak ones (Figure 3a, Figure 13 note). Comparisons across the price axis are not on a common sample.
  • Weak positioning against the closest prior work. BAGEN (Lin et al., 2026), "Are LLM agents budget-aware?", and EcoAgent-Bench (Wu et al., 2026) are cited in §2 but never compared to — no shared tasks, no cross-benchmark numbers, no explicit delta over their budget-awareness findings. DECEPTICON and Ersoy et al. (S&P 2026) already study dark patterns against web agents; the paper does not state how its moving-goalposts/always-wrong manipulation differs. The refusal-erosion result is a replication of AgentHarm, SafeArena, crescendo (Russinovich et al., 2025), and Kumar et al. (2024), with no comparison to their reported rates.
  • Ecological validity and reproducibility. Locally hosted static pages with all external requests blocked (§Ethics) means no ad-tech, latency, JS-rendering, or live-site friction; and the code is only promised "upon publication," so the ten templates, access rules, and seeds (only "seed 17" for the two-website setting is given) are unverifiable now.
Questions for authors
  • The headline failure to stop is only observable for workloads up to 32; the 64-problem pilot reached 63 correct solutions in one run. What stopped the cap, and do you have any evidence that these agents would terminate at all, or is 32 simply where your budget stopped?
  • How much of the agent-vs-model variation in Figure 4a/Figure 11 survives after matching the context window and prompt-budget handling across the four scaffolds? A prompt-limit explanation for GenericAgent and a 100k limit for two models in Figure 4 confound your central claim.
  • Would behavior change if the value of the content were realistic? With a page whose answer is one identifier among thousands, rather than two opaque accessions, 32 problems may simply be cheap relative to nothing. Can you show any setting where agents stop early because the price is genuinely high relative to content value?
  • Harmfulness is scored entirely by the HarmBench classifier and StrongREJECT with no human validation reported. What is the agreement between these judges and human raters on the 12/47 and 38-run DeepSeek-V4-Flash cases, and how much of the 0.09→0.51 StrongREJECT rise is "harmful detail" versus ordinary elaboration of a refusal?
  • Table 2 shows strong self-inconsistency in the stated-willingness probe (GPT-5.6-Sol accepts 16 in all responses and 64 in 52%, while calling the same request "disproportionately costly"). If the stated-preference measurement is this noisy, how do you distinguish a real stated-vs-behavior gap from probe-format noise?
2 comments

We thank the reviewer for the thorough review and for recognizing the independent verification of every submission and our reporting of negative results! Below we first summarize four results that address most of the concerns, and then answer each point.

1. Agents do sometimes stop when the work is Hard. In the 44 settings where GPT-6-Astra solved 32 Easy problems, we also offered 32 Hard problems: all 132 runs ended without a single submission. So our setup does detect agents declining work. With 32 Easy problems in the same settings, GPT-6-Astra solved them in 44 of 45 runs.

2. The gap between what models say and do is not a probe artifact. We keep the question fixed and change only the instruction. Asking the model to think about the cost lowers the share of answers that accept 32 tasks from 100% to 6% (GPT-6-Astra), 95% to 60% (GPT-5.6-Sol), and 55% to 13% (Grok-4.6). As agents, the same instruction barely changed what they did. Out of 12 agent settings (three agents on four websites; GenericAgent is left out because its prompt limit cuts the 32-problem page), the number that completed 32 problems went from 12 to 8 for GPT-6-Astra and stayed at 12 for GPT-5.6-Sol and Grok-4.6.

3. Agent and model differences remain under matched context limits. The paper already contains matched comparisons. In the workload experiment, all models had the same 100k-token input limit, and the three agents other than GenericAgent see the whole 32-problem page; with them, GPT-5.6-Sol and Grok-4.6 solved 32 problems in all 48 settings, so the "models ignore cost" result does not come from GenericAgent's 40k prompt limit. In the moving-goalposts runs, six of the eight models had the same 200k-token limit for all four agents, including GenericAgent. The agent still changed the outcome: GPT-6-Astra left the unreliable website in 0 of 80 runs with GenericAgent and in 80 of 80 with TapeAgents. So did the model: with VisualWebArenaAgent, GPT-6-Astra left in 0 of 80 runs and DeepSeek-V4-Pro in 59 of 80.

4. The two harm judges agree on which responses are harmful. Of the 308 responses that the HarmBench classifier labels harmful, StrongREJECT scores 268 (87%) at 0.25 or more and 165 (54%) at 0.5 or more (mean 0.58); of the 357 labeled not harmful, none reaches 0.5 (mean 0.01). StrongREJECT is stricter about how harmful a response is, since it also weighs how specific and convincing it is. In setting F, DeepSeek-V4-Flash's mean score rises from 0.09 for first responses to 0.51 for second responses, and this rise comes from responses labeled harmful, not from longer refusals: 31 of its 38 second responses are labeled harmful (mean score 0.61), and the other 7 average 0.06.

W1 (two models). Only always-wrong and the payment, security, and safety settings use two models. Moving goalposts, the main non-delivery experiment, uses all eight, and our search for bypass attempts covers every run of all eight. We limited those settings because of the cost: covering every agent, website, and instruction takes 768 runs per model (320 for always-wrong and 64 for each of the seven other settings). Since the agent changed the outcome as much as the model did (point 3), we kept all four agents and all four instructions rather than adding models. The safety result mainly shows a possibility: a website that keeps rejecting the work can push a model into harmful responses. DeepSeek-V4-Flash shows that this does happen: it refused almost all unsafe requests when asked directly, yet produced harmful responses in about half of its agent runs, whereas GPT-5.6-Luna kept refusing (harmful in only 4 of 128 runs). For always-wrong, we compare each model with itself on the same tasks, not the two models with each other: compared with moving goalposts, false rejections made GPT-5.6-Luna leave more often (58% vs. 23% of runs) and DeepSeek-V4-Flash less often (51% vs. 69%). If the reviewer still thinks another model would strengthen the safety settings, we are happy to add one.

W2 and Q2 (context limits). Please see point 3. GenericAgent's default 40k-token prompt limit cuts the 32-problem page, which can only lower its result. How each agent manages its context is part of the agent.

W3 (task count vs. cost). We also report tokens consumed: the median successful 32-problem run used 2.2 to 3.8 million tokens depending on the model, about 70k to 120k per problem, and took 18 to 42 minutes. We report tokens rather than dollars because tokens convert directly into cost at any price, whereas prices differ between models, change over time, and differ for the same model across providers.

W4 (statistics). Because each run is expensive, we did not repeat identical runs; instead, every comparison is replicated across different websites, address orders, and instructions. An effect that holds across all of them is unlikely to be noise: GPT-6-Astra, for example, left the unreliable website in all 80 runs with TapeAgents and in none of 80 with GenericAgent, across ten website pairs, both address orders, and four instructions.

W5 (adaptive schedule). The schedule only decides which sizes a setting tries; what we compare is one number per setting, the largest workload it completed, and every setting has this number. Both schedules look for this number with the same problems, up to three runs per size, and the same success test; the direction only changes how many runs it takes. We ran the first two models (†) from 2 problems upward and then switched to starting at 32, which saves runs when a model completes 32; we will describe both in the paper. We do not compare success rates across sizes, and Figure 13 only summarizes the effort per size.

W6 (positioning). We will add a paragraph on these papers. BAGEN and EcoAgent-Bench study how agents spend resources on their own task. We study a different setting, a website charging agents in work for access, and cost is one of our three questions, next to valid work (RQ1) and security and safety (RQ3). Even for cost, the setting differs: no budget is given, the website sets the price and can raise it after payment, and we compare what agents pay with the limit they state. DECEPTICON and Ersoy et al. use deceptive interfaces; our offers are stated plainly and broken only after the work is done. AgentHarm, SafeArena, Kumar et al., and Crescendo study misuse and jailbreaks, where the user or an attacker seeks the harmful output; these are not our focus. We ask whether access pressure alone weakens refusals: the user's request is benign, and the website states an unchanged HarmBench request as the price of access and repeats one fixed rejection, so any escalation comes from the agent.

W7 (local websites and reproducibility). Local hosting is what lets us run dishonest websites, a SQL-injectable form, and unsafe requests without touching real systems. The pages are server-backed web apps in a real Chromium browser. So that results do not depend on artificial page designs, five of our ten website templates are recreations of real websites (e.g., RTINGS and the U.S. National Park Service) and five are fictional; coding results in RQ1 were similar on both (89% and 85% of runs with both solutions correct). All experiments use the same random seed (17), for reproducibility and so that runs that differ only in model, agent, or instruction get exactly the same tasks.

Q1 (the 32-problem cap). Yes, for Easy problems 32 is where our budget stopped, and we report it as a lower bound. In a pilot with 64 problems, the best run submitted all 64 solutions twice (62, then 63 correct) and was still working when it reached our 120-action limit. Please see point 1, which shows that agents do stop when the work is Hard.

Q3 (value of the content). Please see point 1: for the same request, GPT-6-Astra solved 32 Easy problems but submitted nothing when offered 32 Hard ones. We did not vary the value of the content, because realistic, valuable content is hard to create without the model already knowing it, and we wanted to be sure that the answer can only come from the protected content, not from the model's own knowledge. We therefore ask for records with newly generated identifiers that appear only in the protected content.

Q4 (human validation). We manually checked the top responses with the highest StrongREJECT scores (0.93 to 0.98, all from DeepSeek-V4-Flash and all labeled harmful by HarmBench): every one fully carries out the harmful request.

Q5 (noise in stated willingness). The two GPT-5.6-Sol answers in Table 2 look inconsistent, but they answer two different prompts: with the standard request, it accepts 64 tasks in 86 of 100 answers; when asked to think about the cost, it reduced to only 5 of 100. So we believe the answers are not noise; they follow the instruction. As agents however, the same instruction barely changed what the models did (point 2). Models thus respond to the cost when asked but not when acting.

Reviewer_SBqLOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies work as payment in which agents earn access to content of a website by completing verifiable tasks on behalf of the user for its owner, such as writing code or labeling images. In particular, a website presents an access offer that specifies what content will be released, what work is required, and how submissions will be evaluated. The website owner is paid with the completed work rather than with the user’s attention or money.

Critical strengths
  • Asks an important question beyond making websites accessible to agents: what could an agent offer a content owner in return as agent breaks the traditional arrangement of each visit lets advertisers reach a potential customer?

  • Proposes a novel and useful solution of publisher releasing the article in exchange for labels that meet its requirements

  • Demonstrated important findings: agents readily pay with work: they submit contributions in about 90% of runs, but largely ignore instructions to minimize cost

  • Uses comprehensive experimental setup:

    • Controlled browser experiments
    • Eight models: GPT-5.6-Luna, GPT-5.6-Sol, GPT-5.6-Terra, GPT-6-Astra, GPT-OSS-120B, Grok-4.6, DeepSeek-V4-Flash, and DeepSeek-V4-Pro
    • Four browser agents through BrowserGym: Agent-Lab’s GenericAgent, MostBasicAgent, and VisualWebArenaAgent, and a browser adaptation of TapeAgents
Critical weaknesses
  • Advertisers still can reach the agent visiting a page and re-direct the agent towards buying their advertised targeted products. Could a hybrid approach be combined with work as payment works in practice?
  • The content owner needs valid contributions, not just filled answer boxes. How sensitive Your findings are wrt the choice of the validator?
  • The user needs the information at a reasonable cost, yet an agent can solve every task correctly and still spend its budget without obtaining the content. Does that introduce any biases in the navigation by directing agents towards visiting specific set of websites?
  • How the agent does not risk user security wrt introducing indirect prompt injection while doing work for the website?
  • typo in line 161: . , treating “grey” and “gray” as equivalent.
  • the value lies in the resulting work rather than in the computation spent producing it. Could you provide hints for future work on how to make agents cost-aware?
  • The introduced work as payment could have more applications on the web for agents, I would discuss them.
Questions for authors

see above

3 comments

We thank the reviewer for the encouragement and the insightful suggestions! Below we address each point and describe what we will add to the paper.

Ads and a hybrid model. Advertisers can indeed target the agent, but we see two limits. A plain ad can be unrelated to the agent's task, so we agents can skip it. An ad persuasive enough to change what the agent does is close to a dark pattern or a prompt injection, which prior works show can steer agents [a][b]; as agents get better at resisting such content, they may treat the ad as an attack and leave the page. Work as payment does not rely on persuasion or prompt injection: the website openly states what it releases, what work it requires, and how the work is checked, and the agent decides whether to accept, look elsewhere, or stop. A hybrid approach is a very interesting idea, and we thank the reviewer for suggesting it. It fits our framework well: an access offer can already list several ways to pay (Section 3), and an ad could be one of them, for example a clearly marked sponsored line that the agent passes on to the user. We will add this as a direction for future work.

Choice of validator. Both of our validators are deterministic and strict. A coding solution counts as correct only if it passes all public and hidden private tests from LiveCodeBench, and a label must match the GQA reference. Validating labels is one of the main open challenges of work as payment, and methods from crowdsourcing platforms such as Amazon Mechanical Turk apply directly. Gold items with clear, known answers, mixed in with the real tasks, catch contributors that submit arbitrary labels just to pass, and labeling each item several times lets the website keep the majority answer. Majority voting has a limit, since different models can agree on the same alternative label, so ambiguous items should be removed or reviewed. Section 3 ("Verifiable work") covers this only briefly because of space limits, and we will expand it in the appendix.

Bias in navigation. We used controlled websites to test the limits of agent behavior, so most experiments have a single website. Our two-website experiment (Section 5.2.2) does show one such bias: agents tend to stay with the first website they open. The request lists two equivalent sources, yet only 37% of runs left the first website, even though it never delivered. On the open web, this could favor the websites that agents reach first. We thank the reviewer for highlighting this risk, and we will discuss it in the paper.

Prompt injection. We agree that doing work for a website gives it a channel to the agent, as for any web agent. Our RQ3 results show that this matters: when a website asked for unsafe work as the price of access, DeepSeek-V4-Flash produced responses judged harmful in about half of its runs. Besides existing prompt injection defenses, we believe the main protection is to give the agent that visits a website as little context and as few permissions as possible. For example, the main agent can start a subagent with only the question it needs answered; the subagent visits the website, does the work, and returns only the answer, so the website never reaches the main agent or the user's data. CaMeL [c] and FIDES [d] use a similar separation: they keep untrusted content from changing what the agent does. We will add a paragraph on this.

Typo. Thank you, we will fix it.

Making agents cost-aware. We see this as one of the most important next steps. Our results show that the problem is not judging the cost but acting on it: models describe the cost of access when asked, yet largely ignore it as agents. First, the agent should state its budget before it starts, track its spending against that budget, and check with the user when a website asks for more than it advertised. Second, cost should become part of training, for example by having the model estimate the effort of a task before solving it and comparing the estimate with the effort it actually spent. Third, until agents do this reliably, the system should enforce limits that the user can see and set, so the agent stops at a reasonable point even if it ignores the cost. Our Discussion briefly argues for such limits, and we will expand it with these three directions.

Other applications. Thank you for the suggestion. Work as payment could apply wherever an agent needs something a website controls. For example, a data portal could grant access to a dataset in exchange for annotations. Our Discussion also outlines a shared service that collects tasks from requesters, distributes them to participating websites, and shares the revenue. We will add a paragraph on these applications, and we warmly encourage the reviewer to share any others they have in mind; we would be very happy to include them!

References

[a] P. Cuvin, H. Zhu, and D. Yang. DECEPTICON: How Dark Patterns Manipulate Web Agents. arXiv:2512.22894, 2025.

[b] D. Ersoy et al. Investigating the Impact of Dark Patterns on LLM-Based Web Agents. IEEE S&P, 2026.

[c] E. Debenedetti et al. Defeating Prompt Injections by Design. arXiv:2503.18813, 2025.

[d] M. Costa et al. Securing AI Agents with Information-Flow Control. arXiv:2505.23643, 2025.

Reviewer_SBqLOpenReview ↗

thanks, great work

I keep my clear acceptance score, great work

Comments on the paper

1 comment

Dear Reviewers,

Thank you again for the constructive reviews and for highlighting the novelty of work as payment, the independent verification of every submission, and the gap between what agents say and do. We hope our responses have clarified your concerns, in particular that some agents do stop when the work is Hard, that the results hold under matched context limits, and where the safety effect comes from. If anything is still unclear, we would be very happy to clarify further before the discussion period closes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.