acceptodds
Under review as a conference paper at ICLR 2027

The Pro-Worker AI Benchmark: Measuring Whether LLM Assistance Preserves the User's Cognitive Role

10 views

Abstract

Capability benchmarks do not record whether a model's interaction pattern keeps the user's cognitive role. We introduce the Pro-Worker AI Benchmark (PWB), which draws on the human-AI complementarity literature to define eleven dimensions of interaction behavior, scored on 320 prompts (200 single-turn probes in thirteen professional domains, 16 five-turn scenarios of 80 turns and 40 adversarial stress tests) by a three-judge LLM panel. Four dimensions fall below quadratic-weighted among the judges and are excluded from the composite Pro-Worker Index (PWI); three blind human annotators also agree far less among themselves on those four (Krippendorff against ), and their consensus agrees with the panel median at . Across seven open-weight models from six families the default is answer-first: PWI ranges from 20.4 to 37.8 out of 100, baseline cognitive forcing is zero for five of the seven, and GPT-6 Sol and Gemini 3.1 Pro share the default. A system prompt that describes the scored behaviors raises the index to 50.9-82.8 ( = +25.2 to +53.0; paired over the composite's 120 prompts, ), which measures instructability: on three models a generic prompt of matched length, which instructs answer-first, adds +3.2 points, while a prompt written from a lay brief recovers 39% of the gain and a checklist of the rubric's surface features 68%. A responder that only asks questions beats every baseline, so we add PWI-D, which credits asking first only when a conversation with a simulated user delivers the requested output by the second turn: on six models PWI-D keeps 87-96% of the gain, and on three it drops the question-only responder to 3-8 points. The effect persists in five-turn scenarios ( = 1.61), under adversarial pressure where declining is defensible ( = 1.54), and under a judge from outside the evaluated families. The checks also find costs: on requests the prompt itself says to answer directly, prompted models ask first 47% of the time, and the judged calibration dimensions reward hedging language while the confidence models state on exact-answer questions does not move. PWB measures interaction form and does not establish effects on worker skill or decisions; we specify the study that would. All prompts, rubrics, judge logs and code are released.

Primary area:
datasets and benchmarks
Keywords:
large language models, benchmark, evaluation, human-AI interaction, pro-worker AI, cognitive forcing, deskilling, LLM-as-judge, interaction quality

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_5vnbOpenReview ↗
Rating
1(−3)
Confidence
2
Informativeness
2
Presentation
2(−1)
Soundness
1(−1)
Read review
Paper summary

The paper measures whether an AI assistant keeps the user thinking or hands over the answer. Eleven behaviors are scored, such as asking the user’s own view before advising, handing a decision back to the user, and correcting a wrong premise instead of building on it. Three LLM judges from three model families score each behavior from 0 to 3 on 320 prompts: 200 single-turn professional requests, 16 five-turn conversations, and 40 prompts where the user pushes the model to skip the questions. Four behaviors are dropped from the composite score because neither the judges nor a blind human panel agree on them.

Seven open-weight models from six families run twice: with no system prompt, and with a system prompt describing the wanted behaviors. With no system prompt, every model answers outright and almost never asks the user first; two proprietary models behave the same way. With the system prompt, every model’s composite score roughly doubles. A generic system prompt of the same length adds almost nothing, and the gain holds on held-out prompts, in multi-turn and adversarial settings, and under a judge from outside the evaluated families. The paper also measures the costs of the prompt: a responder that only asks questions beats every baseline, so a second score credits asking only when the model delivers the answer once the user replies; prompted models ask first on half the requests the prompt told them to answer directly; accuracy on calculation questions drops.

Overall important problem with a good benchmark and actionable result

Critical strengths
  1. The best result is that every model tested stops answering outright and starts asking the user first when a system prompt tells it to. This holds on all seven open-weight models from six families, and a generic prompt of the same length adds almost nothing (3.2 points). HumanAgencyBench (Sturgeon et al., arXiv 2025) had already shown that models rarely ask first by default, but could not say whether that is a training choice or a limit of the models. Reversal on every family says training choice, which is the premise the pro-worker AI argument depends on.

  2. I appreciated that the paper tests its own score harder than most benchmark papers do. They build a responder that only asks questions, show it beats every real model, and add a delivery check that drops it to 3 to 8 points. They measure how often prompted models ask when the user wanted an answer. They drop four behaviors because humans could not agree on them either.

Critical weaknesses
  1. “PWB measures interaction form and does not establish effects on worker skill or decisions.” Then what does a high score tell someone deciding whether to use this prompt? The paper’s own results show the prompt that raises the score also makes the model worse in two ways. When the user clearly wants an answer, the prompted model asks a question instead 47% of the time. On calculation questions, it gets the answer right 6.7 points less often, because it holds the answer back. Making the user think may be worth those costs, but the paper cannot say, because no user is ever measured. The title and the name “pro-worker” promise that unmeasured outcome. Either narrow the title and abstract to what is measured, how the model behaves, or run the study already described in future work: give real users the model with and without the prompt and test what they remember and how often they accept a wrong answer.

  2. “Cognitive forcing and complementarity account for 42% (Qwen3.5) to 97% (Gemma) of each model’s ∆PWI.” Almost the whole result comes from two behaviors: asking the user first, and handing the decision back. Were those two checked against human judgment? The judges agree worst on asking first when there is no prompt (κ = 0.41), and the human study has no asking-first items. The human study has “two controls,” but the paper does not say which behaviors. If handing back the decision was not one, then the two behaviors that drive the result were scored by LLM judges only, and the eleven-behavior list in the abstract describes a bigger instrument than the one behind the numbers. Name the controls and add a human check on asking-first items.

Questions for authors

Every scored prompt is one where asking the user first is the right move, so a model that asks on every request gets full credit. Your 30 direct-answer prompts show prompted models ask first on half of them when they should have answered, but those prompts do not count toward the score. Why not include them, so asking when the user wanted an answer costs points?

Reviewer_CMFZOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces a benchmark (PWB) of whether an LLM's interaction pattern preserves the user's cognitive role. It defines eleven dimensions drawn from HCI work, e.g. cognitive forcing, contrastive explanation, complementarity and anti-sycophancy. It has 320 prompts in three layers: 200 single-turn probes, 16 five-turn scenarios and 40 adversarial prompts. A panel of three open-weight LLM Tudges scores responses 0–3, and the median is used. Four dimensions with judge κ<0.70\kappa < 0.70 are excluded. The composite Pro-Worker Index (PWI) weights the remaining six dimensions.

Critical strengths

This paper has No Strengths.

Critical weaknesses

This entire paper is generational AI Slop, it feels like the authors didnt even read what the LLM wrote. Its so bad in presentation and legibility that no one can understand whats going on.

At once instance in future work, the author writes "the checks point to the benchmark’s next composition, which prompt_checks/NEXT_VERSION.md", wow. The authors should have also shared there Skill.md so we know which skill to not use for this sort of paper writing.

The AI use statement literally has 5 paragraphs, at one instance it writes "A consequence worth stating plainly.". The way the negations are written, it obviously shouts claude. and then the authors also say "Generative AI was used throughout this work", yeah obviously.

In the experiments:

  1. The headline result is close to circular, and validity is not shown. In the prompted condition the system prompt "describes each target behavior, with worked examples, in terms close to the rubric anchors" and overlaps five test items. The rubric score then rises. A checklist of the rubric's surface features recovers 68% of the gain. This measures instruction-following toward the rubric, not a property of the models. Nothing links a higher PWI to what workers learn, how they rely on AI, or what they decide. The authors say that the transfer from un-bypassable interface interventions to ignorable chat questions is assumed. The rubric also scores a direct answer to a clearly posed question as 0 on cognitive forcing.
Questions for authors
  1. What evidence links PWI or PWI-D to any user outcome? A small user study comparing a high- and a low-PWI-D configuration of one model on learning or reliance would address the main validity gap.
  2. Without the rubric-derived prompt, the baseline spread across models is 17.4 points. Are the baseline rankings stable across single judges and alternative weights? The paper reports τ\tau as low as 0.52 across single judges at baseline.
Reviewer_AtYEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper addresses a broader question in the field as to whether AIs are helping humans or replacing their thinking. Specifically, the paper asks whether particular patterns of human-AI interaction preserve the human's cognitive role, rather than directly measuring which ones improve or diminish human cognition. The two terms used for this are: Substitutional AI and Augmentative AI assistance.

For this, they build an evaluation framework: Pro-Worker AI Benchmark (PWB). This framework contains 11 dimensions drawn from prior work, from which only 6 are reliable enough when judged by 3 LLMs. These together compose the main score Pro-Worker Index (PWI).

320 simulated professional requests were generated with Claude (reviewed and revised by the authors) based on the earlier 11 dimensions. The main set of single-turn requests span 13 professional domains. 7 open-weight LLMs were asked to respond to these requests, normally and with a special 'pro-worker' system prompt. Their responses were evaluated by 3 LLM judges.

Some insights were that models generally answer first rather than engaging the user's thinking. System prompts can significantly change this behavior, but then can lead to the other extreme where it asks endless questions (scoring high on the PWI benchmark), without getting any useful work done. To balance the effect, they create PWI-D (delivery gated), which rewards asking first only if the requested output is delivered by the second turn.

Overall, the paper shows that LLMs can be steered towards engaging the user's thinking through system prompts, and proposes a way to avoid rewarding question-only behavior while checking whether the requested task is delivered. However, it does not establish whether these interaction patterns actually improve human cognition or work outcomes.

Critical strengths
  1. Formalization. An attempt at formalizing the concept of cognitive offloading vs. cognitive sharpening. This is a question that bugs the field right now, but we have few established ways of tracking it. This paper provides some preliminary insights, like the heavy answer-first tendency of LLMs (which is not surprising given preference tuning toward helpfulness, but still important to surface).
  2. Experiment sequence. Decent methodological sequence, even if the execution leaves some important gaps.
Critical weaknesses
  1. Judges are LLMs, data is generated by LLMs. This is a big problem because while LLMs can approximate many aspects of human behavior, this kind of analysis risks creating a bounded evaluation loop, where errors and assumptions propagate between the stages. While there are measures such as diversity of LLMs being used, and a small human validation study, they do not fully address the lack of grounding in real professional workflows and human outcomes. Much of the evaluation remains self-contained.
  2. Limited rationale behind which LLMs are used where and why. For example, why these particular models for generating requests, responding, and judging? Why are some open-weight and others closed? Some choices are explained through model-family diversity and availability, but the overall selection methodology is not well justified.
  3. Hacks to work around limitations of LLMs, more than core insights. Most importantly, these seem to be more like prompting and evaluation tricks, rather than really looking at the human cognitive mechanisms involved, experimenting with them, evaluating them, and so on. The paper draws on prior cognitive and human-AI interaction research, but does not test whether the measured behaviors actually improve human cognition or develop a mechanistic explanation connecting them to those outcomes.
Questions for authors
  1. Instead of the story being around benchmarks, I think the framing should be around setting up and validating evaluation criteria. A benchmark seems premature for a concept whose measurement is still being established. Benchmarks are useful when there are many competing claims, approaches, and optimization methods to compare. Here, sure, there are noisy claims, but what seems more needed is a scientific exploration into the underlying issues and causes, rather than primarily benchmarking behaviors.
  2. The target behaviors are the source of simulated data, fair. But how are the domains for the 320 requests selected? The paper lists 13 professional domains for the single-turn tasks, but does not give much rationale for why these domains, their proportions, or the tasks within them are representative. They seem somewhat arbitrary.
  3. What are some architectural improvements (motivated by theory) that are needed beyond just a system prompt? For example, I could imagine active inference theory (from neuroscience) being of significance here, because it provides a framework for maintaining and updating a model of the user and the task, and reducing uncertainty through interaction. This could naturally give rise to discussions around the boundaries of how many questions to ask, when they are useful, and when to just provide an answer, instead of relying on fixed prompting rules.

Comments on the paper

4 comments
~Sivenfuu1OpenReview ↗

ai slop

Did the authors even read their own paper before submitting it?

Section 7 literally contains prompt_checks/NEXT_VERSION.md in the main text. The paper reads more like an unfinished AI-generated draft than a carefully written and reviewed academic submission.

The fact that such an obviously unpolished manuscript received a score of 4 (Clear Acceptance) is genuinely astonishing.

I am not claiming that a Markdown file path alone proves AI authorship. But when a paper leaves this kind of material in its main text, it raises serious questions about how carefully the authors reviewed their own work.

This is exactly the kind of low-effort, AI-slop-like academic writing that should receive much closer scrutiny, not a Clear Acceptance.

Maybe the AI that gave clear acceptance is the same AI that left out prompt_checks/NEXT_VERSION.md

time to doubt the credibility of top-tier conferences from now

I gave this paper a high score initially and have now lowered it.

I want to explain why, and to apologize to the other reviewers and the AC. I had a heavy reviewing load alongside multiple concurrent deadlines and read the paper too quickly. On that read I took it as bringing established theory from the human-AI interaction literature into a formal benchmark with human validation behind it. A careful read obviously does not support that, and I should not have submitted a score on the basis of a skim.

I also agree with the other comments that this is a heavily AI-written paper. There are multiple MD file references throughout on top of section 7. The supplement also talks about PLAN.md a couple of times. And the tell-tale signs of AI writing e.g. packing sentences with a ton of statistics, over precision where it's not needed, formally defining something that doesn't need to be defined.

Again, apologize for the confusion and initial score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.