acceptodds
Under review as a conference paper at ICLR 2027

Auditing Accuracy Change in Human–AI Decision-Making: Partial Identification and Behavioral Constraints

Abstract

Research on human–AI decision-making aims to help people make more accurate decisions, but how much improvement after AI advice can existing studies establish? The data reported in these studies may include AI recommendations and final human decisions but may not determine this change when people's independent judgments are missing. We thus provide an analytical lens for evaluating how information and assumptions about human behavior constrain the possible accuracy changes. When people either retain their initial judgment or adopt AI advice, these records leave a range of possible accuracy changes whose width equals the final human–AI agreement rate and cannot be narrowed by collecting more data of the same kind. We examine two ways to narrow these bounds: sparse measurement of initial judgments and behavioral restrictions based on patterns or models. Evaluations on six human-AI decision tasks show that, behavioral patterns narrow the bounds, while the additional contribution of behavioral models varies across tasks. The relative precision of direct measurement and behavioral constraints depends on the task and measurement budget. Using this lens, we audit 94 claim–workflow instances from 56 studies and find sufficient evidence for the conclusions in 80 instances; 14 require additional information or assumptions for stronger interpretations. We provide implications for evaluation and interpretation in human–AI decision-making.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Human-AI decision-making, Partial identification, Evaluation and auditing, Behavioral constraints

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_gsjaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks how much a human-AI study can say about the "accuracy change" (final accuracy minus pre-advice accuracy, Eq. 1) when pre-advice judgments are not recorded. It assumes people either keep their initial answer or adopt the AI's answer. Under that "keep-or-adopt" assumption, Proposition 1 shows that the sharp identified set from final-only records is [-p-, p+]. Its width equals the final human-AI agreement rate g, and no number of extra final-only records can shrink it. The paper then studies ways to narrow this set: (i) sparse direct measurement of pre-advice correctnes, (ii) behavioral patterns, namely a known switch fraction and a bound on the agreement-disagreement gap in pre-advice accuracy and (iii) behavioral models, namely a Bayesian IRT/advice-taking model following Tejeda et al. and the activation-integration model of Vodrahalli et al. The empirical part uses six tasks from two datasets with paired judgments: five binary HAIID tasks and the 16-class Tejeda object-recognition task. Initial judgments are masked, constraints are calibrated on 50% of participants, and bound widths are compared. Finally, the authors audit 96 claim-workflow instances from the 56 studies in the Raees et al. review. They report that 80 of 94 in-scope instances are identified by recorded data or by study design, while 14 need extra information or assumptions. Across 13 papers where it can be recovered, the median final agreement rate is about 0.75, which implies a roughly 75-percentage-point baseline range.

Critical strengths
  1. Interesting Problem: The point that final-only records cannot tell "I already agreed with the AI" apart from "I switched to the AI", and that this leaves an irreducible range of width g, is stated clearly and illustrated well (Fig. 1, Sec. 2). For the HCI/human-AI community it is a useful warning, and the Eq. 2 decomposition (Delta = p+ - g q_agree) is a nice way to see where the uncertainty comes from.

  2. Partial-identification treatment: The bounds are sharp and come with proofs: switch-fraction bounds (Prop. 2, Eq. 12), bounded selection (Prop. 4), the known-kernel row-space condition for the activation-integration model (Prop. 10), and LP-based component relaxations (Prop. 11). There are also minimax lower bounds on estimation error (Prop. 1, App. D.2) and on decision regret (Cor. 5).

  3. Interesting empirical observation on prediction versus identification: Fig. 3 and App. E.3.1 show that the fitted activation-integration kernels cut Brier loss by 31-53% while shrinking bound width by only 0-10%. Better predictive models of human behavior do not automatically give tighter conclusions about the estimand. This is a good message for ML researchers who fit behavior models.

Critical weaknesses
  1. The core theoretical result is straightforward, and the theory as a whole is a standard application of Manski-style worst-case bounds: Given keep-or-adopt, Eq. 2 shows that Delta is linear in q_agree, which lies in [0,1]. Prop. 1 follows at once, and its width-g statement is just the coefficient on q_agree. Prop. 2 is a small LP over beneficial, harmful and neutral switch masses. Prop. 3 and Prop. 4 intersect that set with a band. Theorem 2 and Cor. 3 intersect it with a known accuracy range. The minimax lower bound of g/2 is the usual two-point argument. Almost all of the machinery is textbook partial identification (Manski 1989/1990; Horowitz & Manski 1995; Masten & Poirier 2018), as the paper itself notes ("The interval logic is standard", App. preamble). The paper also does not clearly set its contribution against closely related work it cites: Guo et al. (2024), Ben-Michael et al. (2025), Schemmer et al. (2023b) and Agarwal et al. (2025). Whether pre-advice agreement and reliance are confounded in final-only data is a known issue in the appropriate-reliance literature.

  2. The behavioral constraints need the very data whose absence motivates the paper: In Sec. 4.1, the switch fraction, the agreement-disagreement gap and the behavioral models are all calibrated on 50% of participants from the same study, using their revealed paired initial and final judgments. In a real final-only study, where would a researcher get alpha or the gap bound? Probably from a paired study in a different workflow. But then the transport problem the paper itself raises applies: eliciting a pre-advice judgment may change behavior (Prop. 14, App. B.3, C.3), and none of the experiments test this. The comparison "behavioral constraints vs. sparse direct measurement" therefore compares two kinds of within-study paired data, not a realistic final-only scenario. The paper concedes that "Repeated splits reuse people and retain the original tasks and items; they establish neither independent replication nor new-item or cross-workflow transfer" (App. E.2). That limits how far the empirical conclusions reach.

  3. The tighter bounds often rest on constraints that do not hold: Gain inclusion is close to 100%, but joint validity of the imposed restrictions is much lower: 63% in Census, 72% in Sarcasm, 81% in Cities and Dermatology (Table 8a). It is 89% in pooled HAIID (Table 10) and in Tejeda (Sec. E.3.2). Support is valid in only 94% (HAIID) and 89% (Tejeda) of half splits (App. E.2). Zero-error fitted models are incompatible with the data in 99-100% of splits (Tables 10, 12). With the budget multiplier at 0.25, neither component is valid in any split, yet the gain is still included (Table 11b). Gain inclusion is therefore a weak check: an interval built from false assumptions can still happen to contain the true value. The headline claim that behavioral constraints "narrow the bounds" would be more convincing if the narrowing came with valid restrictions, or if coverage were analysed under misspecification.

  4. Behavioral models add little in most tasks, and the one large gain comes from a single component: With patterns already imposed, the model shrinks width by 0.01, 6.33, 1.31, 0 and 0.01 points in the five binary tasks (App. E.3.1; Table 8b). In pooled HAIID it adds nothing (22.16 to 22.16, Table 10). The large Tejeda reduction (81.07 to 19.70) comes entirely from the initial-accuracy constraint Q. Removing Q restores 81.07, while removing E or P changes nothing (Table 12a). In effect, Q supplies pre-advice accuracy information from the fitting participants' initial judgments. That is closer to direct measurement through a model than to a "behavioral mechanism" constraint, and it weakens the claimed comparison between models and direct measurement for object recognition.

  5. The direct-versus-behavioral comparison is not apples-to-apples: Direct intervals use a Hedged-WoR finite-population construction with a 95% guarantee. Behavioral intervals use a bootstrap with no finite-sample guarantee and different tail allocations (0.05/6, 0.05/8, 0.05/9) (Fig. 2 caption, App. E.4). The paper admits that "Matching participant partitions therefore does not equalize uncertainty constructions" and that the Tejeda comparison "also reflects the different calibration procedures" (App. E.3.2). The "measurement threshold" findings, for example that direct measurement wins in Art beyond 9.7% (Sec. 4.1), may be partly an artifact of these different procedures. There are other oddities too. Direct widths grow from 50% to 75% calibration in all six tasks because the evaluation set shrinks (App. E.4), and Dermatology has only 20 participants, which makes its direct widths of about 77-89 points hard to interpret.

  6. The estimand needs better motivation: The paper notes that pre-advice accuracy in a concurrent (unprompted) workflow may not even be well defined ("If an unprompted pre-advice judgment is not well defined, Delta_tar must first be specified", App. B.3). Randomized comparisons with a no-AI arm (Ben-Michael et al., 2025) already identify the effect of offering advice. The paper says the targets are complementary (Sec. 1), but it never shows a decision or scientific question where Delta, rather than the assistance effect, is what matters and is also not available from a standard randomized design. Without this, the practical stakes of the 75-point range are unclear.

  7. Venue fit and accessibility: The contribution sits mainly in HCI methodology and econometric identification, and there is little machine-learning content. The paper is very dense. The main text leans heavily on a 40+ page appendix, and many central details live there: calibration protocols, the meaning of "requested" versus measured percentages, the support allowance, and how the model enters the LP.

Questions for authors

See weaknesses

2 comments
AuthorsOpenReview ↗

Response to Reviewer gsja [1/2]: theory and positioning (W1), validity of the assumptions (W3)

Thank you for the very detailed review and for finding the problem clearly stated and the gap between prediction and identification interesting. You raised central questions about the empirical comparison, and several of our new analyses respond directly to them. Where results also appear in our responses to XZnv and GymW, we give the main point here and refer to the full numbers there.

W1. The theory is a standard application of Manski-style bounds, and the contribution is not set against closely related work.

We agree that the mathematical tools are established, and prior work recognizes that final agreement with the AI can hide different initial judgments. Our contribution is what this ambiguity allows researchers to conclude about accuracy change, and which additional observations or assumptions strengthen those conclusions. Using an established tool as an analytical lens in this way follows recent ICLR work on human–AI decisions and on bounding ML evaluations (see the general response). Relative to the cited work:

  • Guo et al. (2024) and Schemmer et al. (2023b) define reliance and its appropriateness using recorded initial decisions, and Guo et al.'s reliance is exactly our switch fraction. We study what can still be concluded when the initial decision is not recorded.
  • Ben-Michael et al. (2025) estimate the effect of offering AI by randomizing who receives it. Our target is the within-person change, from AI-assisted logs alone (see W6).
  • Agarwal et al. (2025) study final accuracy as a function of the AI signal, which final records alone determine. This fits our classification: final-performance targets can be read off the logs, while the change from initial to final accuracy cannot.

W3. The tighter bounds often rest on restrictions that do not hold.

Most of these failures come from one assumption, the allowance for departures from keep-or-adopt, rather than from the behavioral constraints themselves. We checked each assumption separately in the paper's repeated 50% participant splits:

Taskdeparture allowance holdsbehavioral constraints holdall hold jointly
Art95%97%92%
Census69%89%63%
Cities88%92%81%
Dermatology95%84%81%
Sarcasm74%95%72%
Object recog.92%96%89%

The allowance fails most often in Census and Sarcasm because departures there are rare (0.15% and 0.39% of trials) and come from a few people, so a bootstrap over participants often misses them. The splits where assumptions fail do not drive the narrowing: mean width over splits where all hold is equal to or slightly larger than the overall mean.

Does the narrowing survive when the assumptions hold? We reran the 50% splits with a conservative finite-sample allowance. The summary-level constraints (switch fraction and gap) and the allowance then hold jointly in 94–97% of splits. This check does not cover the full behavioral models. The behavioral bounds remain much narrower than the final-only range (Art: 49 vs. 99 pp for final decisions alone under the same conservative allowance), but become 5–19 pp wider and no longer show whether accuracy improved or declined. Direct measurement is almost unaffected (Art: 6.5 pp, direction determined in all splits). This is itself a finding: the narrowing from behavioral patterns depends on how departures from keep-or-adopt are bounded, whereas narrowing from initial-accuracy information does not. We therefore report each narrowing together with the assumptions it relies on. We also agree that gain inclusion alone is a weak check, as your examples show. For the fitted models, exact (zero-error) versions are rejected by the paired data in almost all splits of the pooled HAIID and object-recognition analyses, as you note, so we treat model-based narrowing as conditional on the model. Once initial judgments are measured, models add at most 0.2 pp. To test robustness to misspecification directly, we report the largest departure rate each conclusion can tolerate (see XZnv, Q3).

Action: In the final version, we will report assumption validity next to gain inclusion, show results under both allowances, and describe any narrowing together with the validity of the assumptions it relies on.

AuthorsOpenReview ↗

Response to Reviewer gsja [2/2]: transfer (W2), behavioral models (W4), the comparison (W5), the target quantity (W6), fit (W7)

W2. Where would a final-only study get the switch fraction or gap bound? Transfer is not tested.

We tested transfer across HAIID tasks and from the sequential to the concurrent workflow in Tejeda et al. (2022). In these analyses, initial accuracy was more stable than behavioral patterns (the switch fraction and gap) (GymW Q2), although this is not a guarantee for new settings. The workflow comparison also bears on your concern about asking for initial judgments: final AI agreement is 4–8 pp higher in the concurrent workflow, while unaided accuracy differs little. Final agreement alone cannot show whether the underlying advice-taking process changed. Our analysis thus identifies initial accuracy as the more portable quantity and behavioral patterns as specific to the task and workflow, so conclusions that rely on transferred patterns should be reported over a range of their values.

W4. The Tejeda gain comes from Q, which is closer to direct measurement.

This reading matches our ablation. In object recognition, the additional narrowing comes entirely from Q, which carries initial-accuracy information from the participants used to fit the model. As you observed, removing E or P leaves the width unchanged. Section 4.1 now attributes that gain to this accuracy information, rather than to recovering an advice-taking process from final decisions.

W5. The comparison is not apples-to-apples.

We agree. With matched participants, departure allowance and calibration procedure, direct measurement is narrower in all six tasks at the budgets meeting the 90% gain-inclusion check in XZnv Q1. The earlier "measurement threshold" findings came from the different calibration procedures. We therefore treat direct measurement as the reference and ask how close each behavioral source gets to it (general response): behavioral patterns close much of the distance, and their information overlaps with direct measurement once it is available. Both strategies are over-confident at the smallest budgets, so we treat those results as descriptive. Direct widths increase from 50% to 75% calibration because the evaluation sample shrinks, and Dermatology's wide intervals reflect its 20 participants.

W6. Why study the change from initial to final accuracy rather than only the effect of providing AI assistance?

A randomized no-AI condition is the cleanest design when available. Our analysis matters in three cases.

  1. Records without initial judgments. In the audit, 32 of 96 claim–workflow instances come from such workflows, and deployed systems often log only final decisions.
  2. How decisions change within the AI-assisted workflow. A no-AI condition estimates the overall effect of providing assistance, but not how assisted participants revised their own initial judgments. Δ measures this net change. Separating beneficial from harmful switching needs further information. In Rastogi et al. (2022), randomization establishes the effect of time on agreement (−19 pp), but the stricter reading that time reduces harmful switching holds only if extra time raises unaided accuracy on probe trials by less than 19 pp (see XZnv, Q2).
  3. A no-AI condition as additional information. Its unaided accuracy can constrain initial accuracy in the assisted workflow, provided the two conditions are comparable, and then enters the bounds directly.

On whether this change is meaningful in a concurrent workflow: in the Tejeda data, the sequential initial-to-final change and the concurrent randomized effect of AI are numerically close (+5.4 vs. +6.2, +11.0 vs. +9.6, +16.1 vs. +17.2 pp). They measure different quantities, and relating them requires assuming that unaided accuracy carries over across workflows and that blocks do not affect each other.

W7. Venue fit and accessibility.

Human–AI decision-making is an established topic at ICLR, ICML and NeurIPS (see the general response). Two of our findings speak directly to ML researchers who evaluate human–AI systems: better predictions of final decisions need not give a more informative evaluation of accuracy change, and initial human accuracy was more stable than behavioral patterns in the settings examined. To make the paper easier to follow, we have moved the calibration procedure, measurement budgets and relaxed keep-or-adopt analysis into the main text, and will shorten material not needed to assess the claims in the final version.

Thank you again for the careful review. We hope the new analyses address your concerns, and we would be grateful if you would take them into account in your final assessment. We welcome further discussion of the assumption-validity results.

Reviewer_GymWOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether the change in human decision accuracy induced by AI advice can be identified when researchers observe only the ground truth, the AI recommendation, and the human's final decision, but not the human's pre-advice judgment. Under a keep-or-adopt restriction, the paper derives a sharp identified set for the pre-to-final accuracy change and shows that its width is exactly the final human–AI agreement rate. The paper then studies how sparse direct measurements of pre-advice correctness, aggregate behavioral restrictions, and explicit behavioral models can shrink this identified set. Empirically, the framework is evaluated through mask-and-reveal experiments on six human–AI decision tasks and through a literature audit covering 94 in-scope claim–workflow instances.

Critical strengths
  1. The paper gives a clean sharp characterization for the accuracy-change estimand. Under the keep-or-adopt restriction, the identified set takes a simple form and its width is exactly the human–AI agreement rate. This turns the vague statement that “missing initial judgments create ambiguity” into a precise characterization of which values of the estimand remain compatible with the observed data.
  2. The paper usefully reframes identification as an information-design problem. By expressing the remaining uncertainty through latent correctness among agreement cases, the paper places sparse direct measurement, behavioral restrictions, and explicit models in a common framework. This makes it possible to compare what different sources of auxiliary information actually contribute to identification.
  3. The distinction between predictive performance and identification power is particularly informative. The empirical results show that substantially improving prediction of observed behavior does not necessarily translate into substantially tighter identified sets. This is an important reminder that fitting observable outcomes well is not the same as ruling out latent worlds relevant to the scientific estimand.
  4. The mask-and-reveal experiments and literature audit make the problem concrete. In all six empirical tasks, the true pre-to-final accuracy change is positive, yet its sign cannot be determined from final-only observations alone. The audit further illustrates that operational statistics such as agreement are sometimes interpreted as stronger latent behavioral mechanisms without sufficient identifying information.
Critical weaknesses

1. The paper's conceptual novelty overlaps substantially with prior work and should be positioned more narrowly.

Tejeda et al. (2022) already identify the fundamental ambiguity created by missing initial judgments in concurrent human–AI decision-making and propose latent behavioral modeling for this setting. Guo et al. (2024) further formalize the distinction between observed following behavior and stronger notions of AI reliance. Ben-Michael et al. (2025) already use partial identification in human–AI evaluation, although for a different estimand. The present paper therefore should not frame either the missing-initial-judgment ambiguity or partial identification in this domain as broadly new. Its novelty lies more specifically in the sharp characterization of the pre-to-final accuracy-change estimand and the unified analysis of auxiliary information.

2. The main theorem is elegant but relatively direct once the behavioral restriction is imposed.

Under the keep-or-adopt assumption, disagreement cases effectively reveal the pre-advice judgment, while ambiguity remains only in agreement cases. This makes the identified set analytically clean, but also limits the depth of the theoretical development. Given that the underlying observational ambiguity is already known, the paper should explain more clearly why this sharp characterization constitutes a sufficiently substantial methodological advance.

3. The intended practical role of Section 3 needs clearer positioning.

Without credible auxiliary information, the latent accuracy change remains only partially identified, so Section 3 does not recover the target estimand from final-only observations alone. Its main value appears to be in study design and sensitivity analysis, clarifying what additional measurements or assumptions are needed to tighten the identified set. This role should be stated more explicitly.

4. The empirical comparison of auxiliary-information strategies leaves important questions about cost and transportability.

Direct measurement and behavioral methods are not strictly cost-matched, and the empirical evaluation mainly calibrates and evaluates within the same study. In realistic settings, behavioral constraints may need to transfer across populations, interfaces, tasks, or systems, but this transportability is not established here.

5. The keep-or-adopt restriction is central but may be difficult to verify precisely in the workflows where it is most needed.

The main theorem assumes that the final human decision is either the initial human judgment or the AI recommendation. The paper provides a relaxed-support analysis, which is valuable, and the multiclass task shows non-negligible violations of the strict restriction. However, when pre-advice judgments are not collected, the violation rate itself is generally unobservable in the target workflow. This makes the robustness extension more central than a conventional appendix-level sensitivity check.

Questions for authors
  1. Relative to prior work by Tejeda et al. (2022), Guo et al. (2024), and Ben-Michael et al. (2025), what do the authors view as the paper's main methodological novelty? Is the primary contribution the sharp identification of the pre-to-final accuracy-change estimand and the unified analysis of additional information regimes?
    • Tejeda, H. A., et al. (2022). AI-Assisted Decision-making: a Cognitive Modeling Approach to Infer Latent Reliance Strategies.
    • Guo, et al. (2024). A Decision Theoretic Framework for Measuring AI Reliance.
    • Ben-Michael, E., et al. (2025). Does AI Help Humans Make Better Decisions? A Statistical Evaluation Framework for Experimental and Observational Studies.
  2. In practical applications, how should behavioral constraints or model parameters be transported from a calibration setting to a different target population, task, interface, or workflow? Can the authors provide any cross-workflow validation?
  3. Why do substantial improvements in behavioral prediction sometimes lead to only small gains in identification? Which aspects of the behavioral model are actually informative about the latent states relevant to the target estimand?
  4. Given the observed violations of the strict keep-or-adopt assumption, how should practitioners assess whether the assumption is credible when pre-advice judgments are unavailable? Can the relaxed-support analysis be made a more central part of the main paper?
2 comments
AuthorsOpenReview ↗

Response to Reviewer GymW [1/2]: novelty and positioning (Q1, W1, W2)

Thank you for the thoughtful review and for finding the identification result "clean" and the distinction between prediction and identification "particularly informative". Your questions help us state the contribution more precisely: what additional evidence is needed to evaluate whether AI advice changed human accuracy, and why better prediction of human behavior does not necessarily give that evidence. We have organized the revision around these two points.

Q1. "Relative to prior work by Tejeda et al. (2022), Guo et al. (2024), and Ben-Michael et al. (2025), what do the authors view as the paper's main methodological novelty? Is the primary contribution the sharp identification of the pre-to-final accuracy-change estimand and the unified analysis of additional information regimes?"

Yes. Our main contribution is to characterize what final-only human–AI decision logs can establish about the change from initial to final accuracy, and how additional measurements or behavioral assumptions change that conclusion. The tightest possible bounds make the remaining uncertainty explicit and let us compare these sources of information in one framework. We build on prior work on missing initial judgments and on partial identification. Relative to the three cited papers:

  • Tejeda et al. (2022) recognize the ambiguity in concurrent workflows and resolve it with a parametric cognitive model. We ask what can be established without such a model, and what each assumption or measurement adds. Our ablation also shows what such a model contributes: its narrowing in object recognition comes entirely from its initial-accuracy component Q, and removing the other components changes nothing.
  • Guo et al. (2024) define reliance as the probability of following the AI when the person's own recommendation differs. This is exactly our switch fraction, and measuring it requires the initial recommendation to be recorded. Proposition 2 gives the tightest bounds on accuracy change for a known reliance level, showing what their reliance measure implies about accuracy change when the initial recommendation is missing.
  • Ben-Michael et al. (2025) estimate the effect of offering AI by randomizing who receives it. Our target is the within-person change from initial to final accuracy, using logs from the AI-assisted condition only. The two targets are complementary (see our response to gsja, W6).

W2. The main theorem is relatively direct once keep-or-adopt is imposed.

The derivation is short, but it has two useful implications for evaluation. First, the width of the range equals the final AI-agreement rate, which can be computed whenever AI recommendations and final human decisions are logged. Among 13 published papers where we could reconstruct this rate exactly, the median is 0.75, so accuracy change could lie anywhere in a 75-point range. Second, a lower bound of half this width (g/2) shows that collecting more final-only records cannot remove this uncertainty. These results also tell researchers where additional evidence must help: all remaining uncertainty lies in trials that end in agreement with the AI.

Action: We have revised the introduction and Related Work with this comparison, which positions the contribution more narrowly relative to prior work.

AuthorsOpenReview ↗

Response to Reviewer GymW [2/2]: transfer, prediction vs. evaluation, and keep-or-adopt (Q2–Q4, W3–W5)

Q2. "How should behavioral constraints or model parameters be transported …? Can the authors provide any cross-workflow validation?"

In our new analyses, initial human accuracy was much more stable across settings than behavioral patterns (the switch fraction and gap).

  • Across tasks. We calibrated on the other HAIID tasks and evaluated each held-out task using only final decisions. The switch fraction varies widely (0.16–0.56 across tasks and 0.28–0.53 across Tejeda's AI-quality conditions), while initial accuracy stays at 0.66–0.72 (0.49–0.50 across conditions). Using initial accuracy from other tasks with a conservative rule gives intervals of about 13 pp that contain the true gain in all five tasks and show its direction in two. A narrower pooled rule misses the gain in two tasks.
  • Across workflows. In the concurrent-workflow data of Tejeda et al. (2022), people see the AI's advice from the start, so initial judgments are not recorded, but randomized AI-off blocks give a reference for unaided accuracy. Final decisions alone leave the direction of change undetermined in every condition. Initial accuracy from the sequential study is within 2.4 pp of this reference. Assuming unaided accuracy carries over across workflows and blocks do not affect each other, the transferred bounds contain the reference and show an improvement in three of four samples. Final AI agreement is 4–8 pp higher in the concurrent workflow (consistent with Fogliato et al., 2022), so behavioral patterns should not be assumed to transfer unchanged.

This identifies initial accuracy as the more portable quantity: it can come from a paired study, a no-AI condition or a small measured subset. Behavioral patterns are specific to the task and workflow, so conclusions that rely on transferred patterns should be reported over a range of their values.

W4. The strategies are not cost-matched.

We now compare them at the same number of measured people. At budgets meeting the 90% inclusion check, direct measurement is narrower in all six tasks (XZnv, Q1), so we use it as the reference (general response). We also give a simple rule for how many randomly chosen trials to measure in order to determine the direction of change with 80% probability: about (z₀.₉₇₅ + z₀.₈)²·q(1−q)/Δ². In our tasks this is about 160–870 trials (Appendix F). Measuring whole participants requires more, and these numbers do not account for changes in behavior caused by asking.

Q3. "Why do substantial improvements in behavioral prediction sometimes lead to only small gains in identification?"

A behavioral model can predict final decisions accurately without revealing what people would have decided before seeing the AI's advice. Yet accuracy change depends on initial correctness on trials that end in agreement with the AI (Eq. 2). Better prediction therefore narrows the range only when the model credibly constrains those missing initial judgments. In the Bayesian model, the extra narrowing comes from the initial-accuracy component Q, not the advice-taking components E and P. In the activation–integration model, a known response model can rule out some initial responses, and Proposition 10 gives a sufficient condition under which it determines the change exactly. Across the binary tasks, fitted models reduced Brier loss by 31–53% but interval width by only 0–10%, and they cannot be learned from final decisions alone.

Q4 / W5. Assessing keep-or-adopt without initial judgments.

We have moved the relaxed keep-or-adopt analysis into the main text (Theorem 1 in Section 2 and Table 1 in Section 4.1). We suggest three steps for practitioners. First, report the largest departure rate a conclusion can tolerate (η*, tabulated in our response to XZnv, Q3), so readers can judge whether departures that frequent are plausible. Second, use paired studies with a similar response format as a reference (below 2% in our binary tasks and 13% in the 16-class task). Third, record initial judgments for a small subset, which estimates the departure rate together with initial accuracy.

W3. The role of Section 3.

We now state in the first paragraph of Section 3 that its methods cannot pin down accuracy change from final decisions alone. Each adds a measurement or an assumption, which can inform study design (what to measure, and how much) or support a sensitivity analysis.

Thank you for highlighting the information-design perspective. We hope the revision addresses your questions, and we would be grateful if you would take it into account in your final assessment. We welcome further questions about the contribution and the limits of transfer.

Reviewer_XZnvOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper asks what human–AI decision studies can conclude about the change from pre-advice to final accuracy, Δ\Delta, when pre-advice judgments are not recorded. Under a retain-or-adopt assumption, final-only data identify only [−p−,p+][-p_-, p_+], an interval whose width equals the final human–AI agreement rate gg; more final-only data cannot shrink it. Three sources narrow this set: sparse direct measurement of pre-advice correctness, behavioral constraints (a switch fraction and a bound on the accuracy gap between agreement and disagreement trials), and behavioral models (a Bayesian cognitive model and an activation–integration model). The empirical part masks the pre-advice judgments in six tasks from two public studies and compares bound widths against the measurement budget. A literature audit codes 96 claim–workflow instances from 56 studies. Of the 94 in scope, 80 are supported by recorded data or study design, and 14 need more information or assumptions.

Critical strengths
  1. Correct identification analysis with a usable design rule. Without pre-advice judgments, the accuracy-change range has width equal to final agreement. The median agreement reconstructed from 13 published papers is 0.75, i.e. a 75-point range. The sharp bounds for the switch fraction and the agreement–disagreement gap, together with the sparse-measurement result, tell a study designer what to record. The validation against masked ground truth is the right test: every direct interval contains the evaluation-sample Δ\Delta.
Critical weaknesses
  1. Limited novelty and a possibly unfair main comparison.

    • Novelty and fit. Proposition 1 is a direct application of Manski-style worst-case bounds to a two-outcome missing-data problem. Propositions 2–3 intersect that set with linear constraints. The paper has no learning method and no representation question, so its natural audience is HCI or econometrics, not ICLR.
    • The budget tradeoff may be biased. The empirical conclusion that behavioral constraints win at small measurement budgets may be partly an artifact. Section 4.1 says the direct intervals at a 1% budget "remain 100 percentage points wide". That is wider than the final-only identified set (e.g. 85.2 points in Art), which any valid procedure can intersect with. Section 3.1 says the direct bounds "can be combined with the bounds above". It is unclear whether this was done in Figure 2. The two arms also use different calibration (finite-population for direct, bootstrap for behavioral; Figure 2 caption).
    • Scope and impact. All data come from two public studies, and the constraints are calibrated on other participants of the same study. The audit finds 80 of 94 claims supported, and it does not show a single published conclusion that changes under the bounds.
  2. Presentation is padded and carries leftovers from an automated revision pass.

    • Agent changelog. The Reproducibility Statement reads like a changelog from a coding agent: "The direct-measurement refresh preserves the original result files and participant partitions; run direct refresh.py produces the new intervals and verification records, and build direct refresh outputs.py generates the updated manuscript tables." The appendix also reports "refreshed direct widths" and a "refreshed direct branch" with no definition of what was refreshed or why. The AI-use statement says Codex helped implement and run the analyses.
    • Unedited errors. Section 3.2 says "Appendix A.2 gives the proof and Appendix A.2 gives an example." The abstract has a stray comma ("show that, behavioral patterns narrow the bounds"). The abstract and introduction say 94 instances were audited, Section 4.2 says 96, and the reader has to work out that 2 are out of scope.
    • Vague framing. "Analytical lens" appears seven times in the main text for what is a partial-identification analysis. The Related Work section is one paragraph that opens with filler ("Human–AI decision-making is an important research area aimed at improving decision quality").
    • Figures and appendix. Figure 2 packs six panels with axis text too small to read in print. The paper is 53 pages: a 9-page main text (about 5,700 words) against about 32,000 words in total, with 20 tables and 10 formal results. Undefined terms such as "requested 1% of participants" appear in the main text, and the audit coding protocol, the retain-or-adopt checks and the inter-coder agreement are only in the appendix.
Questions for authors
  1. Are the direct-measurement intervals in Figure 2 intersected with the final-only identified set [−p−,p+][-p_-, p_+]? If not, please redo the budget comparison with the intersection and with the same calibration procedure for both arms. Does "behavioral constraints win at small budgets" still hold?
  2. Among the 14 flagged audit instances, does any published conclusion change sign or become unsupported under your bounds? Please give one worked example.
  3. In the six tasks, what fraction of trials violates retain-or-adopt (final answer is neither the initial answer nor the AI's)? How do the bounds change when this assumption is relaxed?
  4. What is the "direct-measurement refresh"? Which numbers changed relative to the earlier version, and why?
3 comments
AuthorsOpenReview ↗

Response to Reviewer XZnv [1/3]: the budget comparison (Q1, Q4)

Thank you for the careful review and for recognizing the identification analysis as correct, with "a usable design rule". Your Q1 identified a real problem in our budget comparison, and Q4 asks about its cause, so we answer these first. Q2 and Q3 follow in [2/3], and the critical weaknesses and presentation points in [3/3].

Q1. "Are the direct-measurement intervals in Figure 2 intersected with the final-only identified set? … Does 'behavioral constraints win at small budgets' still hold?"

The reported small-budget advantage of behavioral constraints does not survive a matched comparison, and we have removed that claim. We first address the intersection, then present the new comparison.

Where the 100 pp came from. The intersection was done, but the final-only range it used was almost uninformative. For the direct-measurement condition, the allowance for departures from keep-or-adopt was set by a conservative bound that, with only two measured people, allowed almost every trial to depart from keep-or-adopt. With the allowance used for the behavioral constraints, the direct interval at the smallest budget equals the final-only range (85.1 pp in Art). Under strict keep-or-adopt, it equals the range in Proposition 1 (83.9 pp). With a matched allowance it is therefore never wider. Appendix E.4 keeps the conservative version and now explains its 100 pp.

Matched comparison. We reran Figure 2 with the same measured participants, the same allowance and the same calibration procedure for both strategies. We report widths only at budgets where both contain the evaluation-sample accuracy change in at least 90% of repeated splits (an empirical check rather than a formal coverage guarantee). At every such budget, in all six tasks, direct measurement gives narrower intervals. Mean width in percentage points (pp), at the smallest such budget / at 50% of participants:

Taskpeoplefinal-onlyswitch + gapdirectdirect + switch + gap
Art13 / 1398555 / 3115 / 6.714 / 6.7
Census8 / 219368 / 6413 / 1012 / 10
Cities15 / 398754 / 5215 / 1214 / 12
Dermatology4 / 108024 / 2417 / 1714 / 15
Sarcasm7 / 738545 / 3018 / 9.317 / 9.3
Object recog.21 / 369680 / 8112 / 1112 / 11

What behavioral information adds after direct measurement. Behavioral patterns and measured initial judgments carry overlapping information. Adding behavioral constraints to direct measurement narrows the intervals by at most about 1.5 pp at these budgets (3 pp in Dermatology, where at most 10 people are measured). The extra narrowing is larger, up to about 5 pp, when only two people per task are measured, but those intervals are not yet reliable: the direct intervals contain the evaluation-sample accuracy change in about 50–75% of splits, and the combined intervals in about 30–55%. We report those points as descriptive, and use the finite-sample intervals in Q4 when a guaranteed coverage level is needed. Adding a full fitted behavioral model on top changes the width by at most a further 0.2 pp at these budgets. The overlap arises because behavioral patterns narrow the bounds through initial accuracy on agreement trials, which direct measurement also supplies. We therefore use direct measurement as the reference and report how close each behavioral source gets to it (see the general response).

Action: We have replaced Figure 2 with this comparison, marked the budgets with low gain inclusion, and removed the small-budget claim (Section 4.1). Section 4.1 and the conclusion now use direct measurement as the reference. Without initial judgments, behavioral patterns close 35–90% of the distance to it in the HAIID tasks, and the sources that come closest carry information about initial accuracy.

Q4. "What is the 'direct-measurement refresh'? Which numbers changed relative to the earlier version, and why?"

"Refresh" means that we recomputed all direct-measurement intervals with a finite-sample confidence interval for sampling without replacement (Waudby-Smith & Ramdas, 2024) instead of the bootstrap. We apologize for leaving this term unexplained. We chose this interval because the bootstrap can be over-confident when few participants are measured, whereas this interval guarantees 95% coverage. Only the direct-measurement numbers changed, and they became wider. As you point out, the behavioral constraints still used the bootstrap, and this difference produced the apparent crossover in Q1.

Action: We have rewritten the Reproducibility Statement in plain terms, described the calibration in Section 4.1 and Appendix F, and removed terms such as "refresh".

AuthorsOpenReview ↗

Response to Reviewer XZnv [2/3]: departures from keep-or-adopt (Q3) and a worked audit example (Q2)

Q3. "In the six tasks, what fraction of trials violates retain-or-adopt …? How do the bounds change when this assumption is relaxed?"

Departures from keep-or-adopt (final answers that are neither the initial judgment nor the AI's) are rare in the five binary tasks (0.15–1.73% of trials) and more common in the 16-class task (13.3%), where people can switch to one of the remaining labels. To show how much our conclusions depend on the assumption, we computed for each task η*, the largest departure rate at which the evidence still shows whether accuracy improved or declined.

Taskobserved departurestolerated rate η*width at 0 / 1% / 5% departures (pp)
Art0.28%1.67%8.0 / 14.2 / 36.1
Census0.15%0.44%4.9 / 17.2 / 20.2
Cities0.29%1.07%9.8 / 16.6 / 33.7
Dermatology1.73%0.93%2.4 / 6.5 / 15.9
Sarcasm0.39%1.08%4.4 / 10.5 / 23.5
Object recog.13.3%2.84%16.3 / 20.3 / 36.4

In four binary tasks, the conclusions hold at departure rates about 3–6 times the observed ones. In Dermatology and object recognition, the observed rate exceeds η*, which explains why the direction of change is not determined there. The last column also shows that the width grows quickly with the allowed departure rate, so this analysis belongs in the main text.

Action: We have moved the relaxed keep-or-adopt result into Section 2 (Theorem 1) and added this table to Section 4.1 (Table 1).

Q2. "Among the 14 flagged audit instances, does any published conclusion change sign or become unsupported under your bounds? Please give one worked example."

No published conclusion changes direction. The 14 instances fall into two groups. In 12, the reported statistic (such as agreement with the AI) is supported by the data, but a stronger behavioral interpretation of it (anchoring, over-reliance, appropriate trust) is not. In the other 2, the target is defined through a behavioral model and holds only under that model. The audit therefore shows which interpretations need more information or assumptions, not that the findings are wrong. Rastogi et al. (2022), from the first group, illustrates this.

The study. In Experiment 1, time per trial (10 or 25 s) is randomized, and on probe trials the AI is always wrong. Initial judgments are not recorded, and there is no matched human-alone condition. Disagreement with the AI on probe trials rises from 48% at 10 s to 67% at 25 s, which the authors interpret as evidence that more time reduces anchoring.

What the experimental design establishes. Because time is randomized, its effect on final agreement (−19 pp) is established. This finding, and the paper's broader conclusion that time affects final decisions, stand.

What final decisions do not establish. To examine a stricter behavioral reading of this de-anchoring effect, we consider harmful switching: being initially correct and then switching to the wrong AI answer. Its share is agreement minus the share of people who were initially wrong anyway: a = A − (1 − q), where q is unaided accuracy on probe trials. Since q is not recorded, this share can be anywhere in [0, 52%] at 10 s and in [0, 33%] at 25 s. Its change therefore lies in [−52, +33] pp, so whether harmful switching fell is unknown.

The extra condition. The change in harmful switching equals −19 pp plus the change in q. The stricter interpretation therefore holds if and only if extra time raises unaided accuracy on probe trials by less than 19 pp. Recording initial judgments on some probe trials, or adding a human-alone condition at each time level, would check this.

Action: We have added this example to Section 4.2, next to the 12 + 2 split.

AuthorsOpenReview ↗

Response to Reviewer XZnv [3/3]: critical weaknesses and presentation

We respond to the remaining points in the order of your review.

Limited novelty and fit. Proposition 1 uses standard partial-identification tools, which bound what data can establish under stated assumptions, and prior work has recognized the ambiguity created by missing initial judgments. The revised introduction now states that the tool is established, and Related Work credits Tejeda et al. (2022) with identifying the ambiguity. Like recent ICLR work that brings established tools into human–AI decision-making (e.g., value of information in Guo et al., 2026) or uses bounds to evaluate ML models (Frauen et al., 2024; Melnychuk et al., 2024), we use this tool as an analytical lens, and the contribution lies in what it reveals. Specifically, we give the tightest possible range of accuracy change from final-only logs, whose width equals the final AI-agreement rate. We then compare direct measurement and behavioral assumptions in one framework, including reliance as defined by Guo et al. (2024), and show what each source adds and how well it transfers. Finally, we apply the framework in a claim-level literature audit. The relevance to ML lies in evaluating AI-assisted decision systems and in what learned behavioral models can establish from incomplete decision records. These questions fall under topics listed in the ICLR call for papers, including uncertainty quantification, societal considerations, and applications to cognitive science. Our results show that better prediction of observed human behavior does not necessarily provide stronger evidence about accuracy gains. What matters is information about initial accuracy. This distinction matters when ML researchers evaluate human–AI systems or use behavioral models to interpret how people respond to AI advice.

The budget tradeoff may be biased. The original comparison used different calibration procedures. We now report a matched comparison in Q1, explain the source of the earlier crossover in Q4, and replace the small-budget claim with the reference-based comparison in the general response.

Scope and impact. We examined transfer across HAIID tasks and from the sequential to the concurrent workflow in Tejeda et al. (2022). In these settings, initial accuracy was more stable than the switch fraction and gap, although the transferred ranges did not always show the direction of change (GymW Q2). The audit likewise does not overturn published numerical findings. Instead, the Rastogi et al. example shows which stronger interpretation requires additional information (Q2).

Presentation is padded and carries leftovers from an automated revision pass. Thank you for pointing these out. We have checked the full manuscript and corrected such text. Specifically, we have:

  • Agent changelog: rewritten the Reproducibility Statement and removed "refresh" and similar internal terms (see Q4).
  • Unedited errors: fixed the duplicated reference to Appendix A.2 and the stray comma in the abstract, and used one count throughout (96 coded instances, 94 in scope), explained at first mention.
  • Vague framing: defined "analytical lens" in the introduction (using partial identification to determine what a study's records and stated assumptions can establish about accuracy change), and revised Related Work with a clearer comparison to Tejeda et al. (2022), Guo et al. (2024) and Ben-Michael et al. (2025).
  • Figures and appendix: redesigned Figure 2 around the matched comparison, stated the participant budgets in the main text as numbers of measured participants, and added the inter-coder agreement (89 of 95 headline classifications after one joint exclusion, Cohen's κ = 0.80) to Section 4.2. In the final version, we will also shorten the appendix by removing material that does not support the main claims.

Thank you for identifying the comparison and presentation issues. We hope the revised comparison and the new analyses address your concerns, and we would be grateful if you would take them into account in your final assessment. We welcome any further questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers: new analyses and revised findings

We thank the reviewers for their detailed and constructive feedback. The reviewers found the identification analysis correct and clean, with a usable design rule (XZnv, GymW), and the gap between predicting behavior and identifying accuracy change informative (GymW, gsja). Their main concerns were the fit to ICLR, the fairness of our measurement comparison, the validity and transfer of behavioral assumptions, and departures from keep-or-adopt. We address fit first, then report new analyses on the other concerns. The revised paper marks all changes in blue.

Fit to ICLR. Our paper follows an established pattern in ML research: a mature tool from another field is applied to an ML question as an analytical lens, and the contribution lies in what it reveals. Examples include item response theory for LLM benchmarks (Maia Polo et al., 2024), Shapley values for data valuation (Ghorbani & Zou, 2019) and statistical frameworks for assessing fairness when outcome labels are selectively observed (Coston et al., 2021). Our question is what decision logs and behavioral models can establish about whether AI advice improves human accuracy when initial judgments are missing. It belongs to the human–AI decision-making work at ICLR, ICML and NeurIPS (e.g., Mozannar & Sontag, ICML 2020; Chen et al., ICLR 2024; Guo et al., ICLR 2026).

Direct measurement as the reference. Under matched calibration, measuring initial judgments directly gives narrower intervals than behavioral patterns or models in all six tasks, at every budget where both contain the evaluation-sample accuracy change in at least 90% of repeated splits (XZnv Q1; gsja W5). This threshold is an empirical check rather than a coverage guarantee. The earlier small-budget advantage of behavioral constraints came from different calibration procedures, and we have removed it. We therefore use direct measurement as the reference and ask how close each other source of information gets when initial judgments cannot be collected. Mean width in pp at 50% of participants:

Taskfinal-only+ behavioral patterns+ behavioral modelinitial accuracy from other tasks*direct measurement
Art8531319.66.7
Census93645814.910.4
Cities87525012.012.1
Dermatology80242418.217.3
Sarcasm85303012.09.3
Object recog.96812012–1811.4

*From the other tasks of the same study, with a conservative rule (GymW Q2).

What the comparison shows. Four results stand out. Whether the assumptions behind these widths hold is reported in XZnv Q3 and gsja W3.

  1. Without any initial judgments, behavioral patterns (the switch fraction and the agreement–disagreement gap) close much of the distance from final-only records to direct measurement in the five HAIID tasks (35–90%) but less in object recognition (17%).
  2. Behavioral models narrow the bounds further when they encode initial accuracy. In object recognition, the model closes 90% of this distance through its initial-accuracy component. In the HAIID tasks, model intervals stay close to pattern intervals.
  3. Initial accuracy transferred from other tasks of the same study comes nearly as close as direct measurement, assuming the tasks are comparable (within 0–5 pp in the HAIID tasks and 1–7 pp in object recognition).
  4. Behavioral patterns and measured initial judgments carry overlapping information: once initial judgments are measured, adding behavioral patterns narrows the intervals by at most about 1.5 pp at these budgets (3 pp in Dermatology), and full models by at most a further 0.2 pp.

Main finding. Across the tasks and information sources we examined, the most useful information for evaluating whether AI advice improves human accuracy is about people's initial accuracy, whether measured, transferred from similar tasks, or carried by a behavioral model. Behavioral patterns supply part of this information when initial judgments are missing, so their information overlaps with measured initial judgments. Better prediction of final decisions strengthens the evidence only if it adds such information.

Main changes in the revised paper (in blue).

  • Section 4.1 and Figure 2 present the matched comparison, with direct measurement as the reference. The small-budget claim is removed.
  • Theorem 1 (Section 2) and Table 1 (Section 4.1) cover departures from keep-or-adopt (XZnv Q3).
  • Section 4.2 adds a worked audit example (XZnv Q2). The introduction and Related Work position the paper more precisely against prior work.
  • Appendix F reports the matched calibration, transfer across tasks and workflows, and the departure analysis. The supplementary code covers all new analyses.

The individual responses give the details. We would be glad to continue the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.