- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper asks how much a human-AI study can say about the "accuracy change" (final accuracy minus pre-advice accuracy, Eq. 1) when pre-advice judgments are not recorded. It assumes people either keep their initial answer or adopt the AI's answer. Under that "keep-or-adopt" assumption, Proposition 1 shows that the sharp identified set from final-only records is [-p-, p+]. Its width equals the final human-AI agreement rate g, and no number of extra final-only records can shrink it. The paper then studies ways to narrow this set: (i) sparse direct measurement of pre-advice correctnes, (ii) behavioral patterns, namely a known switch fraction and a bound on the agreement-disagreement gap in pre-advice accuracy and (iii) behavioral models, namely a Bayesian IRT/advice-taking model following Tejeda et al. and the activation-integration model of Vodrahalli et al. The empirical part uses six tasks from two datasets with paired judgments: five binary HAIID tasks and the 16-class Tejeda object-recognition task. Initial judgments are masked, constraints are calibrated on 50% of participants, and bound widths are compared. Finally, the authors audit 96 claim-workflow instances from the 56 studies in the Raees et al. review. They report that 80 of 94 in-scope instances are identified by recorded data or by study design, while 14 need extra information or assumptions. Across 13 papers where it can be recovered, the median final agreement rate is about 0.75, which implies a roughly 75-percentage-point baseline range.
Critical strengths
-
Interesting Problem: The point that final-only records cannot tell "I already agreed with the AI" apart from "I switched to the AI", and that this leaves an irreducible range of width g, is stated clearly and illustrated well (Fig. 1, Sec. 2). For the HCI/human-AI community it is a useful warning, and the Eq. 2 decomposition (Delta = p+ - g q_agree) is a nice way to see where the uncertainty comes from.
-
Partial-identification treatment: The bounds are sharp and come with proofs: switch-fraction bounds (Prop. 2, Eq. 12), bounded selection (Prop. 4), the known-kernel row-space condition for the activation-integration model (Prop. 10), and LP-based component relaxations (Prop. 11). There are also minimax lower bounds on estimation error (Prop. 1, App. D.2) and on decision regret (Cor. 5).
-
Interesting empirical observation on prediction versus identification: Fig. 3 and App. E.3.1 show that the fitted activation-integration kernels cut Brier loss by 31-53% while shrinking bound width by only 0-10%. Better predictive models of human behavior do not automatically give tighter conclusions about the estimand. This is a good message for ML researchers who fit behavior models.
Critical weaknesses
-
The core theoretical result is straightforward, and the theory as a whole is a standard application of Manski-style worst-case bounds: Given keep-or-adopt, Eq. 2 shows that Delta is linear in q_agree, which lies in [0,1]. Prop. 1 follows at once, and its width-g statement is just the coefficient on q_agree. Prop. 2 is a small LP over beneficial, harmful and neutral switch masses. Prop. 3 and Prop. 4 intersect that set with a band. Theorem 2 and Cor. 3 intersect it with a known accuracy range. The minimax lower bound of g/2 is the usual two-point argument. Almost all of the machinery is textbook partial identification (Manski 1989/1990; Horowitz & Manski 1995; Masten & Poirier 2018), as the paper itself notes ("The interval logic is standard", App. preamble). The paper also does not clearly set its contribution against closely related work it cites: Guo et al. (2024), Ben-Michael et al. (2025), Schemmer et al. (2023b) and Agarwal et al. (2025). Whether pre-advice agreement and reliance are confounded in final-only data is a known issue in the appropriate-reliance literature.
-
The behavioral constraints need the very data whose absence motivates the paper: In Sec. 4.1, the switch fraction, the agreement-disagreement gap and the behavioral models are all calibrated on 50% of participants from the same study, using their revealed paired initial and final judgments. In a real final-only study, where would a researcher get alpha or the gap bound? Probably from a paired study in a different workflow. But then the transport problem the paper itself raises applies: eliciting a pre-advice judgment may change behavior (Prop. 14, App. B.3, C.3), and none of the experiments test this. The comparison "behavioral constraints vs. sparse direct measurement" therefore compares two kinds of within-study paired data, not a realistic final-only scenario. The paper concedes that "Repeated splits reuse people and retain the original tasks and items; they establish neither independent replication nor new-item or cross-workflow transfer" (App. E.2). That limits how far the empirical conclusions reach.
-
The tighter bounds often rest on constraints that do not hold: Gain inclusion is close to 100%, but joint validity of the imposed restrictions is much lower: 63% in Census, 72% in Sarcasm, 81% in Cities and Dermatology (Table 8a). It is 89% in pooled HAIID (Table 10) and in Tejeda (Sec. E.3.2). Support is valid in only 94% (HAIID) and 89% (Tejeda) of half splits (App. E.2). Zero-error fitted models are incompatible with the data in 99-100% of splits (Tables 10, 12). With the budget multiplier at 0.25, neither component is valid in any split, yet the gain is still included (Table 11b). Gain inclusion is therefore a weak check: an interval built from false assumptions can still happen to contain the true value. The headline claim that behavioral constraints "narrow the bounds" would be more convincing if the narrowing came with valid restrictions, or if coverage were analysed under misspecification.
-
Behavioral models add little in most tasks, and the one large gain comes from a single component: With patterns already imposed, the model shrinks width by 0.01, 6.33, 1.31, 0 and 0.01 points in the five binary tasks (App. E.3.1; Table 8b). In pooled HAIID it adds nothing (22.16 to 22.16, Table 10). The large Tejeda reduction (81.07 to 19.70) comes entirely from the initial-accuracy constraint Q. Removing Q restores 81.07, while removing E or P changes nothing (Table 12a). In effect, Q supplies pre-advice accuracy information from the fitting participants' initial judgments. That is closer to direct measurement through a model than to a "behavioral mechanism" constraint, and it weakens the claimed comparison between models and direct measurement for object recognition.
-
The direct-versus-behavioral comparison is not apples-to-apples: Direct intervals use a Hedged-WoR finite-population construction with a 95% guarantee. Behavioral intervals use a bootstrap with no finite-sample guarantee and different tail allocations (0.05/6, 0.05/8, 0.05/9) (Fig. 2 caption, App. E.4). The paper admits that "Matching participant partitions therefore does not equalize uncertainty constructions" and that the Tejeda comparison "also reflects the different calibration procedures" (App. E.3.2). The "measurement threshold" findings, for example that direct measurement wins in Art beyond 9.7% (Sec. 4.1), may be partly an artifact of these different procedures. There are other oddities too. Direct widths grow from 50% to 75% calibration in all six tasks because the evaluation set shrinks (App. E.4), and Dermatology has only 20 participants, which makes its direct widths of about 77-89 points hard to interpret.
-
The estimand needs better motivation: The paper notes that pre-advice accuracy in a concurrent (unprompted) workflow may not even be well defined ("If an unprompted pre-advice judgment is not well defined, Delta_tar must first be specified", App. B.3). Randomized comparisons with a no-AI arm (Ben-Michael et al., 2025) already identify the effect of offering advice. The paper says the targets are complementary (Sec. 1), but it never shows a decision or scientific question where Delta, rather than the assistance effect, is what matters and is also not available from a standard randomized design. Without this, the practical stakes of the 75-point range are unclear.
-
Venue fit and accessibility: The contribution sits mainly in HCI methodology and econometric identification, and there is little machine-learning content. The paper is very dense. The main text leans heavily on a 40+ page appendix, and many central details live there: calibration protocols, the meaning of "requested" versus measured percentages, the support allowance, and how the model enters the LP.
Questions for authors
See weaknesses
2 comments
Response to Reviewer gsja [2/2]: transfer (W2), behavioral models (W4), the comparison (W5), the target quantity (W6), fit (W7)
W2. Where would a final-only study get the switch fraction or gap bound? Transfer is not tested.
We tested transfer across HAIID tasks and from the sequential to the concurrent workflow in Tejeda et al. (2022). In these analyses, initial accuracy was more stable than behavioral patterns (the switch fraction and gap) (GymW Q2), although this is not a guarantee for new settings. The workflow comparison also bears on your concern about asking for initial judgments: final AI agreement is 4–8 pp higher in the concurrent workflow, while unaided accuracy differs little. Final agreement alone cannot show whether the underlying advice-taking process changed. Our analysis thus identifies initial accuracy as the more portable quantity and behavioral patterns as specific to the task and workflow, so conclusions that rely on transferred patterns should be reported over a range of their values.
W4. The Tejeda gain comes from Q, which is closer to direct measurement.
This reading matches our ablation. In object recognition, the additional narrowing comes entirely from Q, which carries initial-accuracy information from the participants used to fit the model. As you observed, removing E or P leaves the width unchanged. Section 4.1 now attributes that gain to this accuracy information, rather than to recovering an advice-taking process from final decisions.
W5. The comparison is not apples-to-apples.
We agree. With matched participants, departure allowance and calibration procedure, direct measurement is narrower in all six tasks at the budgets meeting the 90% gain-inclusion check in XZnv Q1. The earlier "measurement threshold" findings came from the different calibration procedures. We therefore treat direct measurement as the reference and ask how close each behavioral source gets to it (general response): behavioral patterns close much of the distance, and their information overlaps with direct measurement once it is available. Both strategies are over-confident at the smallest budgets, so we treat those results as descriptive. Direct widths increase from 50% to 75% calibration because the evaluation sample shrinks, and Dermatology's wide intervals reflect its 20 participants.
W6. Why study the change from initial to final accuracy rather than only the effect of providing AI assistance?
A randomized no-AI condition is the cleanest design when available. Our analysis matters in three cases.
- Records without initial judgments. In the audit, 32 of 96 claim–workflow instances come from such workflows, and deployed systems often log only final decisions.
- How decisions change within the AI-assisted workflow. A no-AI condition estimates the overall effect of providing assistance, but not how assisted participants revised their own initial judgments. Δ measures this net change. Separating beneficial from harmful switching needs further information. In Rastogi et al. (2022), randomization establishes the effect of time on agreement (−19 pp), but the stricter reading that time reduces harmful switching holds only if extra time raises unaided accuracy on probe trials by less than 19 pp (see XZnv, Q2).
- A no-AI condition as additional information. Its unaided accuracy can constrain initial accuracy in the assisted workflow, provided the two conditions are comparable, and then enters the bounds directly.
On whether this change is meaningful in a concurrent workflow: in the Tejeda data, the sequential initial-to-final change and the concurrent randomized effect of AI are numerically close (+5.4 vs. +6.2, +11.0 vs. +9.6, +16.1 vs. +17.2 pp). They measure different quantities, and relating them requires assuming that unaided accuracy carries over across workflows and that blocks do not affect each other.
W7. Venue fit and accessibility.
Human–AI decision-making is an established topic at ICLR, ICML and NeurIPS (see the general response). Two of our findings speak directly to ML researchers who evaluate human–AI systems: better predictions of final decisions need not give a more informative evaluation of accuracy change, and initial human accuracy was more stable than behavioral patterns in the settings examined. To make the paper easier to follow, we have moved the calibration procedure, measurement budgets and relaxed keep-or-adopt analysis into the main text, and will shorten material not needed to assess the claims in the final version.
Thank you again for the careful review. We hope the new analyses address your concerns, and we would be grateful if you would take them into account in your final assessment. We welcome further discussion of the assumption-validity results.
Response to Reviewer gsja [1/2]: theory and positioning (W1), validity of the assumptions (W3)
Thank you for the very detailed review and for finding the problem clearly stated and the gap between prediction and identification interesting. You raised central questions about the empirical comparison, and several of our new analyses respond directly to them. Where results also appear in our responses to XZnv and GymW, we give the main point here and refer to the full numbers there.
W1. The theory is a standard application of Manski-style bounds, and the contribution is not set against closely related work.
We agree that the mathematical tools are established, and prior work recognizes that final agreement with the AI can hide different initial judgments. Our contribution is what this ambiguity allows researchers to conclude about accuracy change, and which additional observations or assumptions strengthen those conclusions. Using an established tool as an analytical lens in this way follows recent ICLR work on human–AI decisions and on bounding ML evaluations (see the general response). Relative to the cited work:
W3. The tighter bounds often rest on restrictions that do not hold.
Most of these failures come from one assumption, the allowance for departures from keep-or-adopt, rather than from the behavioral constraints themselves. We checked each assumption separately in the paper's repeated 50% participant splits:
The allowance fails most often in Census and Sarcasm because departures there are rare (0.15% and 0.39% of trials) and come from a few people, so a bootstrap over participants often misses them. The splits where assumptions fail do not drive the narrowing: mean width over splits where all hold is equal to or slightly larger than the overall mean.
Does the narrowing survive when the assumptions hold? We reran the 50% splits with a conservative finite-sample allowance. The summary-level constraints (switch fraction and gap) and the allowance then hold jointly in 94–97% of splits. This check does not cover the full behavioral models. The behavioral bounds remain much narrower than the final-only range (Art: 49 vs. 99 pp for final decisions alone under the same conservative allowance), but become 5–19 pp wider and no longer show whether accuracy improved or declined. Direct measurement is almost unaffected (Art: 6.5 pp, direction determined in all splits). This is itself a finding: the narrowing from behavioral patterns depends on how departures from keep-or-adopt are bounded, whereas narrowing from initial-accuracy information does not. We therefore report each narrowing together with the assumptions it relies on. We also agree that gain inclusion alone is a weak check, as your examples show. For the fitted models, exact (zero-error) versions are rejected by the paired data in almost all splits of the pooled HAIID and object-recognition analyses, as you note, so we treat model-based narrowing as conditional on the model. Once initial judgments are measured, models add at most 0.2 pp. To test robustness to misspecification directly, we report the largest departure rate each conclusion can tolerate (see XZnv, Q3).
Action: In the final version, we will report assumption validity next to gain inclusion, show results under both allowances, and describe any narrowing together with the validity of the assumptions it relies on.