- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper essentially checks the agents' downstream behaviour when given prior agents' verdict or verdict plus reasoning / "explanation"
Critical strengths
I think the paper looks at an interesting problem and runs some interesting experiments. I actually think there is a really interesting story and analogy here that is completely missed by the authors -- how humans behave when shown an AI verdict vs a verdict + explanations. There are some analogous results to how agents act here, but it's not quite the same and I think the difference can be one of the most interesting findings here.
Critical weaknesses
This paper is almost unreadable -- this is the main reason for my recommendation to reject. A paper should be a tool of communication and should bring the reader into the project, carefully explaining the choices and reasoning. If I were not obliged to try and understand this paper due to being assigned to review it, I cannot imagine myself putting the required effort. This is more stark, as this is not a complicated paper in terms of narrative or setup, so I do not see a reason not to invest more in making the paper engaging and clear to the reader.
Due to this, I am also not able to properly verify the soundness of the paper -- although I didn't find any clear issues with the design.
Questions for authors
-- Did you, honestly, read this paper top to bottom before submitting it? Do you think it is clear and tells a strong narrative of the problem, your choices, your results and their implications that would be clear to someone not familiar with the work?
-- Why did you only focus on one problem / setup? Seems easy to expand this to at least 1-2 more tasks.
3 comments
Reply part 2
Your suggestion: a story we had missed, how people behave when shown an AI's verdict versus a verdict with an explanation, where the difference between people and our agents could be one of the most interesting findings.
Thank you so much for raising this interesting discussion; we did some exploration and added a new paragraph in Section 7. We find that people tend to rely too much on an AI's verdict. Nine of our ten models do the opposite with a bare one: under a header saying that the verifier checked the note, they follow a correct bare verdict in at most one retail conversation in five with our standard note (Appendices D and E). Once an explanation is added, they follow the verifier more, right or wrong, as people do. When the verifier is right, the whole assessment is followed at least as often as the bare verdict in all 30 model-task pairs. When it is wrong, the three models we tested follow a wrong whole assessment in 16–68% of conversations and a wrong bare verdict in 1–2%. There is one more parallel, with a correction that repeats the false claim, which is what quoting the false line does: in the airline task under the plain header, FALSE quoting the false line leaves all ten models fewer forbidden writes than the bare FALSE, 1–32% against 25–99% (Table 2). The comparison draws on published studies of people (further investigation and comparison with people will be in our future plan). Studies of automation and of AI-assisted decisions document over-reliance on automated and AI advice [1, 2]. An explanation raises acceptance of AI advice whether the advice is right or wrong [3, 4]. Even a bare predicted label raises people's accuracy in deception detection [5], and a correction that repeats the false claim can work better than one that does not [6].
If the rewritten paper now lets you verify its soundness, we hope you will reconsider your score.
References
[1] R. Parasuraman and D. H. Manzey. Complacency and bias in human use of automation: An attentional integration. Human Factors, 52(3):381–410, 2010.
[2] Z. Buçinca, M. B. Malaya, and K. Z. Gajos. To trust or to think: Cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1):1–21, 2021.
[3] G. Bansal, T. Wu, J. Zhou, R. Fok, B. Nushi, E. Kamar, M. T. Ribeiro, and D. Weld. Does the whole exceed its parts? The effect of AI explanations on complementary team performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–16, 2021.
[4] S. S. Y. Kim, J. Wortman Vaughan, Q. V. Liao, T. Lombrozo, and O. Russakovsky. Fostering appropriate reliance on large language models: The role of explanations, sources, and inconsistencies. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages 1–19, 2025.
[5] V. Lai and C. Tan. On human predictions with explanations and predictions of machine learning models: A case study on deception detection. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 29–38, 2019.
[6] U. K. H. Ecker, J. L. Hogan, and S. Lewandowsky. Reminders and repetition of misinformation: Helping or hindering its retraction? Journal of Applied Research in Memory and Cognition, 6(2):185–192, 2017.
Dear Authors,
This is a rebuttal, not a revise and resubmit. I do not intend to re-read your paper, it was difficult enough the first time. I don't think the issues that I found are fixable without a full rewrite, which is out of scope within this process.
I highly suggest you give the draft to a human friend or colleague to test readability before submitting again.
Reply part 1
Thank you for your valuable review and constructive suggestions. Here are our replies to your main concerns.
Weakness. the paper is almost unreadable even though its narrative and setup are not complicated, so you could not properly verify its soundness, although you found no clear issue with the design.
Thanks for your feedback. We've revised the main text. The story is: A verifier agent reads a colleague's note that falsely says the airline cancelled a flight, which would allow a refund. It judges the note FALSE and plans to KEEP the booking. If this whole assessment reaches the agent that acts, that agent rarely makes the forbidden cancellation. If only the two labels, FALSE and KEEP, reach it, the forbidden cancellation becomes much more common: 36% more on average over our ten models. Most of the protection comes back once the verdict says which claim is false and what is true instead: one sentence that states the true fact does this in 28 of 30 model-task pairs, which cover all ten models in all three tasks. In all three tasks we now test, the bare verdict is never safer than the whole assessment.
The introduction explains this with a familiar picture, a clerk who receives a form together with a slip that says only REJECTED. Section 2 explains each design choice, for example why we fix the labels and keep the false note in view, and Table 1 shows one concrete example per task. Section 6 turns the results into five rules for system builders, each with the case in which it fails (Table 4): check permission in code where possible, otherwise pass on the whole assessment from a check you trust, and if only a short field can pass, make it say which claim it rejects and what is true. We hope this also makes the design easy to check.
Question 1. whether it is clear and tells a strong narrative of the problem, our choices, our results and their implications to a reader not familiar with the work.
The revised introduction now gives that narrative in order: the problem (a correct verdict that arrives without its reason), our choices (a correct assessment held fixed while the false claim stays in view, read from the final database), our results (the bare verdict fails, and saying which claim is false and what is true restores protection) and their implications (the rules of Table 4). Its last paragraph says what is new: in the airline task the same FALSE protects far more when it quotes the false line than when it quotes a true one, and an order to stop protects more, for nine of our ten models in at least one chat role, when it appears to come from another agent (Table 3, Appendix R). We would be glad to hear whether it now reads clearly to you.
Question 2. why we focused on only one problem and setup, when it seemed easy to expand to at least one or two more tasks.
Thanks for your suggestions. We began with τ-bench airline because in its permission rules a false claim directly unlocks a forbidden write, and the final database shows whether it happened (Section 2). We've added two tasks: a retail task adapted from τ-bench and a document-sharing task. The core comparison now runs in all three and inside a LangGraph supervisor, each with five open and five closed models. In all three tasks the bare verdict is never safer than the whole assessment, and one sentence of the true fact closes most of the gap in 28 of the 30 model-task pairs. In each task, a permission rule turns on one fact that the record shows directly or by inference, such as whether the airline cancelled the flight. In our audit of the policies of τ-bench airline and retail and τ²-bench telecom, 83 of the 134 rules that decide an action (62%) rest on a fact that a tool output states or implies, the structure our three tasks share (Section 2, Appendix T).