- Rating
- 4(+2)
- Confidence
- 4
- Informativeness
- 3(+1)
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
Explanations are usually provided to human users, but are often optimized for higher task performance by AI models or are preferred by human users. This paper shifts the target of explanations to help people make better decisions. They used an AI-human writing "Turing test" as the task to test whether LLM explanations generally help humans improve their performance on the task and whether they can train them to be better at helping humans. The authors find that humans with LLM-generated explanations can only do the task at 51.3% accuracy, while after applying training human decision simulators and also using GRPO on Qwen to optimize decision quality (which are likely to be used by humans, as indicated by the decision simulator), human performance can improve to around 65% accuracy. This suggests an actual improvement of the explanations that help humans make better decisions. But comparing different training baselines, the authors confirm the necessity of those explanations, as well as the human decision simulator.
Critical strengths
I like the general idea of improving human decision quality by training the explanations, rather than merely making AI models better at tasks or more preferred by humans. I think there are several critical strengths:
- Real human participants are involved in the paper, not just human simulators or virtual participants.
- A loop of the intervention. The authors show the performance of training by observing actual performance improvement by humans.
- Several baselines and "ablations" in the training process show the necessity of using explanations and human decision simulators during training guidance.
- The human decision simulator design is nice since it can help efficiently score the rollouts during RL post-training of Qwen rather than having human data to label online.
Critical weaknesses
The weaknesses are also significant in this paper. I will start with my overall impression and then move to specific points.
-
Overall experimental logic. When I read the paper, I found the experimental setup somewhat hard to follow. This is not because the authors did not describe the steps clearly, but because the logic connecting them has several gaps. When I try to follow a consistent axis, it is difficult to understand the designs, their rationales, and what we should expect from each step. The distinction in the introduction is already clear: an explanation that improves an LLM’s task performance or is preferred by users does not necessarily help humans make better decisions. Therefore, the project should distinguish two axes: how useful the explanations are for AI models performing the task, and how effectively the information in these explanations can be transferred to humans (quality × transferability). With these two axes, many experimental components could be placed and understood more clearly. This would also make the missing pieces in the experimental setup apparent, as discussed below.
-
Explanation quality. There are several missing baselines. For example, the authors use GPT, Gemini, and Claude to generate explanations before presenting them to humans, but do not report how well these models can use their own explanations to support decisions on this task. A related result is that Qwen achieves approximately 76% accuracy when using these explanations. However, without a text-only baseline for Qwen, we do not know whether the explanations contribute to this performance. This matters because, when humans receive these explanations and achieve only 51.3% accuracy, is it because the explanations are poor and miss the relevant signals, or because they contain useful information that humans cannot efficiently use? Without addressing this question, we do not know what the authors are “calibrating”: the quality of the explanations, their transferability, or both.
-
Explanation transferability and the human behavioral baseline. Human participants begin the experiment with explanations, so we do not know how well they perform without any explanations. This baseline would establish whether the initial explanations help humans, have no effect, or even mislead them. For example, suppose humans originally achieve 65% accuracy, the initial explanations reduce performance to 51.3%, and the RL framework brings it back to approximately 65%. These numbers come from different experimental stages in the paper, so this is a hypothetical scenario rather than an established trajectory. Nevertheless, the current design cannot rule it out. In that case, the framework improves performance relative to the initial explanations, but why do we need it in practice? Why not simply let humans perform the task without explanations? The same need for a no-explanation baseline applies to Qwen.
-
The added value of the calibration framework. The lack of a clear separation between these axes, together with the missing baselines, makes me skeptical about whether the calibration is sufficient and necessary. There may be easier ways to achieve the goal. For example, can prompting optimization help state-of-the-art LLMs generate human-friendly explanations without RL training? Could some historical human decision data be included in context to ask an LLM to adapt its explanations to the target population, or to a specific user when individual history is available? I do not require the authors to evaluate every baseline or alternative approach. The key point is to establish a reasonably clear understanding of why the method works, what it calibrates, and whether the additional training is needed to obtain the improvement. The evidence should show that the framework improves human decisions beyond reasonable baselines, rather than merely restoring performance that the initial explanations had degraded.
-
Missing related work. Several highly relevant papers from AI and behavioral/cognitive modeling are not cited or discussed. These works provide important context for both the human decision simulator and the use of RL to train LLMs with human behavioral data:
-
Binz, M., & Schulz, E. (2024). Turning large language models into cognitive models. International Conference on Learning Representations.
This paper presents a closely related approach to adapting LLM representations to predict human behavior. The authors should discuss how their human decision simulator relates to this approach and clarify the methodological differences.
-
Binz, M., Akata, E., Bethge, M., Brändle, F., Callaway, F., Coda-Forno, J., ... & Schulz, E. (2025). A foundation model to predict and capture human cognition. Nature, 644(8078), 1002–1009.
Although the approach is less directly similar, this work provides important context for training language models to predict human behavior and should be discussed when positioning the human decision simulator.
-
Zhu, J. Q., Xie, H., Arumugam, D., Wilson, R., & Griffiths, T. L. (2026). Using reinforcement learning to train large language models to explain human decisions. International Conference on Learning Representations.
This paper also uses RL post-training with human behavioral data to train LLMs to explain decisions. Its objective differs from the present paper: explaining human decisions versus generating explanations that improve human decisions. Nevertheless, the methodological connection is relevant and should be discussed.
-
Questions for authors
I tried to make a clearer list of questions (as an extension of the weakness statement) so that the authors can potentially respond during rebuttal:
-
Human performance without explanations. How accurately do participants perform the task without any explanations, under the same experimental conditions? This would establish whether HD-RL provides a net benefit over independent judgment, rather than improving upon explanations that may themselves reduce performance.
-
Explanations from untrained Qwen. How well do participants perform with explanations generated by the original Qwen model using the same prompt and format? This would isolate the benefit of post-training within the final randomized experiment.
-
Human-oriented prompting. Could an untrained model achieve comparable results when explicitly prompted to generate explanations that help non-expert humans make accurate decisions? A reasonable prompting baseline would help establish the added value of the behavioral training pipeline.
I do not expect an exhaustive comparison against all possible alternatives. These comparisons would address the main uncertainty: whether the proposed framework improves human decisions beyond independent judgment and straightforward explanation-generation approaches.
7 comments
Response to Reviewer JjYU (Comment 2)
2. Explanation quality
Thank you for raising this question. We ran both baselines you suggested:
(1) Self-Evaluation, where GPT, Claude, and Gemini classify texts using their own generated explanations.
| Model | Overall | Human Texts | AI Texts |
|---|---|---|---|
| GPT 5.4 | 90.7% | 97.5% | 83.9% |
| Claude Opus 4.5 | 89.1% | 95.8% | 82.4% |
| Gemini 3.1 | 83.0% | 91.9% | 74.2% |
(2) Text-Only baseline for Qwen 3.6-27B, where the model only sees the raw text.
| Model | Overall | Human Texts | AI Texts |
|---|---|---|---|
| Qwen (Text-Only) | 70.9% | 82.7% | 59.2% |
| Qwen (Text + Unoptimized Explanation) | 76.4% | 86.5% | 66.4% |
Going back to your question: "when humans receive these explanations and achieve only 51.3% accuracy, is it because the explanations are poor and miss the relevant signals, or because they contain useful information that humans cannot efficiently use?" The above results show that the explanations do contain useful signal that Qwen can use to boost it’s score. However, Qwen also starts with greater overall accuracy on the task than humans. So it is likely both humans having worse unaided performance and not extracting all the available information from the explanations.
As shown in baseline 2, on the raw text alone, Qwen already achieves 71% accuracy. Adding the unoptimized explanations increases Qwen's accuracy to 76%. However, when humans read these same explanations, they achieve only 51% accuracy, suggesting these signals are difficult to transfer to humans.
This helps motivate our method of calibrating explanations to human decisions. This process is a way to learn which signals in the unoptimized explanations actually help human readers and which signals mislead them.
Response to Reviewer JjYU (Comment 3)
3. Explanation transferability and the human behavioral baseline
Thank you for pointing this out. To address your concern, we evaluated two additional human baselines on the held-out test set:
- Text-only: Human participants see only the text without an explanation.
- Raw Qwen: Human participants see the text and an explanation generated by the untrained Qwen 3.6-27B model.
As shown in the Table 1 below, the Text-only baseline reaches 50.4% GLMM-estimated mean human accuracy, with 55.2% on human-written texts and 45.6% on AI-generated texts. This rules out the possibility that human participants already achieve high accuracy on this task without explanations.
One interesting observation is that providing untrained Qwen explanations barely changes aggregate accuracy (50.4% to 51.3%), but shifts human decisions toward the "Human" label (increasing accuracy on human texts from 55.2% to 60.0% while reducing accuracy on AI texts from 45.6% to 42.5%). This observation connects with our Section 6.3 that the baselines in our main draft (LLM_decision, DPO-RLAIF, and VR-GRPO) often highlight cues that imply human authorship even on AI texts. Raw Qwen exhibits this same problem.
Table 1
| Method | Aggregate Acc. (95% CI) | Human Text Acc. (95% CI) | AI Text Acc. (95% CI) |
|---|---|---|---|
| HD-RL (Ours) | 63.7% [60.4%, 66.8%] | 66.6% [62.8%, 70.1%] | 60.7% [56.6%, 64.6%] |
| Text-only | 50.4% [45.9%, 55.0%] | 55.2% [49.8%, 60.5%] | 45.6% [40.2%, 51.1%] |
| Raw Qwen | 51.3% [46.7%, 55.8%] | 60.0% [54.7%, 65.0%] | 42.5% [37.2%, 47.9%] |
Response to Reviewer JjYU (Comment 1)
We thank the reviewer for your positive feedback and insightful suggestions.
Thank you for the suggestions. We agree that our initial exposition was not as clear as it should have been, and have revised since the submission deadline to clarify the motivation behind each decision we made. We have also conducted additional experiments to fill out the missing comparisons, and discuss your two proposed axes. Specifically, we evaluate Axis 1 (how useful the explanations are for AI models) in our response to Comment 2 ("Explanation quality") below, and we evaluate Axis 2 (how effectively the explanations transfer to humans) in our responses to Comment 3 ("Explanation transferability") and Comment 4 ("The added value of the calibration framework") below. We will update Section 5 in the revised manuscript.