- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces DistressBench, a benchmark for evaluating large language models' responses to suicide and self-harm crises. It contains 718 clinician-authored conversations, task-specific weighted rubrics covering seven dimensions of crisis support, and clinician-edited reference responses. Evaluating 25 models, the authors find that models perform substantially better at avoiding harmful content than at providing actionable support, particularly de-escalation and crisis-resource routing. The paper also investigates the gap between crisis recognition and intervention, and examines multi-turn evaluation and long-horizon safety failures. The benchmark highlights the limitations of refusal-based evaluation, although its conclusions depend on rubric validity, judge reliability, and the use of simulated rather than real-world conversations.
Critical strengths
- Important evaluation goal. The benchmark assesses whether models provide meaningful crisis support, extending evaluation beyond refusal and harmful-content avoidance.
- Detailed, scenario-specific rubrics. Expert involvement and task-specific criteria enable more informative analysis than a single overall safety label.
- The evaluation covers 25 models with three independent runs per model. The paper also examines dimension-level performance, multi-turn scoring rules and response omissions, providing a broader analysis than a conventional benchmark leaderboard.
Critical weaknesses
- The clinical meaning of the aggregate score remains uncertain. A weighted rubric satisfaction rate does not automatically represent the proportion of necessary care delivered.
- The selected panel's advantage over its nearest competitor is not statistically significant, and its ranking is sensitive to the removal of the disclaimers dimension. It is unclear whether using an alternative panel would change the model rankings or main findings.
Questions for authors
- Among the conversations contributing to the reported 35.3% gap, how many provide no effective help-seeking pathway, rather than incomplete resource guidance?
- How were criterion weights justified, and do the main conclusions remain stable under equal weighting or reasonable weight changes?
- How do the authors distinguish behavioral or product choices from differences in model capability, response length, and evaluation settings? Would controlling for these factors change the conclusions?
2 comments
Thanks for additional information of this paper. I would like to keep my score as 3.
Thank you for the review and for the specific questions.
Q1: How many gap conversations provide no effective help-seeking pathway? We sampled 200 gap conversations at random and coded them against a fixed codebook, with an LLM first pass checked by one of the authors (new Appendix E.10; our response to Reviewer VGW3 has the details). Of these:
On a stratified subset, the author's independent codes matched the LLM's on whether a route was offered in 19 of 20 conversations. So the gap mostly reflects incomplete resource guidance. The abstract, introduction, and Section 4.6 now report this breakdown and no longer suggest that users in the gap received no route to help.
Clinical meaning of the aggregate score. We agree, and no longer describe the score as the proportion of necessary care delivered. In our calibration data, clinicians' final-turn "would I send this" ratings do track rubric scores, though only modestly (AUC 0.63; Appendix E.11), and those ratings were not reliable across clinicians. A holistic clinical validation remains open, and we list it first in future work.
Q2: How were the weights set, and do the results hold under other weightings? Each clinician assigned weights from 1 to 20 within their own task while revising its rubric, replacing the drafting model's weights in every case. The drafts always used 1 to 7 once each, whereas the final per-task totals range from 25 to 291 (Section 2.3). A weight therefore expresses priority within a single conversation. The results are not sensitive to the weights (new Appendix E.8):
Choice of judge panel.
Q3: Separating choices from capability, response length, and evaluation settings.