acceptodds
Under review as a conference paper at ICLR 2027

DistressBench: Evaluating Multi-Dimensional Crisis Support in Large Language Models

Abstract

Conversational language models have been adopted across everyday settings faster than the evaluation practices built to govern them. A concerning share of use is becoming affective rather than informational. Within that traffic users disclose suicide and self-harm (SSH) crises. Existing safety benchmarks nonetheless score models primarily on refusal. In this setting refusal is not the desired behavior; what the conversation requires is a substantive supportive response. We introduce DistressBench, 718 English conversations with clinician-authored user turns spanning 23 SSH subcategories, each paired with a task-specific rubric of weighted atomic criteria across seven dimensions of crisis support and a clinician reference response. A model's score is the weighted average over the criteria its reply satisfies. Because each criterion specifies a discrete action a clinician judged necessary, this quantity admits a direct operational reading as the fraction of required care delivered, rather than a position on a latent quality scale. We evaluated 25 frontier models and the best performing model satisfies 88.4% of weighted criteria and the median model 71.4%, against 98.7% for clinician references on the same instrument. However, pooled across all 25 models, in 35.3% of the conversations where a model satisfied every clinician-written criterion for recognizing the crisis, it failed to route the user toward human help. The median model leaves out nearly three in ten of the actions the conversation called for, and the user may have that conversation only once. The shortfall concentrates in the dimensions that require the model to act rather than to withhold. Contrasting each dimension's median model against its best separates capability limits from choices: de-escalation reaches 74.6% in the strongest model against a median of 46.0%, and safety disclaimers 84.6% against a median of 8.3%. Multi-turn conversations score below single-turn ones (68.5% against 74.2%). We release the corpus, the rubrics, the judge calibration labels, and a companion annex of 20 long-horizon red-team transcripts. These results argue for evaluating crisis safety on the care a model delivers rather than on what it declines to say, and for treating de-escalation and resource routing as deployment requirements.

Primary area:
datasets and benchmarks
Keywords:
crisis support, suicide and self-harm safety, LLM safety evaluation, benchmark, clinician-authored rubrics, LLM judge calibration, recognition-action gap, multi-turn evaluation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_6W7uOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces DistressBench, a benchmark for evaluating large language models' responses to suicide and self-harm crises. It contains 718 clinician-authored conversations, task-specific weighted rubrics covering seven dimensions of crisis support, and clinician-edited reference responses. Evaluating 25 models, the authors find that models perform substantially better at avoiding harmful content than at providing actionable support, particularly de-escalation and crisis-resource routing. The paper also investigates the gap between crisis recognition and intervention, and examines multi-turn evaluation and long-horizon safety failures. The benchmark highlights the limitations of refusal-based evaluation, although its conclusions depend on rubric validity, judge reliability, and the use of simulated rather than real-world conversations.

Critical strengths
  1. Important evaluation goal. The benchmark assesses whether models provide meaningful crisis support, extending evaluation beyond refusal and harmful-content avoidance.
  2. Detailed, scenario-specific rubrics. Expert involvement and task-specific criteria enable more informative analysis than a single overall safety label.
  3. The evaluation covers 25 models with three independent runs per model. The paper also examines dimension-level performance, multi-turn scoring rules and response omissions, providing a broader analysis than a conventional benchmark leaderboard.
Critical weaknesses
  1. The clinical meaning of the aggregate score remains uncertain. A weighted rubric satisfaction rate does not automatically represent the proportion of necessary care delivered.
  2. The selected panel's advantage over its nearest competitor is not statistically significant, and its ranking is sensitive to the removal of the disclaimers dimension. It is unclear whether using an alternative panel would change the model rankings or main findings.
Questions for authors
  1. Among the conversations contributing to the reported 35.3% gap, how many provide no effective help-seeking pathway, rather than incomplete resource guidance?
  2. How were criterion weights justified, and do the main conclusions remain stable under equal weighting or reasonable weight changes?
  3. How do the authors distinguish behavioral or product choices from differences in model capability, response length, and evaluation settings? Would controlling for these factors change the conclusions?
2 comments

Thank you for the review and for the specific questions.

Q1: How many gap conversations provide no effective help-seeking pathway? We sampled 200 gap conversations at random and coded them against a fixed codebook, with an LLM first pass checked by one of the authors (new Appendix E.10; our response to Reviewer VGW3 has the details). Of these:

  • 5.5% [3.1, 9.6] offer no route to human help anywhere in the conversation;
  • 18% offer none in the final reply;
  • 8.0% offer nothing adequate for the level of risk the user expressed.

On a stratified subset, the author's independent codes matched the LLM's on whether a route was offered in 19 of 20 conversations. So the gap mostly reflects incomplete resource guidance. The abstract, introduction, and Section 4.6 now report this breakdown and no longer suggest that users in the gap received no route to help.

Clinical meaning of the aggregate score. We agree, and no longer describe the score as the proportion of necessary care delivered. In our calibration data, clinicians' final-turn "would I send this" ratings do track rubric scores, though only modestly (AUC 0.63; Appendix E.11), and those ratings were not reliable across clinicians. A holistic clinical validation remains open, and we list it first in future work.

Q2: How were the weights set, and do the results hold under other weightings? Each clinician assigned weights from 1 to 20 within their own task while revising its rubric, replacing the drafting model's weights in every case. The drafts always used 1 to 7 once each, whereas the final per-task totals range from 25 to 291 (Section 2.3). A weight therefore expresses priority within a single conversation. The results are not sensitive to the weights (new Appendix E.8):

  • With all criteria weighted equally, the ranking correlates with the original at Spearman 0.996, the top model is unchanged, and no model moves more than two places.
  • Across 200 random perturbations, each multiplying every weight by independent lognormal noise (σ = 0.5), Spearman never drops below 0.996 and the top model never changes.
  • The recognition-action gap uses only pass/fail verdicts, so the weights do not affect it.

Choice of judge panel.

  • Choosing the panel on training tasks and scoring it on held-out tasks gives κ between 0.721 and 0.726, compared with 0.732 in-sample (Appendix B.4).
  • We also rescored attempt 1 for all 25 models with a panel that shares no judges with ours: Gemini 3.5 Flash, Mistral Large 2512, and GPT-5.6 Luna (Appendix B.5, Table 5). This panel ranks 308th of 455 on calibration agreement (κ = 0.718 vs. 0.732), which is still above the clinicians' agreement with each other (0.640). We could not rerun the runner-up panel from selection because four of the candidate judges are no longer served.
  • The two panels give nearly identical rankings (Spearman 0.996, Kendall 0.973). No model moves more than two places, the top five are the same and in the same order, and the top model's lead over second place is significant under both.
  • The alternative panel is stricter across the board, scoring models 0.045 lower on average, but this lowers scores without changing the order.
  • The main findings carry over: the gap is 37.3% vs. 35.4%, disclaimers and de-escalation still show the largest gaps between the best and median models, and the restraint dimensions remain near ceiling.

Q3: Separating choices from capability, response length, and evaluation settings.

  • Table 1 now labels dimensions as "Achieved / Partly / Saturated," and Section 4.2 has a new title. The roster shows that some deployed model achieves a behavior, but it cannot tell us why other models do not.
  • Response length does raise scores, and the top model's lead is partly due to length. The dimension-level pattern and the gap hold across length quartiles (36.5% to 33.7%). Section 4.2 and new Appendix E.9 report a length-adjusted ranking (see our response to Reviewer L2Nz).
  • Seven models were run at temperature 1 because the inference proxy does not accept lower values, and three were given higher output-token limits (Appendices E and G). Scores are very stable across the three runs (Spearman of at least 0.975 between runs), which limits how much sampling noise could matter.
  • We have narrowed the interpretation of the probe as described in our response to Reviewer VGW3.

Thanks for additional information of this paper. I would like to keep my score as 3.

Reviewer_VGW3OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DistressBench evaluates how language models respond to simulated suicide and self-harm crises. It contains 718 clinician-authored conversations spanning 23 subcategories, with 6,901 weighted criteria across seven dimensions of crisis support. Rubrics and reference responses were largely drafted by a model and subsequently revised by clinicians. The authors evaluate 25 models over three runs using a three-model judging panel selected against clinician annotations on 75 tasks. The best model satisfies 88.4% of weighted criteria, while the median reaches 71.4%. Models generally perform better on avoiding harmful or moralizing content than on de-escalation and resource provision. Additional analyses examine scoring sensitivity, judge effects, and models’ preferences between their own responses and rubric-compliant alternatives. A separate annex provides 20 long-horizon adversarial conversations with one model.

Critical strengths

The task-specific evaluation of supportive behavior is a useful contribution. The benchmark distinguishes recognition, compassionate engagement, de-escalation, and resource provision instead of treating the absence of harmful content as sufficient evidence of good crisis support. The breadth of scenarios and substantial clinician revision make this a potentially valuable resource for identifying concrete response omissions. Its clearest contribution beyond the closest work discussed in the paper is the combination of fixed expert-authored conversations and scenario-specific weighted rubrics.

The authors make a substantial effort to examine their evaluation procedure. Three response-generation runs, clinician annotations, dimension-level agreement statistics, judge-removal analyses, and a disclaimer ablation provide useful checks on the results. I particularly appreciate the decision to withdraw turn-level safety claims after the corresponding clinician labels showed poor reliability. These disclosures make the benchmark easier to assess and improve.

Critical weaknesses

The interpretation of the benchmark scores is stronger than the measurement supports. The recognition-action gap in Section 3.2 counts a response as failing when it misses at least one actionable-resource criterion. Section 4.6 then describes these responses as routing the user nowhere. These are not equivalent unless every such failure entails the absence of any appropriate referral. A response could provide a useful route to human help while missing another requested resource or detail. The final-turn scoring rule introduces a further ambiguity because assistance supplied earlier in the conversation receives no credit. Consequently, the headline 35.3% figure establishes incomplete resource-criterion satisfaction following full identification-criterion satisfaction, but does not yet establish that these users received no route to help.

The description of the overall score as the fraction of required care delivered has a related problem. The weights encode within-scenario judgments of importance, and the study validates agreement in applying those criteria rather than the adequacy of the resulting care. The authors acknowledge the absence of outcome validation, but the abstract and discussion still use a stronger interpretation. Independent clinician assessment of whether flagged conversations actually lack an adequate route to help would materially strengthen the central finding.

The experiments do not establish that the main constraint is a disagreement over standards rather than a limitation in generation capability. A large difference between the best and median models shows that some evaluated systems can produce the behavior; it does not show why another system omits it. The discrimination probe is informative, but recognizing which of two responses satisfies a disclosed criterion is different from generating an appropriate response unaided. Moreover, choosing between complete responses does not isolate agreement with the particular criterion, because the responses can differ in length, tone, and other content.

The authors’ own controls demonstrate the importance of these distinctions. Preference for the alternative rises from 10.4% to 34.0% when another model’s compliant response replaces the clinician reference, while the elicitation intervention recovers the missing criterion in only 53.2% of the selected cases. These findings support a narrower conclusion about response preferences under the probe conditions. They do not justify assigning 85% of forfeited criterion weight to a mechanism that explains the original generation failures. This matters because the capability-versus-choice interpretation is presented as a central scientific result.

Questions for authors

Among the cases contributing to the 35.3% recognition-action gap, how many provide no appropriate route to human help anywhere in the conversation, and how many provide some appropriate assistance but fail an additional criterion or omit repetition in the final turn? An independently reviewed breakdown could change my assessment of the headline finding.

Was judge performance evaluated on tasks held out from the selection of the 455 candidate panels? Appendix B appears to use the same 75 tasks for selection and reported agreement. Please clarify whether a held-out or task-level cross-validation estimate exists. Please also clarify whether the reported κ of 0.732 is against majority clinician consensus, as Section 3.3 states, or averaged against individual clinicians, as Table 5 indicates.

Does the standard-divergence result persist when the alternatives differ only in the failed behavior, with other content and length held approximately constant? If such evidence is unavailable, would the authors narrow the claim to the response preferences actually measured?

2 comments
AuthorsOpenReview ↗

Response to Reviewer VGW3 (1/2)

Thank you for a very thorough review. We agree with your main concern, that several of our interpretations went beyond what the measurements support, and we have revised the paper to match.

Q1: How many gap conversations provide no route to help? To answer this directly, we sampled 200 of the 12,824 gap conversations at random and coded each one against a fixed codebook, without knowing which model produced it (new Appendix E.10). The codebook defines four kinds of route (crisis line, emergency services, a professional, or a trusted person) and specifies what does not count. An LLM coder did the first pass, reading each conversation in full, and every route it recorded is backed by a quote we checked against the transcript.

Of 200 gap conversationsShare [95% CI]
Route to human help anywhere94.5% [90.4, 96.9]
Route in the final reply82.0% [76.1, 86.7]
Crisis line or emergency service anywhere77.0% [70.7, 82.3]
No route anywhere5.5% [3.1, 9.6]
No route adequate to the risk expressed8.0% [5.0, 12.6]

Mapping these onto the categories in your question:

  1. No appropriate route anywhere in the conversation: 5.5%, which is about 1.9% of all conversations where the model recognized the crisis.
  2. A route offered in an earlier turn but missing from the final reply: 21% of the multi-turn gap conversations.
  3. Some route offered, but another resource criterion missed: the remaining conversations. Usually the missing piece is a specific service or safety step the scenario called for.

So the data do not support our original phrase "routes the user nowhere," and we have removed it. The abstract, introduction, and Section 4.6 now give this breakdown instead. To check the LLM coding, one of the authors independently coded a stratified sample of 20 conversations without seeing the LLM's codes. After adjudication, the two agreed on whether any route was offered in 19 of 20 cases (κ = 0.88), on crisis lines or emergency services in 20 of 20, and on adequacy in 19 of 20 (κ = 0.89).

The gap measure itself does not depend on the LLM judges. On the 300 calibration conversations, computing the gap from the clinicians' majority labels gives 38.0% (37.5% to 39.6% for individual clinicians), compared with 38.6% from the panel, and the two agree on which conversations fall in the gap 98.5% of the time.

"Fraction of required care delivered." We agree and have removed this phrase. The score is now described as the share of clinician-specified, weighted criteria a reply satisfies. Our validation shows agreement in how the criteria are applied, but it does not show that the resulting care is adequate. This is now described in Section 5.1 and Appendix F.

AuthorsOpenReview ↗

Response to Reviewer VGW3 (2/2)

Q2: Held-out evaluation of the judges, and what κ = 0.732 is measured against. You are right on both counts.

  • The 0.732 is the panel's κ against each clinician individually, averaged over the three (0.737, 0.733, 0.725), as in the agreement table (Table 5 in the submission, Table 6 in the revision). Section 3.3 incorrectly described it as agreement with the consensus label, and we have fixed this. Against the majority label, the panel's κ is 0.821. We think the per-clinician figure is the fairer comparison with how well the clinicians agree with each other (0.640).
  • Selection and evaluation did use the same 75 tasks. We have now cross-validated the whole selection procedure at the task level: in each fold, we pick the best of the 455 panels on the training tasks and score it on the held-out ones. This gives κ = 0.726 with leave-one-task-out and 0.721 with five-fold cross-validation repeated 20 times (range 0.716 to 0.725), so selection inflates the estimate by only about 0.01. The selected panel is chosen in 73 of the 75 leave-one-task-out folds (new Appendix B.4, referenced in Section 3.3).

Q3: Does the standard-divergence result hold when the two replies differ only in the failed behavior? Our current design cannot fully isolate that, and we accept the narrower reading you suggest. With the data we have, we can at least look at length. Preference for the clinician reference goes up as it gets longer relative to the model's own reply, from 5.3% to 15.8% across length-ratio quintiles. Even so, on the 607 conversations where the two replies are within 20% of each other in length, models still prefer their own non-compliant reply 92.8% of the time. Tone and other content can still differ between the two replies, so this only partly answers your question. In the revision, we:

  • describe the 85% as a breakdown of probe outcomes, i.e., how often a model that can pick out the compliant reply when given the criterion still does not prefer it (Section 4.3, Table 14);
  • removed the statements that the deficit is "dominated by disagreement ... rather than by inability" and that "the binding constraint is therefore a disagreement" (Section 4.3, Discussion);
  • report the length-stratified results (Section 4.3, Appendix E.7).

We also agree with your observation that the 53.2% elicitation rate, and the jump from 10.4% to 34.0% when a model-written alternative replaces the clinician reference, both point toward this narrower interpretation.

Reviewer_L2NzOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces DistressBench, a benchmark for evaluating how language models respond to suicide and self-harm crises. It contains 718 clinician-authored conversations across 23 categories, with scenario-specific rubrics covering seven dimensions of crisis support. The authors evaluate 25 models using a three-model LLM judging panel calibrated against clinician annotations. The models are generally better at recognising distress and avoiding harmful responses than at providing de-escalation and actionable support.

Critical strengths

The benchmark evaluates concrete actions expected in crisis support. The involvement of clinicians in constructing scenarios, revising rubrics, and validating the judges strengthens the relevance of the evaluation. The evaluation covers 25 models, reports uncertainty estimates, and examines different dimensions of crisis support. The calibration of the LLM judges against independent clinician annotations and the analysis of scoring choices are particularly valuable.

Critical weaknesses

Although the rubrics are clinician-informed, it remains unclear if the higher scores consistently reflect better clinical support. The evaluation may also favour longer responses that satisfy more criteria, without necessarily being more appropriate for users in crisis. Most multi-turn scenarios contain only three user turns, with fixed user responses and evaluation focused on the final assistant turn. This limits the benchmark's ability to assess how models adapt to escalating distress or changing user needs during longer conversations.

Questions for authors

Have the authors examined whether higher rubric scores correspond to clinicians' overall judgments of response quality?

How sensitive are the results to response length, given that longer responses may satisfy more criteria?

1 comment

Thank you for the positive review and for your two questions, both of which led to new analyses in the revision.

Q1: Do higher rubric scores correspond to clinicians' overall judgments? This is a fair point. Our calibration shows that the judges apply individual criteria the way clinicians do, but it does not show that a higher overall score means a better response in a clinician's eyes. We now make this limitation explicit in Section 5.1 and Appendix F. The closest evidence we have comes from the final-turn ratings clinicians gave during calibration (new Appendix E.11). A clinician's own rubric score for a reply was higher when they said they would send that reply (0.665 vs. 0.582, AUC 0.63) and lower when they rated it unsafe (0.475 vs. 0.635). Within the same clinician and task, the correlation is 0.28. We see this as modest support at best, because the same clinician gave both labels and agreement between clinicians on these ratings was very low (κ ≈ 0.05; Appendix B.6). A proper holistic validation study is now listed first in our future work, and we have dropped the description of the score as "the fraction of required care delivered."

Q2: How sensitive are the results to response length? More than our original draft acknowledged (new Appendix E.9).

  • Within a task, the correlation between log reply length and score is 0.43. Comparing the same model's three attempts on the same task, it is 0.26.
  • The effect is largest for compassionate engagement (0.40), identification (0.31), and de-escalation (0.27), and essentially zero for harmless response (-0.04) and moralization (0.00).
  • The recognition-action gap barely changes across length quartiles (36.5% to 33.7%), so it is not driven by short replies.
  • The top model's lead does depend partly on length. Its median reply is 747 words, compared with 320 across all models, and after a simple linear adjustment for length it drops from first to seventh (Spearman 0.88 with the original ranking).

We suspect this adjustment overcorrects, since things clinicians want to see, such as a crisis line or a safety plan, also make replies longer. We therefore report the adjusted ranking as a sensitivity analysis alongside the main one, and Section 4.2 now discusses the top model's reply length.

Multi-turn depth. We agree this is a real limitation. 416 of the 419 multi-turn tasks have three user turns, and those turns are fixed. A fixed script means every model answers the same conversation, which keeps cross-model comparison clean, but it also means we cannot test how a model adapts when the user escalates in response to what it said. The revision states this scope in Section 5 and lists adaptive simulated-user evaluation as future work in Appendix F. Doing that well requires reliable per-turn labels, which our withdrawn per-turn annotation showed we do not yet have. DistressBench-LH is a first, descriptive step in that direction.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of revisions: gap breakdown, robustness checks, and narrowed claims

We thank the reviewers for careful, constructive reviews. We have uploaded a revised PDF; revised text is shown in blue, and the main text stays within 10 pages. Three concerns recur: (1) what the aggregate score and the 35.3% recognition-action gap mean clinically, (2) sensitivity to length, weights, and judge-panel choice, and (3) how far the probe supports a "choice versus capability" reading. We agree the paper's language ran ahead of the measurement in places, and the revision narrows those claims.

What the gap measures (Section 4.6, Appendix E.10). We coded 200 randomly sampled gap conversations against a fixed codebook, with model identities hidden. An LLM coder read each conversation in full and had to cite a verbatim quote for every route it recorded; an author then blind-coded a stratified subset and adjudicated disagreements. 94.5% offer some route to human help somewhere in the conversation, and 82.0% do so in the final reply. 5.5% [3.1, 9.6] offer none at all, and 8.0% offer no route adequate to the risk the user expressed. The gap therefore mostly records incomplete or generic resource guidance, and "routes the user nowhere" overstated it; that wording is removed. The gap also reproduces under clinician labels alone: on our 300 calibration conversations, clinician consensus gives 38.0% and the panel 38.6%.

Robustness.

  • Weights (Appendix E.8). Equal weighting and random weight perturbations leave the leaderboard essentially unchanged (Spearman at least 0.996, same leader).
  • Panel selection (Appendix B.4). Selecting the panel on training tasks and scoring it on held-out tasks gives kappa 0.726, against 0.732 in-sample and 0.640 among clinicians.
  • Alternative panel (Appendix B.5). A panel sharing no judge with ours gives Spearman 0.996 with our ranking, the same top five in the same order, and a recognition-action gap of 37.3% against 35.4%.
  • Length (Appendix E.9). Longer replies score higher within a task (r = 0.43), and the leader writes the longest replies. The gap and the dimension pattern hold across length quartiles, while the leader's margin does not survive a simple length adjustment. We now state this in Section 4.2.

Other changes.

  • The score is described as the share of clinician-specified, scenario-weighted criteria satisfied (Abstract, Introduction).
  • The kappa description in Section 3.3 is corrected.
  • Table 1 now reads "Achieved / Partly / Saturated."
  • The probe is presented as measuring preferences between finished replies (Section 4.3, Discussion).
  • New limitations cover holistic validity, reply length, fixed user scripts, and the gap coding (Section 5, Appendix F).
  • The AI use statement covers the LLM-assisted gap coding.

Numbering note: the clinician agreement table cited as Table 5 in the reviews is Table 6 in the revision, and the withdrawn per-turn labels are Appendix B.6.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.