acceptodds
Withdrawn from ICLR 2027

REVEAL: Evaluating Medical LLMs with Risk-Grounded Virtual Patients

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Medical LLMs are commonly evaluated with single-turn clinical questions, yet real online consultations often unfold through missing histories, delayed disclosure, corrections, and evolving risk information. We present **REVEAL**, a risk-aware virtual-patient framework for multi-turn medical LLM safety evaluation on **753 real text-only online consultations**. REVEAL builds source-extracted patient profiles and compares three patient simulators: a **prompt-only patient simulator** that conditions a patient-role prompt on the extracted profile, a **risk-hiding virtual patient** that withholds latent risk attributes unless elicited, and a **DialoguePlan-based risk-replay virtual patient** that reproduces source-derived risk-emergence trajectories. We evaluate patient fidelity before downstream safety: in a 753-case human arena with three annotators, risk-replay is selected as the closest source-style transcript in 71.2% of cases (Fleiss’ κ = 0.62). We then test five evaluated doctor models under both answer-first and history-taking settings with a rubric-based medical-risk judge. Under a source-matched but non-equivalent single-turn protocol, all three multi-turn simulators produce higher severe-failure rates than the single-turn setting (32.14–38.18% versus 22.67%), indicating complementary interaction-level safety failures. Mechanism checks show that risk-hiding and risk-replay induce distinct disclosure regimes: 3.60% versus 99.49% tracked-risk disclosure. An exploratory challenge-behavior perturbation does not increase unsafe outcomes overall, but information contradiction exposes selected boundary failures. REVEAL therefore separates three quantities that are often conflated in medical LLM evaluation: patient realism, risk-information emergence, and downstream doctor safety.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Medical LLM, Patient Simulation, Multi-turn Evaluation, Medical Safety

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_biBkOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces a dataset of patient profiles containing various risk attributes, generated from 753 real online consultation records. To support the use of this dataset, the authors designed three patient simulation strategies: a patient guided by a generic prompt, a risk-hiding patient that discloses risk information only after relevant inquiries, and a risk-replay patient that discloses information according to the risk trajectory of the original consultation. The authors evaluated five large language models under two doctor-interaction settings and found that different risk-disclosure strategies correspond to different safety outcomes, with higher severe-failure rates in the multi-turn settings than in the separately configured single-turn diagnosis control.

Critical strengths

This study investigates interactional safety risks in real-world scenarios that commonly used QA datasets cannot capture. In practice, patients may withhold risk information, a situation that medical QA datasets primarily presenting case information all at once cannot examine.

Critical weaknesses

1. Insufficient reliability and reproducibility of the dataset, patient profile extraction, and core implementation

Although the paper explicitly lists the dataset as one of its contributions, its description of the specific sources, acquisition methods, screening procedures, de-identification process, and final dataset construction for the 753 online consultations is inadequate. Patient profiles are generated using large language models, but the paper does not adequately clarify whether they underwent manual review, nor does it report the accuracy of risk-attribute extraction, manual review criteria, or inter-annotator agreement.

Meanwhile, the core risk-control mechanisms lack sufficient implementation details. The paper does not provide the specific semantic matching rules, prompts, thresholds, or procedures for handling synonymous expressions and questions involving multiple risks in risk-hiding. It also does not fully describe the DialoguePlan construction process, doctor-act classifier, or triggering conditions. At the time of this review, the anonymous repository promised by the authors still contains no usable code or complete prompts. Consequently, readers cannot determine whether the main results arise from reliable risk-control mechanisms or from undisclosed prompts and heuristic rules. This directly affects the verifiability, reproducibility, and credibility of the paper’s conclusions.

2. The safety results provide insufficient support for the proposed mechanisms

The three simulators differ in the amount and timing of information disclosure and in dialogue length. The existing experiments do not distinguish the effects of increased information disclosure from those of using DialoguePlan. Furthermore, although the authors acknowledge that the single-turn and multi-turn protocols are not equivalent, they primarily support their contribution through differences in overall severe-failure rates, without paired failure analyses for the same cases and doctor models. The evidence is therefore insufficient to establish that multi-turn evaluation reveals interaction-level safety failures missed by single-turn evaluation.

Questions for authors
  1. Table 9 presents the taxonomy used to extract risk attributes from source consultations into patient profiles, but Table 7 reports only the frequencies of specific attributes, while Table 6 uses a different set of consultation-level risk categories. Could the authors clarify how the extracted risk attributes map to the categories in Table 9 and how the classification schemes in Tables 6 and 9 relate to each other?

  2. How is the semantic matching policy in risk-hiding implemented? What criteria and thresholds determine whether a doctor’s question matches a particular elicitation intent? How does the policy handle synonymous expressions, indirect inquiries, questions involving multiple risks, and cases where the same risk already appears in the medical history?

  3. What are the implementation details of DialoguePlan?

  4. At the time of this review, the anonymous repository still contains no usable code, complete prompts, or configuration files. Could the authors provide the complete implementation and example intermediate outputs for generating profiles, running risk-hiding, constructing DialoguePlan, classifying doctor acts, and calculating disclosure rates?

Reviewer_5DX3OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper starts from a real gap: medical LLMs are evaluated with single-turn, well-formed questions, while actual online consultations unfold through missing histories, delayed disclosure and corrections. The authors build a virtual-patient framework on 753 real text-only online consultations, extract structured patient profiles from them, and instantiate those profiles in three patient simulators — a prompt-only baseline, a risk-hiding simulator that keeps risk attributes latent until they are elicited, and a DialoguePlan-based risk-replay simulator that follows source-derived risk trajectories. Fidelity is then evaluated, and on that basis risk-replay is adopted as the final test form, under which five frontier doctor models are evaluated in two doctor settings with a rubric-based safety judge. The reported conclusions concern the framework's patient realism, its handling of risk information, and downstream medical safety.

Critical strengths
  • The motivation is sound. The mismatch between single-turn benchmark evaluation and the interactional structure of real online care is a genuine evaluation problem, and treating the timing of safety-relevant information as the object of evaluation is the right instinct.

  • Grounding the simulated patients in real consultations rather than author-written personas gives the instrument a better claim to realism than most synthetic-patient evaluation work, and the three-simulator design does separate interesting information regimes (reported disclosure rates 3.60%, 52.10% and 99.49%).

Critical weaknesses

W1. Patient fidelity is not rigorously defined. The paper uses the term for at least four different measurements (transcript perplexity, average turns, clinical-term coverage, and human judgment of stylistic similarity to the source), and never states which of these constitutes fidelity or how they relate.

W2. **The one substantive design decision — choosing risk-replay as the final test form — is licensed by that undefined construct. If fidelity is not defined, the evaluation that selects among the three simulators cannot justify the selection, and the entire downstream evaluation inherits that weakness.

W3. Methodologically the framework reduces to prompt-level variants over a patient simulator plus an LLM rubric judge. Aside from the three simulator policies, there is no new mechanism; the comparison, the judging and the safety scoring are all established components. Ask: what the framework contributes that a role-prompted patient simulator with an LLM judge does not already provide.

W4. The findings are descriptive and have no theoretical support. The paper proposes to separate patient fidelity, risk-information emergence and downstream doctor safety, but never states how the three are related, offers no account of why the reported differences arise, and makes no prediction that could fail. Ask: the relation among the three axes, and the observation that would refute it.

Questions for authors
  1. Definition. What precisely is "patient fidelity" in this paper? Section 4.4 lists perplexity, average turns, clinical-term coverage and a masked human arena as fidelity evidence, while Section 5.1 reports that these do not agree in direction (the simulator with the lowest perplexity is not the one selected by the annotators). Which measurement is the definition, which are diagnostics, and what are the units and the decomposition?

  2. Theoretical status of the three axes. The paper's stated contribution is that the three quantities — fidelity, emergence, safety — should be separated. What is the claimed relation between them? For instance, does a more faithful simulator necessarily produce a stricter safety evaluation, and is there any configuration in which these three axes should diverge in a specified direction? Without such a statement there is no result that could contradict the framework.

  3. Practical consequence. If a practitioner is choosing between models, what should they do differently after reading this paper than after reading a single-turn benchmark and a role-prompted patient simulator? The paper concludes that interaction-level evaluation complements static QA, but does not say what the complement implies for a deployment decision.

Reviewer_wPrKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes an evaluation benchmark for LLM safety in medical conversations. It compares three patient simulators, uses LLM judge to evaluate 5 different LLMs.

Critical strengths
  1. The paper address an important evaluation problem about medical LLM.
  2. The development of three different simulators could be useful for future related studies.
Critical weaknesses
  1. The data source lacks transparency. Which platform were the consultations collected from? What filters were applied? Were the cases reviewed and validated by experts? The quality and characteristics of the data remain unclear, making the findings difficult to evaluate.

  2. Relatedly, it is unclear whether the dataset contains clinical ground truth, such as diagnoses or treatment information. For real clinical consultations, subsequent diagnoses and treatment decisions could provide a reference for evaluating the responses. Without this information, the reported “failure rate” does not establish actual clinical failures, because it remains unclear what the correct answer should be.

  3. The reliability of safety scoring is insufficiently validated. Final failure labels are mainly assigned by an LLM judge rather than experts. The 100-case human validation does not clearly verify the accuracy of the “severe-failure” labels themselves, and the specific red-line criteria lack transparency.

  4. The patient-fidelity evaluation is circular. Risk-replay was selected as closest to the original consultation in 71.2% of cases, but it had already used that same consultation to construct the DialoguePlan guiding its generation. Evaluating similarity against the same source raises a data-leakage concern. Greater similarity is therefore expected and does not demonstrate that the method simulates patients realistically more generally.

Questions for authors
  1. Which platform supplied the consultations, how were the 753 cases selected, and what expert validation was performed?
  2. What clinical ground truth supports the reference answers and failure labels? Are verified diagnoses or treatment records available?
  3. What are the specific red-line criteria, and how accurately does the judge reproduce expert severe-failure labels in the 100-case validation?
  4. How does the fidelity evaluation account for risk-replay using information from the same source consultation against which its output is evaluated?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.