acceptodds
Under review as a conference paper at ICLR 2027

EHR-RobustGym: Benchmarking and Training Agents for Robust Clinical Reasoning

Abstract

In hospital workflows, electronic health records (EHRs) are often noisy, and may not contain the evidence needed to confirm events or measurements referenced in a clinical query. Even when database retrieval succeeds, clinical agents can overlook such discrepancies and return plausible but unsupported answers. We introduce EHR-RobustGym, a scalable and interactive environment for evaluating and training robust clinical agents grounded in noisy EHRs. Built on MIMIC-IV hospital records (365K patients, 31 tables, and over 500M records), EHR-RobustGym comprises 5,486 Clean-Noise pairs spanning six clinical intents and both patient-level and population-level queries. The pairs test robustness to Record-level, Value-level, and Query-level noise, while interactive SQL/Python execution and outcome verification support trajectory collection and training. Evaluating multiple LLMs reveals substantial robustness gaps: average task success across proprietary and large-scale open-weight models drops from 62.2% on Clean questions to 37.9% on Noise questions. At k=4, pass^k consistency falls below 50% for most evaluated models, exposing instability in clinical task completion. Supervised fine-tuning and reinforcement learning in EHR-RobustGym improve performance, with gains generalizing to five external EHR benchmarks. Together, these results position EHR-RobustGym as a testbed for evaluating and improving the evidence-grounded robustness of clinical agents.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
electronic health records, medical agents, interactive environments, benchmarking, robustness

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_Su22OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces EHR-RobustGym, an interactive environment designed to evaluate and train the robustness of Electronic Health Record (EHR) agents. The authors address a key limitation of existing agents: their tendency to generate answers unsupported by evidence when database records are missing, field values fail to satisfy specified requirements, or query constraints cannot be met.

Built upon MIMIC-IV, the paper constructs 5,486 Clean–Noise question pairs spanning six categories of clinical tasks, three types of noise, and both patient-level and population-level queries. Clean questions have correct answers supported by the database, whereas Noise questions require agents to interact with the database, gather evidence, and appropriately abstain from answering when sufficient evidence is unavailable.

EHR-RobustGym provides a SQL/Python execution environment, interaction trajectory collection, and Outcome Verification, supporting agent training through Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), Proximal Policy Optimization (PPO), and Group Relative Policy Optimization (GRPO).

Experiments evaluating 12 large language models (LLMs) reveal substantial performance degradation under Noise conditions. The authors further demonstrate that supervised fine-tuning and reinforcement learning based on successful interaction trajectories can significantly improve task success rates, with transfer gains also observed across five external EHR benchmarks.

Critical strengths
  1. The paper addresses an important limitation of existing EHR agents: their ability to determine whether database evidence actually satisfies query requirements. The Noise tasks require agents to collect evidence through tool interactions to justify abstention, making the setting more realistic and clinically relevant.
  2. The Clean–Noise paired design changes only one necessary condition while preserving the original clinical intent, enabling controlled comparisons. The authors also conduct SQL-based validation, LLM review, and manual inspection. Notably, 344 challenging test pairs were manually reviewed, with 92.15% found valid before correction and the remaining 27 pairs revised.
  3. The paper evaluates 12 LLMs and multiple training methods, including SFT, DPO, PPO, and GRPO, along with external benchmark evaluations. Additionally, the analysis of 12,096 agent trajectories across four error categories provides useful insights into agent failure modes.
Critical weaknesses
  1. 850 out of 2,000 demonstrations are generated with teacher-only reference explanations and queries. However, this may introduce issues in the teacher-generated reasoning and action trajectories since the teacher is input with additional information. Even if the guidance is removed from student inputs, directly training students on these trajectories could lead to problems such as hallucinations. Additionally, the authors should provide examples of the actual SFT training traces.

  2. Also, the authors should conduct some experiments with the coding agent harness. Currently, it seems that the harness used in this paper only includes code execution. The authors could evalaute e.g., codex + gpt, claude code + opus/sonnet, qwen + pi.

  3. The authors can include some of the latest frontier models, such as GPT-6 Astra and Claude Opus 5.5, to assess the difficulty of the tasks in this paper. Moreover, many of the evaluated open-weight models in the paper are relatively outdated and exhibit low baseline success rates, because these open-source models in this paper have few agentic capabilities. Including newer open-weight models, such as Qwen3.8-27B, would help establish the benchmark's practical value for current agentic systems.

  4. The improvements achieved through SFT and reinforcement learning may result from better tool-use capabilities rather than stronger evidence-grounded reasoning alone. This concern is particularly relevant for smaller models with near-zero baseline success rates, which may struggle with basic SQL/Python execution and database interactions. Although the paper provides tool-call statistics and error analysis, these do not fully disentangle the two factors. A comparison with models trained on general EHR tool-use trajectories, without Clean–Noise supervision, would help isolate the contribution of robustness-specific training.

Questions for authors

see the weaknesses.

5 comments
AuthorsOpenReview ↗

Response to Reviewer Su22 (Part 1/3)

Dear Reviewer Su22,

Thank you for recognizing the clinical relevance of requiring agents to gather evidence and justify abstention, and the controlled Clean–Noise paired design that changes one necessary condition while preserving clinical intent. We appreciate your positive assessment of the SQL-based validation, LLM review, and manual quality checks, as well as the broad evaluation across models, training methods, and external benchmarks and the insights from trajectory-based failure analysis. We address your concerns below by clarifying teacher guidance and SFT quality checks, providing actual SFT examples, evaluating coding-agent harnesses and newer models, and examining robustness gains beyond basic tool use.

[W1\textcolor{#B91C1C}{\textbf{W1}}] Teacher guidance and SFT trajectory quality

W1. 850 out of 2,000 demonstrations are generated with teacher-only reference explanations and queries. However, this may introduce issues in the teacher-generated reasoning and action trajectories since the teacher is input with additional information. Even if the guidance is removed from student inputs, directly training students on these trajectories could lead to problems such as hallucinations. Additionally, the authors should provide examples of the actual SFT training traces.

All 850 reference-guided demonstrations had already been checked after removing teacher-only guidance and before student training and the original submission to verify that their reasoning and conclusions remained self-contained and supported by student-visible tool evidence. Actual Clean and Noise SFT examples, including reference-guided demonstrations, are now included in the supplementary material. The revised Appendix D.1 describes the original quality-control procedures in greater detail.

(1) Reference guidance identifies the clinical requirements and evidence to retrieve. Reference explanations and queries guide the teacher’s reasoning and database interactions during demonstration construction. ReTool [1] uses existing reasoning to construct trajectories with code calls and execution feedback, followed by answer verification. GenRM [2] uses reference solutions to generate verification rationales, filters them by verification correctness, and removes the references before verifier fine-tuning. In our setting, guided trajectories must still support their conclusions using the original question, database schema, and actual tool feedback available to the student.

(2) Guided trajectories were checked for self-containment before student training. As described in Appendix D.1 of the submitted manuscript, the 850 reference-guided demonstrations underwent naturalness and self-containment checks by Gemini-3.1-Pro after teacher-only reference explanations and queries were removed. These checks required reasoning and conclusions to follow from the original question, database schema, executed code, and preceding tool feedback. Trajectories could not report database observations before the corresponding tool output or use teacher guidance as evidence.

(3) Question validation and demonstration selection. Training questions are checked through reference SQL execution and model review (Appendix A.3). The demonstration dataset contains 1,150 unguided and 850 reference-guided trajectories selected through rejection sampling before student training. Accepted trajectories must pass task evaluation and support their answers or abstentions with evidence obtained through actual tool interaction.

(4) As requested, we have added actual student-visible SFT examples to the supplementary material. These cover Clean and Noise tasks, including demonstrations generated with reference guidance. Each example shows the question, tool calls, tool feedback, and final answer visible to the student after teacher-only guidance is removed.

AuthorsOpenReview ↗

Response to Reviewer Su22 (Part 2/3)

[W2\textcolor{#B91C1C}{\textbf{W2}}] Evaluation with coding-agent harnesses

W2. Also, the authors should conduct some experiments with the coding agent harness. Currently, it seems that the harness used in this paper only includes code execution. The authors could evalaute e.g., codex + gpt, claude code + opus/sonnet, qwen + pi.

Thank you for suggesting these three model–harness comparisons. Codex, Claude Code, and Pi each achieve higher overall success than their corresponding baselines in the native model comparisons (Table R1). Our original interface lets agents issue SQL/Python queries through a common JSON action format, inspect the results, and query again. Codex, Claude Code, and Pi use the same clinical task instructions and EHR tool, with each harness adding its own system instructions and native tool interface. We evaluated each combination under the same limits as its corresponding baseline, with up to 65,536 input tokens, 4,096 output tokens per response, and 20 model responses per question (Appendix C.1, Table 9).

Table R1. Native coding-agent harness evaluations and corresponding baselines (%).

Model and harnessClean SRNoise SROverall SR
GPT-5.4 (xhigh)78.2755.7567.01
GPT-5.4 (xhigh) + Codex86.2156.3571.28
Sonnet 4.6 (thinking disabled)62.5026.2944.39
Sonnet 4.6 (thinking disabled) + Claude Code84.6233.4359.03
Qwen3.5-9B (thinking disabled)28.2727.0827.68
Qwen3.5-9B (thinking disabled) + Pi53.4739.1946.33

All three harnesses improved success more on Clean questions than on Noise questions. For GPT-5.4, Codex raised Clean SR by 7.94 percentage points and Noise SR by 0.60 points (Appendix E.7, Table E.5).

[W3\textcolor{#B91C1C}{\textbf{W3}}] Task difficulty for newer models

W3. The authors can include some of the latest frontier models, such as GPT-6 Astra and Claude Opus 5.5, to assess the difficulty of the tasks in this paper. Moreover, many of the evaluated open-weight models in the paper are relatively outdated and exhibit low baseline success rates, because these open-source models in this paper have few agentic capabilities. Including newer open-weight models, such as Qwen3.8-27B, would help establish the benchmark's practical value for current agentic systems.

The benchmark remains challenging for newer frontier and open-weight models, especially when abstention must be justified by evidence. GPT-5.6 Sol (xhigh) achieves 81.0% Clean SR but 32.2% Noise SR and 26.4% Pair SR. Qwen3.8-27B (thinking disabled) achieves 50.6% Clean SR but 11.1% Noise SR (Table R2). These results establish that the difficulty extends beyond models with near-zero baseline success. Strong supported-query performance does not ensure reliable handling of unsupported counterparts. The additional results and analysis are included in Appendix E.6, Table E.3.

Table R2. Task success and pair-level success for newer models (%).

ModelClean SRNoise SRPair SR
GPT-5.6 Sol (xhigh)81.032.226.4
GPT-5.5 (xhigh)58.934.825.9
GLM-5.2 (max)66.631.724.1
Kimi K3 (max)59.021.615.0
Qwen3.8-27B (thinking disabled)50.611.15.9
AuthorsOpenReview ↗

Response to Reviewer Su22 (Part 3/3)

[W4\textcolor{#B91C1C}{\textbf{W4}}] Distinguishing tool-use gains from evidence-grounded robustness

W4. The improvements achieved through SFT and reinforcement learning may result from better tool-use capabilities rather than stronger evidence-grounded reasoning alone. This concern is particularly relevant for smaller models with near-zero baseline success rates, which may struggle with basic SQL/Python execution and database interactions. Although the paper provides tool-call statistics and error analysis, these do not fully disentangle the two factors. A comparison with models trained on general EHR tool-use trajectories, without Clean–Noise supervision, would help isolate the contribution of robustness-specific training.

Clean–Noise supervision yields stronger evidence-grounded robustness than training on general answerable-EHR trajectories. Clean-only uses general EHR tool-use trajectories without Noise or abstention demonstrations. These are successful SQL/Python trajectories for answerable questions in our training set. Starting from the same Qwen3.5-9B Base model, Mixed SFT improves Noise SR by 48.5 percentage points and Pair SR by 36.1 points over Clean-only SFT, while Clean SR decreases from 67.26% to 59.23%. This comparison demonstrates the additional value of robustness supervision beyond answerable-EHR training. The full ablation and results across model scales are included in Appendix E.3, Tables E.1 and E.2, and our response to Reviewer u1NZ [W2\textcolor{#B91C1C}{\textbf{W2}} & Q2\textcolor{#B45309}{\textbf{Q2}}], Tables R2(a) and R2(b).

(1) The Clean-only comparison demonstrates the additional value of Noise supervision. We use Qwen3.5-9B for its stronger baseline agentic tool-use capabilities. Clean-only, Noise-only, and Mixed SFT each use 250 updates and 1,000 sample exposures, while the number of unique demonstrations and token counts differ. Clean-only improves supported-query accuracy but reduces Noise and Pair SR below Base. Mixed improves all three metrics over Base and achieves the highest Pair SR, while Noise-only over-abstains on answerable questions. The original Qwen3-8B composition ablation yields consistent findings (Section 4.3, Figure 4(b), and Appendix E.3).

(2) Robustness gains also occur without higher tool-call success. We extend training experiments to Qwen3.5-9B, Qwen3.5-35B-A3B, and Qwen3.5-122B-A10B. For Qwen3.5-9B, GRPO increases Noise SR from 60.6% to 68.4%, while Noise tool-call SR decreases from 78.5% to 72.9%. Tool-call SR measures successful execution among issued calls. It does not establish whether the resulting evidence supports the final answer or abstention.

(3) Unsupported eICU queries provide direct evidence of robustness transfer. On 96 additional patient-level unsupported eICU queries, Qwen3-14B’s success increases from 7.3% to 37.5% after SFT while the overall abstention rate remains almost unchanged (Appendix E.8, Table E.6). This supports transfer of better-supported abstention beyond more frequent abstention. GRPO does not provide a further overall gain, achieving 35.4%. Evidence validity among submitted abstentions improves, but more interactions terminate without a final response.

References

[1] Feng et al. ReTool: Reinforcement Learning for Strategic Tool Use in LLMs. ICLR 2026.

[2] Zhang et al. Generative Verifiers: Reward Modeling as Next-Token Prediction. ICLR 2025.

Reviewer_7Z4SOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

EHR-RobustGym introduces an interactive benchmark for evaluating clinical agents on 5,486 Clean-Noise pairs from MIMIC-IV using SQL/Python interactions and trajectory-based training. It evaluates 12 models**,** including GPT-5.4, GPT-4.1, Gemini 3.1 Pro, Claude Sonnet 4.6, Kimi K2.5, DeepSeek V3.2, Qwen3-235B, four smaller Qwen3 models (4B to 32B), and Llama 3.1 8B. Average success across the seven API-based models drops from 62.2% on Clean to 37.9% on Noise questions, with improvements shown through SFT and RL training.

I find the evidence-supported abstention requirement particularly useful, as it avoids rewarding NULL responses without verification, which is a problem we have seen in other clinical agent benchmarks. The interactive environment also allows examining actual tool trajectories. However, the noise is primarily introduced through query modifications rather than actual EHR inconsistencies, and the LLM-based abstention verifier needs stronger independent validation. This raises some questions about how well the results reflect broader clinical robustness. Nevertheless, the benchmark makes a useful contribution to clinical agent evaluation, supporting a weak acceptance.

Critical strengths

The interactive SQL/Python environment enables collection of actual tool trajectories rather than relying only on generated answers. This is valuable for future training and failure analysis.

The focus on whether agents can distinguish answerable questions from those lacking sufficient evidence.

Critical weaknesses

The noise is primarily created by modifying query conditions rather than introducing actual inconsistencies in EHR records. Also, abstention validity depends on an LLM judge which is not sufficiently independently validated. This raises questions on how much of the measured performance reflects genuine clinical reasoning versus satisfying the evaluation criteria.

Questions for authors

How representative are the Clean-Noise pairs of real EHR data quality issues?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 7Z4S (Part 1/2)

Dear Reviewer 7Z4S,

Thank you for recognizing the usefulness of evidence-supported abstention, particularly the requirement to verify whether abstention is supported by clinical evidence, and the importance of distinguishing answerable questions from those lacking sufficient evidence. We also appreciate your positive assessment of the interactive SQL/Python environment, which provides actual tool trajectories for future training and failure analysis. We explain which real EHR problems the paired questions capture, then report how we validated the abstention verifier.

[W1.1\textcolor{#B91C1C}{\textbf{W1.1}} & Q1\textcolor{#B45309}{\textbf{Q1}}] Real-world grounding and representativeness

W1. The noise is primarily created by modifying query conditions rather than introducing actual inconsistencies in EHR records.

Q1. How representative are the Clean-Noise pairs of real EHR data quality issues?

Real hospital EHRs already contain gaps and inconsistencies. We construct queries that expose their consequences for clinical answers. An evaluation that asks only about events with a documented answer can overlook these problems. We therefore change one condition in the question while keeping the clinical goal and the underlying MIMIC-IV records unchanged.

(1) The records and clinical information needs come from practice. MIMIC-IV contains hospital records for approximately 365,000 patients. Its documentation gives a concrete example of incomplete linkage. Some laboratory results recorded during a hospital stay lack a stored admission identifier. Our clinical questions draw on information needs identified by medical experts reviewing discharge summaries in DiSCQ and by 222 hospital staff surveyed for EHRSQL.

(2) Changing the question exposes a distinction already present in the chart. In one admission, the first insulin entry records an administered dose. The second is marked “Not Given per Sliding Scale” and has no administered dose. Asking for the dose recorded in the first insulin entry tests retrieval. Changing first to second tests whether the agent recognizes why it must abstain, even though it can retrieve a plausible dose from a later entry. EHRSQL includes questions beyond the database schema or requiring external knowledge, and EHR-ChatQA studies ambiguity and terminology mismatches. Here, both questions are unambiguous and fit the schema. The record for the requested event determines whether a dose can be reported.

(3) The test set covers three kinds of evidence gaps. Of the 1,008 test pairs, 336 test whether a required record or event is available (Record-level), 336 test whether a field or recorded state supports the request (Value-level), and 336 test whether records satisfy conditions such as event order or encounter identity (Query-level). Each group contains 168 patient-level and 168 population-level pairs. These counts describe coverage across six clinical intents.

AuthorsOpenReview ↗

Response to Reviewer 7Z4S (Part 2/2)

[W1.2\textcolor{#B91C1C}{\textbf{W1.2}}] Independent outcome verification

W1. Also, abstention validity depends on an LLM judge which is not sufficiently independently validated. This raises questions on how much of the measured performance reflects genuine clinical reasoning versus satisfying the evaluation criteria.

Our Noise score measures whether an agent can use clinical evidence from its own tool interactions to establish why a particular request requires abstention. The verifier’s judgments of whether that evidence supports abstention closely agree with independent human assessments.

(1) Clinical reasoning demonstrated in the trajectory. The verifier examines the SQL/Python calls in the order the agent executed them, the results the agent actually saw, and the explanation accompanying its final answer. To accept an abstention, the verifier must identify the tool feedback that establishes which requirement of the question is unmet. For example, two agents may both abstain when asked for the administered dose at the second insulin entry. An agent that retrieves that entry, observes “Not Given per Sliding Scale,” and explains why no administered dose can be reported earns credit. An abstention backed only by a failed query or a dose from a later entry earns none.

(2) Human validation directly tests whether the verifier recognizes evidence-supported abstention. All 400 Noise trajectories in the validation set contained executed tool calls and ended in explicit abstention. They covered Base, SFT, and RL models, both query scopes, and all three Noise types. Two annotators with medical AI and EHR analysis experience first checked each question and reference outcome, then independently assessed the visible tool evidence without seeing the model identity or judge decision. Their initial agreement was 98.0% (κ = 0.960). On the adjudicated set of 200 valid and 200 invalid abstentions, GPT-5.2 (high) achieved 98.00% agreement with human judgments (κ = 0.960), compared with 98.25% for GPT-5.4 (high) and 96.00% for Gemini-3.1-Pro. GPT-5.2 accepted 2.00% of human-invalid abstentions and rejected 2.00% of human-valid ones. We retain the original GPT-5.2 configuration for evaluation and online RL because it combines high human agreement with faster, more stable service in our setting (Appendix C.3, Table C.1).

AuthorsOpenReview ↗

Follow-Up on Review and Feedback

Dear Reviewer 7Z4S,

Thank you for your positive recommendation and for recognizing EHR-RobustGym as a useful contribution to clinical agent evaluation. We appreciate your comments on evidence-supported abstention and the value of SQL/Python trajectories for training and failure analysis. We keep the MIMIC-IV records unchanged and use paired questions to show when existing gaps or inconsistencies leave a clinical request unsupported. We also compared the verifier’s judgments with independent human assessments of whether the agents’ tool results justified abstention.

We would be glad to hear your thoughts on our rebuttal. If any questions or concerns remain, please let us know.

Best regards,
The Authors

Reviewer_EdeoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces EHR-RobustGym, an interactive MIMIC-IV environment comprising 5,486 Clean-Noise question pairs across six clinical intents to evaluate and train clinical database agents on both answerable requests and evidence-grounded abstention under Record-, Value-, and Query-level noise. Experiments across eleven proprietary and open-weight models document a substantial drop in accuracy from Clean to Noise queries, and the authors further evaluate SFT and RL post-training (DPO, PPO, GRPO) on Qwen3 and Llama-3.1 backbones.

Critical strengths
  • Formulating clinical database robustness through paired Clean and Noise requests across Record-, Value-, and Query-level mismatches addresses an important and practical failure mode where clinical agents produce unsupported answers from incomplete or noisy EHR tables.
  • Providing an executable SQL and Python environment over MIMIC-IV with multi-turn trajectory collection offers a valuable testbed for studying tool-using clinical agents and their abstention behavior.
  • Evaluating eleven diverse frontier and open-weight models alongside multiple post-training paradigms (SFT, DPO, PPO, GRPO, and self-improvement) provides a broad empirical reference for the community.
Critical weaknesses
  • Splitting the 2,000 training questions across Stage 1 (SFT-250, 1,000 shuffled questions) and Stage 2 (RL and self-improvement, remaining 1,000 questions) appears to split individual questions rather than Clean-Noise pairs (Section 4.3, p. 7; Appendix D.1, p. 25), meaning Stage 2 rollouts may operate on paired counterparts whose underlying patient records and SQL structure were already encountered during SFT.
  • Human quality auditing in Appendix B (p. 20, Table 8) only inspects the 344 test pairs where both GPT-5.4 and Gemini-3.1-Pro failed (finding benchmark issues in 7.85%, 27/344), leaving the remaining test and training pairs unaudited and tying test-set filtering to two of the evaluated models.
  • Outcome verification for Noise abstention justifications and RL rewards relies on a single LLM judge (GPT-5.2, Section 4.1, p. 6; Appendix C.3, p. 22) without reporting agreement against human clinical annotators, which makes it harder to gauge reward reliability during RL training.
Questions for authors
  • Could the authors clarify whether Section 4.1 (p. 6, which describes GPT-5.2 verifying both Clean and Noise requests) or Appendix C.3 (p. 22, which states Clean questions use deterministic string matching) reflects the actual evaluation pipeline, and report human-judge agreement on Noise verification?
  • How would Stage 2 RL performance change if Clean-Noise pairs were kept strictly pair-disjoint between the SFT and RL training splits, and what drives the RL performance drops on Qwen3-32B (GRPO) and Llama-3.1-8B (PPO) in Table 3?
  • Why were Base models more accurate on Noise queries in the structure-disjoint split than in the in-domain split (e.g., 62.7% vs. 48.8% for GPT-5.4 in Table 16), and how would Qwen3-14B trained with PPO perform on the external transfer benchmarks in Figure 6(a)?
5 comments
AuthorsOpenReview ↗

Response to Reviewer Edeo (Part 1/3)

Dear Reviewer Edeo,

Thank you for recognizing the practical importance of addressing unsupported answers from incomplete or noisy EHR tables through paired Clean–Noise requests across Record-, Value-, and Query-level mismatches. We appreciate your positive assessment of the executable SQL/Python environment over MIMIC-IV and multi-turn trajectory collection as a valuable testbed for studying clinical agents and their abstention behavior, as well as the broad empirical reference provided by our evaluations across frontier and open-weight models and multiple post-training methods. Below, we address your concerns about training splits, data quality, and outcome verification through additional experiments and human review, and clarify the observed RL performance decreases, structure-disjoint Base results, and PPO transfer to external EHR benchmarks.

[W1\textcolor{#B91C1C}{\textbf{W1}} & Q2.1\textcolor{#B45309}{\textbf{Q2.1}}] Clean–Noise pair separation between SFT and RL

W1. Splitting the 2,000 training questions across Stage 1 (SFT-250, 1,000 shuffled questions) and Stage 2 (RL and self-improvement, remaining 1,000 questions) appears to split individual questions rather than Clean-Noise pairs (Section 4.3, p. 7; Appendix D.1, p. 25), meaning Stage 2 rollouts may operate on paired counterparts whose underlying patient records and SQL structure were already encountered during SFT.

Q2. How would Stage 2 RL performance change if Clean-Noise pairs were kept strictly pair-disjoint between the SFT and RL training splits […]

Stage 2 RL still improves both Clean and Noise success when Clean–Noise pairs are kept strictly pair-disjoint between SFT and RL. Starting from the retrained Qwen3-14B SFT baseline of 45.0% Clean SR and 42.9% Noise SR, PPO reaches 59.5% and 48.5%, while GRPO reaches 58.4% and 45.0%, respectively (Table R1). The gains are smaller than under question-level splitting, but persist without paired counterparts spanning the two training stages.

Training allocation. The original question-level split is shared across models and optimization methods to support consistent comparisons. Under this split, 50.8% of pairs remain within one stage and 49.2% span SFT and Stage 2. Following your suggestion, we assign 500 complete Clean–Noise pairs to SFT and the remaining 500 pairs to Stage 2. We retrain the SFT model on its assigned pairs before applying PPO or GRPO on the Stage 2 pairs.

Table R1. Qwen3-14B results under question-level and strict pair-level separation (%).

Training splitTraining stageClean SRNoise SR
By questionSFT42.434.7
By questionSFT + PPO70.462.6
By questionSFT + GRPO67.954.8
By pairSFT45.042.9
By pairSFT + PPO59.548.5
By pairSFT + GRPO58.445.0
AuthorsOpenReview ↗

Response to Reviewer Edeo (Part 2/3)

[Q2.2\textcolor{#B45309}{\textbf{Q2.2}}] Performance decreases after GRPO and PPO

Q2. […] and what drives the RL performance drops on Qwen3-32B (GRPO) and Llama-3.1-8B (PPO) in Table 3?

The two RL performance drops have distinct observed failure patterns. Qwen3-32B with GRPO more often exhausts the available budget without a final response, whereas Llama-3.1-8B with PPO deteriorates in tool execution and abstention validity. For Qwen3-32B, budget-limited termination on Noise questions rises from 3.8% to 33.3%. For Llama-3.1-8B, evidence validity among abstentions falls from 66.3% to 34.4%. More interaction or more frequent abstention therefore does not necessarily translate into higher task success. These analyses are included in Appendix E.8.

(1) Qwen3-32B’s longer interactions after GRPO increase the risk of exhausting the available budget. Prior comparisons have observed longer responses from larger variants of several dense model families [1], suggesting that model scale can be relevant to generation length. In our experiments, GRPO increases Qwen3-32B’s average interaction length on Noise questions from 4.1 to 12.8 turns, while the proportion terminating without a final response because of interaction or input limits rises from 3.8% to 33.3%. Although evidence validity among submitted abstentions improves from 58.5% to 83.5%, Noise success decreases from 40.3% to 37.7%. Clean success increases from 48.8% to 62.8%. These results suggest that extended interaction can limit overall Noise performance when the model does not complete evidence checking and submit a final response within the available budget.

(2) Llama-3.1-8B abstains more often after PPO, but most abstentions lack sufficient evidence. Between the first and final 50 updates, mean training reward decreases from 0.440 to 0.273, while tool-call errors increase from 50.8% to 67.5%. All four Qwen models increase training reward and reduce tool-call errors under the same PPO settings. On Noise questions, Llama’s abstention rate increases from 48.0% to 67.3%, but evidence validity falls from 66.3% to 34.4%. More frequent abstention therefore fails to compensate for weaker tool execution and evidence support. This model-dependent behavior under shared settings is consistent with prior findings that PPO performance is sensitive to training choices [2].

[W2\textcolor{#B91C1C}{\textbf{W2}}] Human review of test pairs and validation of training data

W2. Human quality auditing in Appendix B (p. 20, Table 8) only inspects the 344 test pairs where both GPT-5.4 and Gemini-3.1-Pro failed (finding benchmark issues in 7.85%, 27/344), leaving the remaining test and training pairs unaudited and tying test-set filtering to two of the evaluated models.

Human review extends beyond the original model-based selection through random review of the remaining test pairs. Training questions are checked through reference SQL execution and model review, and teacher demonstrations are selected through rejection sampling before student training. The original review of 344 challenging test pairs is reported in Appendix B, Table 8, and the training procedures in Appendices A.3 and D.1.

(1) The original audit includes successfully answered questions, and corrected pairs remain in the benchmark. A pair was selected when at least one of its questions was answered incorrectly by both GPT-5.4 and Gemini-3.1-Pro. Both questions were then reviewed. The 344 pairs therefore contain 688 questions, including 298 answered correctly by at least one screening model. Two annotators reviewed the pairs independently, achieving 98.26% agreement before adjudication. We corrected and retained the 27 affected pairs, corresponding to 7.85%, and repeated the affected evaluations for all agents.

(2) Random sampling provides quality evidence beyond the original selection. Two annotators with medical backgrounds reviewed 100 randomly sampled Clean–Noise test pairs from the remaining test set and identified no errors.

(3) Question validation and demonstration selection. Training questions are checked through reference SQL execution and model review. All 2,000 retained teacher demonstrations are selected through rejection sampling before student training. Accepted trajectories must pass task evaluation and support their answers or abstentions with evidence obtained through actual tool interaction.

AuthorsOpenReview ↗

Response to Reviewer Edeo (Part 3/3)

[W3\textcolor{#B91C1C}{\textbf{W3}} & Q1\textcolor{#B45309}{\textbf{Q1}}] Outcome verification and human agreement

W3. Outcome verification for Noise abstention justifications and RL rewards relies on a single LLM judge (GPT-5.2, Section 4.1, p. 6; Appendix C.3, p. 22) without reporting agreement against human clinical annotators, which makes it harder to gauge reward reliability during RL training.

Q1. Could the authors clarify whether Section 4.1 (p. 6, which describes GPT-5.2 verifying both Clean and Noise requests) or Appendix C.3 (p. 22, which states Clean questions use deterministic string matching) reflects the actual evaluation pipeline, and report human-judge agreement on Noise verification?

Appendix C.3 describes the actual evaluation pipeline. Clean questions use deterministic string matching, and GPT-5.2 verifies the evidence supporting Noise abstention. GPT-5.2 (high) has been independently validated against adjudicated human judgments on 400 agent-generated Noise trajectories containing executed tool calls and ending in explicit abstention, achieving 98.00% agreement and Cohen’s κ of 0.960. We have corrected Section 4.1 to state this distinction explicitly and included the human validation in Appendix C.3.

(1) Independent human assessment supports the Noise verifier’s reliability. We audit 400 agent-generated Noise trajectories that end in an explicit abstention and include executed tool calls, spanning Base, SFT, and RL models, both query scopes, and all three Noise types. Two annotators with medical AI backgrounds and EHR analysis experience assess the evidence without seeing model identities or judge decisions, achieving 98.0% agreement before adjudication (κ = 0.960).

(2) We compare three judges against adjudicated human labels. Agreement with adjudicated human labels is 98.25% for GPT-5.4 (high), 98.00% for GPT-5.2 (high), and 96.00% for Gemini-3.1-Pro. GPT-5.2 has a 2.00% false-acceptance rate among human-invalid abstentions and a 2.00% false-rejection rate among human-valid abstentions. We retain the original GPT-5.2 configuration for evaluation and online RL because it combines high human agreement with more stable service and faster responses in our setting.

[Q3.1\textcolor{#B45309}{\textbf{Q3.1}}] Higher Base performance on the structure-disjoint Noise split

Q3. Why were Base models more accurate on Noise queries in the structure-disjoint split than in the in-domain split (e.g., 62.7% vs. 48.8% for GPT-5.4 in Table 16) […]

Structural separation does not imply lower Base performance because the Base models have not undergone our SFT or RL training. Here, “structure-disjoint” is defined relative to the construction programs used in that downstream training. We exclude six construction programs to evaluate whether trained agents can generalize clinical reasoning and evidence-grounded abstention to new task structures. For GPT-5.4, most of the Noise success gap comes from patient-level questions (54.0% in-domain versus 77.0% structure-disjoint). Within patient-level tasks, in-domain Query-level Noise includes longer chains of diagnosis, laboratory, and medication conditions, while the structure-disjoint program checks ICU care-unit and lactate conditions across admissions. The corresponding success rates are 26.2% and 56.0%, respectively.

[Q3.2\textcolor{#B45309}{\textbf{Q3.2}}] PPO performance on external EHR benchmarks

Q3. […] and how would Qwen3-14B trained with PPO perform on the external transfer benchmarks in Figure 6(a)?

We appreciate your suggestion to evaluate PPO on the external EHR benchmarks. Qwen3-14B + SFT + PPO achieves the highest success rate among the compared variants on all five external EHR benchmarks, averaging 38.7% versus 27.4% for SFT (Table R2). These results support transfer of learned EHR task capabilities and are included in Section 4.3 and Figure 6(a). They are also consistent with PPO’s leading Clean and Noise performance among the Qwen3-14B variants in Table 3.

Table R2. External EHR task success rates (%).

External benchmarkBaseSFTSFT + GRPOSFT + PPO
TREQS35.446.851.252.5
EHRSQL (eICU)30.224.532.137.2
EHRSQL (MIMIC-III)31.436.647.752.3
EHR-SeqSQL22.924.637.241.6
EHR-Complex6.84.69.09.8
Average25.327.435.438.7

References

[1] Xu, Z., Jiang, F., Niu, L., Lin, B. Y., and Poovendran, R. Stronger Models are Not Always Stronger Teachers for Instruction Tuning. NAACL, 2025.

[2] Xu, S., Fu, W., Gao, J., et al. Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study. ICML, 2024.

Reviewer_u1NZOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces EHR-RobustGym, an interactive benchmark and training environment for evaluating evidence-grounded robustness in EHR agents. It constructs 5,486 Clean–Noise pairs across six clinical intents and three noise types by modifying query conditions while keeping the underlying EHR data unchanged. The environment supports SQL/Python execution, evidence-grounded abstention verification, and trajectory-based training. Experiments across 12 LLMs reveal substantial performance gaps between Clean and Noise queries, while SFT and RL improve task success, with additional gains reported on five external EHR benchmarks.

Critical strengths
  1. The paper addresses a meaningful challenge in EHR agents when available evidence does not support a clinical query. Evaluating evidence-grounded abstention alongside answer correctness is important for developing reliable clinical agents.
  2. The Clean–Noise paired design provides a clear controlled setting for evaluating answerability, while interactive SQL/Python execution and evidence-based verification support both benchmarking and training.
  3. The experiments cover diverse models, training methods, and external EHR benchmarks, demonstrating the practical utility of the proposed environment.
Critical weaknesses
  1. Limited novelty. The benchmark constructs unsupported queries by modifying query conditions without perturbing the underlying EHR data. Thus, it primarily evaluates evidence-grounded answerability rather than robustness to noisy clinical records more broadly. While the paper provides a clear and systematic formulation of this problem, prior work has already explored unanswerable EHR queries, query mismatches, and abstention. The paper could better articulate what additional insights or capabilities its unified benchmark enables beyond these existing approaches.
  2. Insufficient evidence for robustness-specific training gains. Training is limited to relatively small models (4B–32B) with very low baseline success rates, making it difficult to distinguish improvements in general EHR querying and tool use from genuine robustness gains. Additional DPO/iDPO stages show inconsistent benefits, while GRPO provides limited additional improvements over SFT on general and medical benchmarks. Furthermore, gains on external EHR benchmarks do not directly establish the generalization of robustness to unsupported queries.
Questions for authors
  1. Given the Clean–Noise paired design, could the authors report pair-level success rates to assess whether agents can both answer supported queries and correctly abstain from their unsupported counterparts?
  2. To what extent do the training improvements reflect enhanced robustness rather than basic EHR querying and tool-use capabilities? Could the authors provide comparisons against training on standard EHR tasks with a similar training budget?
  3. Has the LLM-based outcome verifier been independently validated against human judgments on agent-generated trajectories, particularly for evidence-grounded abstention?
7 comments
AuthorsOpenReview ↗

Response to Reviewer u1NZ (Part 1/4)

Dear Reviewer u1NZ,

Thank you for recognizing the importance of evaluating evidence-grounded abstention alongside answer correctness for reliable clinical agents. We appreciate your positive assessment of the controlled Clean–Noise paired design, the interactive SQL/Python execution and evidence-based verification supporting both benchmarking and training, and the practical utility demonstrated by our experiments across models, training methods, and external EHR benchmarks. We address your concerns below by clarifying our distinctions from prior work and providing additional pair-level results, evidence for robustness-specific training gains and transfer, and independent human validation of the outcome verifier.

[W1\textcolor{#B91C1C}{\textbf{W1}}] Novelty and distinctions from prior work

W1. Limited novelty. The benchmark constructs unsupported queries by modifying query conditions without perturbing the underlying EHR data. Thus, it primarily evaluates evidence-grounded answerability rather than robustness to noisy clinical records more broadly. While the paper provides a clear and systematic formulation of this problem, prior work has already explored unanswerable EHR queries, query mismatches, and abstention. The paper could better articulate what additional insights or capabilities its unified benchmark enables beyond these existing approaches.

EHR-RobustGym is the first hospital-scale gym for evaluating and training robust clinical agents under real-world EHR noise, while existing EHR work has overlooked the importance of this noise for agent robustness. Real hospital records already contain gaps and inconsistencies. An evaluation limited to answerable questions can overlook cases where a request assumes something the chart does not support. We therefore keep the MIMIC-IV records and clinical intent fixed and change one necessary condition in the question. This lets us test whether the same agent can answer the supported request and use retrieved evidence to justify abstention on its counterpart.

(1) A retrieved value must satisfy the clinical conditions in the request. Our tasks draw on questions raised during chart review in DiSCQ and on hospital workflows documented in EHRSQL. Both questions in each pair fit the database schema, and both reference SQL queries execute successfully. The clinical challenge is to determine whether the retrieved evidence applies to the requested event or encounter. For example, the question asked for the earliest blood potassium result during the patient’s first hospitalization, assuming that atrial fibrillation had been documented during that stay. The agent recognized that only the second hospitalization met this condition, yet used the potassium value from that second stay (4.3 mEq/L) to answer the question about the first.

(2) Paired evaluation reveals failures that answer accuracy or abstention frequency alone can miss. EHRSQL includes questions beyond the database schema or requiring external knowledge, SCARE studies answerability classification and SQL correction, and EHR-ChatQA studies ambiguity and terminology mismatches (Section 2). TrustEHRAgent uses confidence to guide abstention. Our paired evaluation checks whether the agent can answer the supported question and show from its tool results why the other requires abstention. This evidence requirement matters in practice. GPT-4.1 abstains on 80.6% of Noise requests, yet achieves evidence-verified success on only 28.5% of all Noise requests (Section 4.2).

(3) A joint training objective for robust clinical agents. MedAgentGym supports interactive training and trajectory-based outcome verification, and EHR-Complex studies hospital-scale reasoning. EHR-RobustGym brings correct answering and evidence-grounded abstention into the same training objective, using verified trajectories for SFT and outcome feedback for RL. SFT on successful trajectories improves both Clean and Noise success across all five trained backbones (Section 4.3).

AuthorsOpenReview ↗

Response to Reviewer u1NZ (Part 2/4)

[Q1\textcolor{#B45309}{\textbf{Q1}}] Pair-level success

Q1. Given the Clean–Noise paired design, could the authors report pair-level success rates to assess whether agents can both answer supported queries and correctly abstain from their unsupported counterparts?

Thank you for suggesting that we report pair-level success rates alongside the separate Clean and Noise results. The pair-level results show that training improves agents’ ability to both answer supported queries and correctly abstain from their unsupported counterparts. Across all 1,008 Clean–Noise test pairs, Qwen3-14B’s Pair SR increases from 21.7% after SFT to 44.4% after GRPO (Table R1). A pair succeeds only when the agent correctly answers its Clean question and provides evidence-grounded abstention on its Noise counterpart.

Table R1. Representative pair-level results (%).

ModelClean SRNoise SRPair SR
GPT-5.478.355.845.4
Gemini-3.1-Pro83.458.453.1
Kimi-K2.582.437.133.8
Qwen3-14B + SFT42.434.721.7
Qwen3-14B + SFT + GRPO67.954.844.4

(1) Joint success improves together with both component metrics. GRPO also raises Qwen3-14B’s Clean SR from 42.4% to 67.9% and Noise SR from 34.7% to 54.8%. The Pair SR improvement establishes that more question pairs are handled correctly under both supported and unsupported conditions.

(2) Strong Clean performance does not ensure strong joint performance. Gemini-3.1-Pro and Kimi-K2.5 achieve similar Clean SR of 83.4% and 82.4%, but their Pair SR differs substantially at 53.1% and 33.8%. Across the seven API-based models in the submitted paper, mean Pair SR is 30.9%, compared with 62.2% mean Clean SR. The complete results and analysis are included in Appendix E.6, Tables E.3 and E.4.

[W2\textcolor{#B91C1C}{\textbf{W2}} & Q2\textcolor{#B45309}{\textbf{Q2}}] Robustness gains beyond basic EHR querying and tool use

W2. Insufficient evidence for robustness-specific training gains. Training is limited to relatively small models (4B–32B) with very low baseline success rates, making it difficult to distinguish improvements in general EHR querying and tool use from genuine robustness gains. Additional DPO/iDPO stages show inconsistent benefits, while GRPO provides limited additional improvements over SFT on general and medical benchmarks. Furthermore, gains on external EHR benchmarks do not directly establish the generalization of robustness to unsupported queries.

Q2. To what extent do the training improvements reflect enhanced robustness rather than basic EHR querying and tool-use capabilities? Could the authors provide comparisons against training on standard EHR tasks with a similar training budget?

Clean–Noise supervision provides substantial robustness gains beyond training on standard answerable EHR tasks. Clean-only uses successful SQL/Python trajectories for answerable questions in our training set, covering the EHR querying and tool use required for standard EHR tasks. Starting from the same Qwen3.5-9B Base model, Mixed SFT improves Noise SR by 48.5 percentage points and Pair SR by 36.1 points over Clean-only SFT (Table R2(a)). Mixed achieves the highest joint success, with Clean SR decreasing from 67.26% to 59.23%. The additional analyses examine gains across model scales, evidence use beyond successful tool execution, and transfer to unsupported eICU queries. The data-composition and model-scale results are included in Section 4.3 and Appendix E.3, Tables E.1 and E.2.

(1) Mixed training achieves the strongest joint performance. We use Qwen3.5-9B, which has stronger baseline agentic tool-use capabilities. Clean-only, Noise-only, and Mixed SFT each use 250 updates and 1,000 sample exposures. Mixed improves all three metrics over Base. Noise-only achieves the highest Noise SR but over-abstains on answerable questions. The original Qwen3-8B composition ablation yields consistent findings under equal demonstration counts, updates, and pair coverage (Section 4.3, Figure 4(b), and Appendix E.3).

Table R2(a). Qwen3.5-9B SFT data ablation with the Base reference (%).

Training conditionClean SRNoise SRPair SR
Base (no EHR training)28.327.110.3
Clean-only SFT67.2612.107.24
Noise-only SFT0.0064.090.00
Mixed SFT59.2360.6243.35
AuthorsOpenReview ↗

Response to Reviewer u1NZ (Part 3/4)

[W2\textcolor{#B91C1C}{\textbf{W2}} & Q2\textcolor{#B45309}{\textbf{Q2}}] Robustness gains beyond basic EHR querying and tool use (continued)

(2) The gains persist with stronger initial performance and additional model scales. The new Qwen3.5 experiments cover SFT at 9B, 35B, and 122B total parameters, plus GRPO at 9B (Table R2(b)). The 9B Base model already achieves 28.3% Clean SR and 27.1% Noise SR, showing that the benefits are not confined to near-zero baselines. GRPO further improves Qwen3.5-9B’s Noise SR despite a decrease in Noise tool-call SR from 78.5% to 72.9%. Qwen3-8B shows the same pattern. Tool-call SR measures successful execution among issued calls, so it does not establish whether the retrieved evidence supports the final response.

Table R2(b). Training results across models (%).

ModelTraining stageClean SRNoise SRPair SR
Qwen3-8BBase3.50.50.0
Qwen3-8BSFT33.822.89.9
Qwen3-8BSFT + GRPO51.839.526.9
Qwen3.5-9BBase28.327.110.3
Qwen3.5-9BSFT59.260.643.4
Qwen3.5-9BSFT + GRPO69.068.455.2
Qwen3.5-35B-A3BBase18.121.410.3
Qwen3.5-35B-A3BSFT65.961.647.4
Qwen3.5-122B-A10BBase8.36.03.2
Qwen3.5-122B-A10BSFT72.870.657.8

(3) Evidence validity and trajectories show improvements beyond successful execution. (i) Evidence validity. For Qwen3-14B, GRPO increases Noise SR from 34.7% to 54.8% and the proportion of abstentions supported by tool evidence from 51.7% to 89.5%, despite fewer abstentions (Appendix E.8, Table E.6). Performance across the eight general and medical benchmarks remains essentially unchanged. (ii) A medication example. All tool calls succeed under both SFT and GRPO. However, the SFT model substitutes the third record’s 325 mg despite observing that the requested second acetaminophen record is marked “Not Given” and lacks an administered dose. After GRPO, it verifies the requested record and correctly abstains because the available evidence does not establish an administered dose for that event. (iii) Results across models and training methods. DPO on Qwen3-32B and iDPO in the separate Qwen3-14B self-improvement experiment also improve Noise SR despite lower tool-call SR. Conversely, Qwen3-32B with GRPO improves tool execution but reduces Noise SR from 40.3% to 37.7%, with more interactions terminating without a final response. These results distinguish successful execution from evidence-grounded task completion.

(4) SFT improves evidence-grounded abstention on unsupported queries from another database. On 96 additional patient-level unsupported eICU queries, evaluated using the same evidence-grounded abstention criterion, Qwen3-14B’s success increases from 7.3% to 37.5% after SFT while the overall abstention rate remains almost unchanged. This provides direct evidence of robustness transfer beyond an increased tendency to abstain. SFT + GRPO achieves 35.4%. Evidence validity among submitted abstentions improves, but more interactions terminate without a final response. Thus, SFT transfers to these unsupported queries, while GRPO does not provide a further overall success-rate gain. The query construction and full results are included in Appendix E.8, Table E.6.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to All Reviewers

We thank all reviewers for their thoughtful feedback. Reviewers recognize that our work addresses “a meaningful challenge” (u1NZ) and “an important and practical failure mode” (Edeo). They highlight the controlled Clean–Noise paired design (u1NZ, Su22), “a valuable testbed” (Edeo), and trajectories “valuable for future training and failure analysis” (7Z4S). They also recognize our experiments as “demonstrating the practical utility of the proposed environment” (u1NZ) and providing “a broad empirical reference for the community” (Edeo).

We have added training comparisons, evaluations of larger and newer models and coding-agent harnesses, pair-level analysis, independent human validation, and external EHR transfer experiments. Key findings follow, with detailed responses in each reviewer’s thread.

(1) Our gym’s contribution and pair-level success rates. EHR-RobustGym is the first hospital-scale gym for evaluating and training robust clinical agents under real-world EHR noise, while existing EHR work has overlooked the importance of this noise for agent robustness (Reviewer u1NZ, [W1\textcolor{#B91C1C}{\textbf{W1}}]; Reviewer 7Z4S, [W1.1\textcolor{#B91C1C}{\textbf{W1.1}} & Q1\textcolor{#B45309}{\textbf{Q1}}]). Pair SR requires a correct Clean answer and evidence-grounded abstention on its Noise counterpart. Qwen3-14B’s Pair SR increases from 21.7% after SFT to 44.4% after GRPO (Reviewer u1NZ, [Q1\textcolor{#B45309}{\textbf{Q1}}]; Appendix E.6, Tables E.3 and E.4).

(2) Comparison with training on standard EHR tasks. Clean-only SFT uses successful SQL/Python trajectories for our answerable training questions. On Qwen3.5-9B, Mixed SFT improves Noise SR by 48.5 percentage points and Pair SR by 36.1 points over this baseline. For Qwen3-14B, GRPO raises Noise SR from 34.7% to 54.8% and evidence validity among abstentions from 51.7% to 89.5%, despite fewer abstentions (Reviewer u1NZ, [W2\textcolor{#B91C1C}{\textbf{W2}} & Q2\textcolor{#B45309}{\textbf{Q2}}]; Reviewer Su22, [W4\textcolor{#B91C1C}{\textbf{W4}}]; Appendix E.3, Table E.1; Appendix E.8, Table E.6).

(3) Abstention verification and data quality. GPT-5.2 (high) achieves 98.00% agreement with adjudicated human judgments on 400 Noise trajectories containing executed tool calls and ending in explicit abstention (Reviewer u1NZ, [Q3\textcolor{#B45309}{\textbf{Q3}}]; Reviewer Edeo, [W3\textcolor{#B91C1C}{\textbf{W3}} & Q1\textcolor{#B45309}{\textbf{Q1}}]; Reviewer 7Z4S, [W1.2\textcolor{#B91C1C}{\textbf{W1.2}}]; Appendix C.3, Table C.1). Additional random review of 100 Clean–Noise test pairs from the remaining test set identified no errors (Reviewer Edeo, [W2\textcolor{#B91C1C}{\textbf{W2}}]). Before student training, we checked the selected questions through reference SQL execution and model review and screened the teacher demonstrations using rejection sampling (Reviewer Su22, [W1\textcolor{#B91C1C}{\textbf{W1}}]; Appendices A.3 and D.1). As requested, we added actual Clean and Noise SFT traces to the supplementary material.

(4) Larger models and an additional comparison of SFT/RL splits. SFT improves Clean, Noise, and Pair SR over Base across Qwen3.5 models with 9B, 35B, and 122B total parameters. Qwen3.5-122B-A10B reaches 72.8% Clean SR, 70.6% Noise SR, and 57.8% Pair SR (Reviewer u1NZ, [W2\textcolor{#B91C1C}{\textbf{W2}} & Q2\textcolor{#B45309}{\textbf{Q2}}]; Appendix E.3, Table E.2). The original split was by question. With a split by complete Clean–Noise pairs, PPO and GRPO both improved Clean and Noise SR over the retrained Qwen3-14B SFT baseline (Reviewer Edeo, [W1\textcolor{#B91C1C}{\textbf{W1}} & Q2.1\textcolor{#B45309}{\textbf{Q2.1}}], Table R1).

(5) Newer models and coding-agent harnesses. GPT-5.6 Sol (xhigh) achieves 81.0% Clean SR but 32.2% Noise SR (Reviewer Su22, [W3\textcolor{#B91C1C}{\textbf{W3}}]; Appendix E.6, Table E.3). Codex, Claude Code, and Pi each improve overall success in the native model comparisons (Reviewer Su22, [W2\textcolor{#B91C1C}{\textbf{W2}}]; Appendix E.7, Table E.5).

(6) External EHR transfer. Qwen3-14B + SFT + PPO achieves the highest success among the compared variants on all five external EHR benchmarks, averaging 38.7% versus 27.4% after SFT (Reviewer Edeo, [Q3.2\textcolor{#B45309}{\textbf{Q3.2}}]; Section 4.3, Figure 6(a)). On 96 unsupported patient-level eICU queries, SFT increases Qwen3-14B’s success from 7.3% to 37.5% while the abstention rate remains almost unchanged, providing direct evidence of robustness transfer (Reviewer u1NZ, [W2\textcolor{#B91C1C}{\textbf{W2}} & Q2\textcolor{#B45309}{\textbf{Q2}}]; Reviewer Su22, [W4\textcolor{#B91C1C}{\textbf{W4}}]; Appendix E.8, Table E.6).

AuthorsOpenReview ↗

Update of Manuscript

Dear Reviewers,

We have revised the manuscript in response to your feedback and highlighted the changes in blue for ease of review. Below, we summarize the main updates to the manuscript and supplementary material.

Main Text

  • Experimental setup and verification [Section 4.1]. Added human validation of the Noise verifier and introduced coding-agent harnesses.
  • Pair-level success and newer models [Section 4.2]. Added Pair SR and results from newer models.
  • Training and generalization [Section 4.3]. Added Qwen3.5 training results, Mixed versus Clean-only comparisons, PPO results on five external benchmarks (Figure 6(a)), and unsupported eICU evaluation.

Appendix

  • Judge validation [Appendix C.3, Table C.1]. Added independent human assessment and comparisons between outcome judges.
  • Training comparisons [Appendix E.3, Tables E.1–E.2]. Added detailed results across training compositions and model scales.
  • Agent evaluation [Appendices E.6–E.7, Tables E.3–E.5]. Added complete results for pair-level success, newer models, and coding-agent harnesses.
  • Tool execution and evidence use [Appendix E.8, Table E.6]. Added tool and evidence analyses, including evaluation on unsupported eICU queries.

Supplementary Material

  • SFT trajectory examples. Actual Clean and Noise SFT examples are included in the updated supplementary material.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.