- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces EHR-RobustGym, an interactive environment designed to evaluate and train the robustness of Electronic Health Record (EHR) agents. The authors address a key limitation of existing agents: their tendency to generate answers unsupported by evidence when database records are missing, field values fail to satisfy specified requirements, or query constraints cannot be met.
Built upon MIMIC-IV, the paper constructs 5,486 Clean–Noise question pairs spanning six categories of clinical tasks, three types of noise, and both patient-level and population-level queries. Clean questions have correct answers supported by the database, whereas Noise questions require agents to interact with the database, gather evidence, and appropriately abstain from answering when sufficient evidence is unavailable.
EHR-RobustGym provides a SQL/Python execution environment, interaction trajectory collection, and Outcome Verification, supporting agent training through Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), Proximal Policy Optimization (PPO), and Group Relative Policy Optimization (GRPO).
Experiments evaluating 12 large language models (LLMs) reveal substantial performance degradation under Noise conditions. The authors further demonstrate that supervised fine-tuning and reinforcement learning based on successful interaction trajectories can significantly improve task success rates, with transfer gains also observed across five external EHR benchmarks.
Critical strengths
- The paper addresses an important limitation of existing EHR agents: their ability to determine whether database evidence actually satisfies query requirements. The Noise tasks require agents to collect evidence through tool interactions to justify abstention, making the setting more realistic and clinically relevant.
- The Clean–Noise paired design changes only one necessary condition while preserving the original clinical intent, enabling controlled comparisons. The authors also conduct SQL-based validation, LLM review, and manual inspection. Notably, 344 challenging test pairs were manually reviewed, with 92.15% found valid before correction and the remaining 27 pairs revised.
- The paper evaluates 12 LLMs and multiple training methods, including SFT, DPO, PPO, and GRPO, along with external benchmark evaluations. Additionally, the analysis of 12,096 agent trajectories across four error categories provides useful insights into agent failure modes.
Critical weaknesses
-
850 out of 2,000 demonstrations are generated with teacher-only reference explanations and queries. However, this may introduce issues in the teacher-generated reasoning and action trajectories since the teacher is input with additional information. Even if the guidance is removed from student inputs, directly training students on these trajectories could lead to problems such as hallucinations. Additionally, the authors should provide examples of the actual SFT training traces.
-
Also, the authors should conduct some experiments with the coding agent harness. Currently, it seems that the harness used in this paper only includes code execution. The authors could evalaute e.g., codex + gpt, claude code + opus/sonnet, qwen + pi.
-
The authors can include some of the latest frontier models, such as GPT-6 Astra and Claude Opus 5.5, to assess the difficulty of the tasks in this paper. Moreover, many of the evaluated open-weight models in the paper are relatively outdated and exhibit low baseline success rates, because these open-source models in this paper have few agentic capabilities. Including newer open-weight models, such as Qwen3.8-27B, would help establish the benchmark's practical value for current agentic systems.
-
The improvements achieved through SFT and reinforcement learning may result from better tool-use capabilities rather than stronger evidence-grounded reasoning alone. This concern is particularly relevant for smaller models with near-zero baseline success rates, which may struggle with basic SQL/Python execution and database interactions. Although the paper provides tool-call statistics and error analysis, these do not fully disentangle the two factors. A comparison with models trained on general EHR tool-use trajectories, without Clean–Noise supervision, would help isolate the contribution of robustness-specific training.
Questions for authors
see the weaknesses.
5 comments
Response to Reviewer Su22 (Part 2/3)
[] Evaluation with coding-agent harnesses
W2. Also, the authors should conduct some experiments with the coding agent harness. Currently, it seems that the harness used in this paper only includes code execution. The authors could evalaute e.g., codex + gpt, claude code + opus/sonnet, qwen + pi.
Thank you for suggesting these three model–harness comparisons. Codex, Claude Code, and Pi each achieve higher overall success than their corresponding baselines in the native model comparisons (Table R1). Our original interface lets agents issue SQL/Python queries through a common JSON action format, inspect the results, and query again. Codex, Claude Code, and Pi use the same clinical task instructions and EHR tool, with each harness adding its own system instructions and native tool interface. We evaluated each combination under the same limits as its corresponding baseline, with up to 65,536 input tokens, 4,096 output tokens per response, and 20 model responses per question (Appendix C.1, Table 9).
Table R1. Native coding-agent harness evaluations and corresponding baselines (%).
| Model and harness | Clean SR | Noise SR | Overall SR |
|---|---|---|---|
| GPT-5.4 (xhigh) | 78.27 | 55.75 | 67.01 |
| GPT-5.4 (xhigh) + Codex | 86.21 | 56.35 | 71.28 |
| Sonnet 4.6 (thinking disabled) | 62.50 | 26.29 | 44.39 |
| Sonnet 4.6 (thinking disabled) + Claude Code | 84.62 | 33.43 | 59.03 |
| Qwen3.5-9B (thinking disabled) | 28.27 | 27.08 | 27.68 |
| Qwen3.5-9B (thinking disabled) + Pi | 53.47 | 39.19 | 46.33 |
All three harnesses improved success more on Clean questions than on Noise questions. For GPT-5.4, Codex raised Clean SR by 7.94 percentage points and Noise SR by 0.60 points (Appendix E.7, Table E.5).
[] Task difficulty for newer models
W3. The authors can include some of the latest frontier models, such as GPT-6 Astra and Claude Opus 5.5, to assess the difficulty of the tasks in this paper. Moreover, many of the evaluated open-weight models in the paper are relatively outdated and exhibit low baseline success rates, because these open-source models in this paper have few agentic capabilities. Including newer open-weight models, such as Qwen3.8-27B, would help establish the benchmark's practical value for current agentic systems.
The benchmark remains challenging for newer frontier and open-weight models, especially when abstention must be justified by evidence. GPT-5.6 Sol (xhigh) achieves 81.0% Clean SR but 32.2% Noise SR and 26.4% Pair SR. Qwen3.8-27B (thinking disabled) achieves 50.6% Clean SR but 11.1% Noise SR (Table R2). These results establish that the difficulty extends beyond models with near-zero baseline success. Strong supported-query performance does not ensure reliable handling of unsupported counterparts. The additional results and analysis are included in Appendix E.6, Table E.3.
Table R2. Task success and pair-level success for newer models (%).
| Model | Clean SR | Noise SR | Pair SR |
|---|---|---|---|
| GPT-5.6 Sol (xhigh) | 81.0 | 32.2 | 26.4 |
| GPT-5.5 (xhigh) | 58.9 | 34.8 | 25.9 |
| GLM-5.2 (max) | 66.6 | 31.7 | 24.1 |
| Kimi K3 (max) | 59.0 | 21.6 | 15.0 |
| Qwen3.8-27B (thinking disabled) | 50.6 | 11.1 | 5.9 |
Response to Reviewer Su22 (Part 3/3)
[] Distinguishing tool-use gains from evidence-grounded robustness
W4. The improvements achieved through SFT and reinforcement learning may result from better tool-use capabilities rather than stronger evidence-grounded reasoning alone. This concern is particularly relevant for smaller models with near-zero baseline success rates, which may struggle with basic SQL/Python execution and database interactions. Although the paper provides tool-call statistics and error analysis, these do not fully disentangle the two factors. A comparison with models trained on general EHR tool-use trajectories, without Clean–Noise supervision, would help isolate the contribution of robustness-specific training.
Clean–Noise supervision yields stronger evidence-grounded robustness than training on general answerable-EHR trajectories. Clean-only uses general EHR tool-use trajectories without Noise or abstention demonstrations. These are successful SQL/Python trajectories for answerable questions in our training set. Starting from the same Qwen3.5-9B Base model, Mixed SFT improves Noise SR by 48.5 percentage points and Pair SR by 36.1 points over Clean-only SFT, while Clean SR decreases from 67.26% to 59.23%. This comparison demonstrates the additional value of robustness supervision beyond answerable-EHR training. The full ablation and results across model scales are included in Appendix E.3, Tables E.1 and E.2, and our response to Reviewer u1NZ [ & ], Tables R2(a) and R2(b).
(1) The Clean-only comparison demonstrates the additional value of Noise supervision. We use Qwen3.5-9B for its stronger baseline agentic tool-use capabilities. Clean-only, Noise-only, and Mixed SFT each use 250 updates and 1,000 sample exposures, while the number of unique demonstrations and token counts differ. Clean-only improves supported-query accuracy but reduces Noise and Pair SR below Base. Mixed improves all three metrics over Base and achieves the highest Pair SR, while Noise-only over-abstains on answerable questions. The original Qwen3-8B composition ablation yields consistent findings (Section 4.3, Figure 4(b), and Appendix E.3).
(2) Robustness gains also occur without higher tool-call success. We extend training experiments to Qwen3.5-9B, Qwen3.5-35B-A3B, and Qwen3.5-122B-A10B. For Qwen3.5-9B, GRPO increases Noise SR from 60.6% to 68.4%, while Noise tool-call SR decreases from 78.5% to 72.9%. Tool-call SR measures successful execution among issued calls. It does not establish whether the resulting evidence supports the final answer or abstention.
(3) Unsupported eICU queries provide direct evidence of robustness transfer. On 96 additional patient-level unsupported eICU queries, Qwen3-14B’s success increases from 7.3% to 37.5% after SFT while the overall abstention rate remains almost unchanged (Appendix E.8, Table E.6). This supports transfer of better-supported abstention beyond more frequent abstention. GRPO does not provide a further overall gain, achieving 35.4%. Evidence validity among submitted abstentions improves, but more interactions terminate without a final response.
References
[1] Feng et al. ReTool: Reinforcement Learning for Strategic Tool Use in LLMs. ICLR 2026.
[2] Zhang et al. Generative Verifiers: Reward Modeling as Next-Token Prediction. ICLR 2025.
Response to Reviewer Su22 (Part 1/3)
Dear Reviewer Su22,
Thank you for recognizing the clinical relevance of requiring agents to gather evidence and justify abstention, and the controlled Clean–Noise paired design that changes one necessary condition while preserving clinical intent. We appreciate your positive assessment of the SQL-based validation, LLM review, and manual quality checks, as well as the broad evaluation across models, training methods, and external benchmarks and the insights from trajectory-based failure analysis. We address your concerns below by clarifying teacher guidance and SFT quality checks, providing actual SFT examples, evaluating coding-agent harnesses and newer models, and examining robustness gains beyond basic tool use.
[W1] Teacher guidance and SFT trajectory quality
All 850 reference-guided demonstrations had already been checked after removing teacher-only guidance and before student training and the original submission to verify that their reasoning and conclusions remained self-contained and supported by student-visible tool evidence. Actual Clean and Noise SFT examples, including reference-guided demonstrations, are now included in the supplementary material. The revised Appendix D.1 describes the original quality-control procedures in greater detail.
(1) Reference guidance identifies the clinical requirements and evidence to retrieve. Reference explanations and queries guide the teacher’s reasoning and database interactions during demonstration construction. ReTool [1] uses existing reasoning to construct trajectories with code calls and execution feedback, followed by answer verification. GenRM [2] uses reference solutions to generate verification rationales, filters them by verification correctness, and removes the references before verifier fine-tuning. In our setting, guided trajectories must still support their conclusions using the original question, database schema, and actual tool feedback available to the student.
(2) Guided trajectories were checked for self-containment before student training. As described in Appendix D.1 of the submitted manuscript, the 850 reference-guided demonstrations underwent naturalness and self-containment checks by Gemini-3.1-Pro after teacher-only reference explanations and queries were removed. These checks required reasoning and conclusions to follow from the original question, database schema, executed code, and preceding tool feedback. Trajectories could not report database observations before the corresponding tool output or use teacher guidance as evidence.
(3) Question validation and demonstration selection. Training questions are checked through reference SQL execution and model review (Appendix A.3). The demonstration dataset contains 1,150 unguided and 850 reference-guided trajectories selected through rejection sampling before student training. Accepted trajectories must pass task evaluation and support their answers or abstentions with evidence obtained through actual tool interaction.
(4) As requested, we have added actual student-visible SFT examples to the supplementary material. These cover Clean and Noise tasks, including demonstrations generated with reference guidance. Each example shows the question, tool calls, tool feedback, and final answer visible to the student after teacher-only guidance is removed.