- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors propose SepsisAgent, a LLM agent trained to query a world model before giving a treatment recommendation. They train this agent with a 3-step process to first add awareness of patient dynamics and guidelines, then achieve a propose-simulate-refine workflow, and finally use RL to inform its treatment recommendation. The authors demonstrate empirically that providing the LLM with a world model and clinical priors alone is insufficient for improving performance on treatment recommendation. They then show that their 3-step training process helps the model perform significantly better.
Critical strengths
- The experiments are set up well. The authors define a reward function that combines terminal outcome, intermediate physiology, and guideline consistency. They then measure off-policy evaluations of this reward, adherence to sepsis guidelines, and unsafe actions. They first demonstrate with 3 different LLMs that adding a world model and clinical priors is insufficient to improve performance and then demonstrate the efficacy of their 3-step training process. They also compare to results from both LLM-only and RL-only models. They also evaluate the effect of adding each stage of their training process.
- The paper is well-written. The motivation for interacting with a world model and the description of the 3-stage agent training process are clear. The prompts are provided in the appendix for reproducibility. The example reasoning trace provided in the appendix also helps with seeing how the model works.
Critical weaknesses
- I wonder if the clinical world model described in Section 3.1 is overly simplistic as next-step and final outcome prediction may be subpar compared to RL. If the LLM were given an RL model without training on how to use it, how would it perform?
- The authors switch from using o3, Gemini-3-Flash, and GPT4.1-mini to Qwen3-4B Instruct when comparing the world model only to their 3-step training process. This comparison does not seem fair as Qwen3-4B-Instruct performs worse than the other LLMs as shown in Table 3. The LLM used should be held constant for vanilla LLM, world-model augmented LLM, and Sepsis Agent. Why was the 3-step SepsisAgent process not tried with the better-performing LLMs? How does Qwen3-4B-Instruct perform with just the world model (and clinical prior)?
Questions for authors
In addition to the questions in the weaknesses section above:
-
When you used RL to simulate trajectories, how many trajectories did you roll out? How did the model handle uncertainty / variance across trajectories?
-
The example trace is great to see! The final response is not good though. If the patient received very high IV fluid over the past 4 hours, their condition is not considered stabilized sufficiently for discharge. What proportion of recommendations ended in an unsafe discharge while on fluids or vasopressors?