- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The main idea of the paper is to use EMR data to automatically fill out a document for the patient process it through a guideline constructing Python executable versions using LLMs.
Critical strengths
The paper addresses a timely problem.
Critical weaknesses
Major Issues
- The main contribution of the paper is to limit the LLMs output to machine readable format for EMR data; constructing Python executable decision graphs which then are checked by some predetermined steps. Training the smaller model and showing similar performance doesn't really have novelty as they are already widely known. I don't think the paper is novel enough for ICLR.
- The figure doesn't match up to ICLR standards.
- The manuscript reads more like a report rather then a scientific paper.
- The evaluation is being done by another LLM without any human validation
- The accuracy of the generated graphs are not evaluated.
- The accuracy of the generated EMR's are not reported.
Minor Issues
- The code type font on ``ClinicalAction'' should be removed.
- The usage of the word ``diff'' is informal.
- The third contribution is not mentioned at all in the abstract.
- The table presented should be self sufficient whereas the code type fonts makes it harder and the caption doesn't provide enough information.
- Phrases such as ``What is covered in this submission'' should be removed.
Questions for authors
- How accurate is the LLM at filling in the patient record on its own?
- How accurate is the LLM at writing rules from the guidelines?
- Did you check that the GPT-5.4 judge agrees with human doctors?