- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies pre-arrival care navigation under shared, limited capacity, where routing decisions must account for both individual patients’ needs and their effects on subsequent arrivals. The authors introduce TRIAGEROUTE and the SICL. Experiments show improved urgent, timely access compared with the evaluated LLM-based policies, with a trade-off in capacity efficiency.
Overall, I think the direction is very promising, but the main concerns are insufficient benchmark analysis and the need to strengthen the related work comparison. If the author could resolve those, I would increase the score.
Critical strengths
- Individually reasonable recommendations can collectively overload scarce medical resources and delay urgent patients. Moving beyond isolated patient-level evaluation toward sequential decisions under shared capacity is practically meaningful.
- The benchmark is interesting and potentially useful. Combining clinical descriptions with heterogeneous care sites, travel times, and evolving queues enables evaluation of consequences that static triage benchmarks cannot capture.
Critical weaknesses
- TRIAGEROUTE is a central contribution, but much of the information needed to assess its construction and validity is deferred to the appendix. The main paper should devote more space to case selection, clinical-label construction, the care-network configuration, and validation of the simulator.
- In this one, several consequential choices remain insufficiently justified, including the six-site network and the treatment of recorded resource use as required care. Please see the details in the question part
-
At an abstract level, the task resembles traditional online allocation or routing with heterogeneous resources, uncertain demand, capacity constraints, and congestion externalities. What are the main differences between the health-care area and those domains? The paper should clarify which characteristics of the medical setting create technical challenges beyond this established formulation.
-
The paper includes LLM baselines with the same observations, but the classical routing references receive ground-truth acuity and resource needs, whereas SICL must infer them. A key missing baseline is a modular pipeline in which an LLM receives the same inputs as SICL, predicts acuity and resource requirements, and passes these predictions to a conventional routing algorithm. This comparison would help determine whether SICL’s gains require its proposed learning mechanism.
Questions for authors
-
Care-network configuration and resource labels. Why does the benchmark use exactly six care sites? Could the authors evaluate additional configurations? Furthermore, what validates the extracted resource requirements?
-
Of the 69,043 ED stays linked to discharge summaries, 6,166 become detailed record-grounded cases. How many records were retained or excluded at each stage, and for what reasons?
-
The 1,800 supplementary low-acuity cases follow a different construction process and have a thinner information schema. Could these differences introduce textual shortcuts that allow a model to identify low-acuity cases from presentation style or information completeness rather than clinical content? Has this possibility been tested?
-
Clinical validation of generated content. Were the reconstructed interviews and extracted or inferred resource requirements reviewed by clinicians on a representative sample? If so, what were the error rates and agreement levels?
-
The motivating example emphasizes preserving high-capability resources by diverting lower-acuity patients. However, SICL routes 75% of ESI 3–5 patients to high-capability sites, compared with 69% for PPO. Which specific routing behaviors improve urgent, timely access rate?
6 comments
Response to Reviewer vj8Z (2/4)
Q2. Case funnel from 69,043 to 6,166.
The counts below come from the construction scripts; we will show them as a funnel in §5.1. The split by patient comes before case generation: no evaluation encounter appears in the pool, and record-grounded pool cases share no patient with the evaluation encounters.
| Stage | ED stays | Patients | Kept when |
|---|---|---|---|
| MIMIC-IV-ED v2.2 | 425,087 | 205,504 | – |
| Linked to a discharge summary | 69,043 | 52,344 | the stay's admission has one ED stay and one MIMIC-IV-Note discharge summary |
| Repeat-visit patients | 54,482 | 8,001 | the patient has more than three ED stays, so earlier encounters can be shown |
| ED-course extraction | 42,220 | 6,344 | extraction of the ED course succeeds and acuity is recorded |
| Acuity–resource consistency | 36,506 | 6,334 | documented resources do not contradict the nurse-assigned ESI |
| Split by patient | 7,187 pool / 29,319 held out | 1,266 / 5,068 | no patient on both sides |
| Record-grounded cases | 6,166 | 1,255 | interviews generated and pass validation |
Discharge summaries longer than 10,000 tokens are also excluded along the way. The 1,800 supplemented low-acuity cases (App. C.4) are added on the pool side, giving 7,966 cases from 2,650 patients.
Q3. Text shortcuts in the supplemented cases.
We tested this in two places, and the answer differs between the pool and the evaluation.
- No evaluation arrival is a supplemented case. Evaluation streams are drawn only from held-out record-grounded encounters (Q2); supplemented cases exist only on the pool side. Every evaluated low-acuity arrival therefore has the full record-grounded schema, and its source cannot be read from its format.
- Same input template for every case. Every policy reads every case through the same template, with the same fields in the same order. No policy sees source text, construction prompts or the source of a case.
- Tested on the evaluation inputs. A logistic-regression probe that uses only missingness, structural counts and text lengths, with five-fold cross-validation grouped by patient, identifies ESI-4 cases at balanced accuracy 0.53 (label-permuted controls 0.43–0.57) and AUROC 0.57.
In the pool, where both sources coexist, the two formats are distinguishable. The same probe trained on the pool and applied to the evaluation stream does not detect ESI-4 arrivals (AUROC 0.55), so the format signal does not carry over to record-grounded encounters. We will state the schema difference and these checks in §5.1, and release a source flag with every pool case so that users can stratify or reweight by source.
Response to Reviewer vj8Z (3/4)
W2. Difference from online allocation.
We agree that, abstractly, TriageRoute is an online allocation problem with heterogeneous resources and congestion externalities. When the type is revealed, classical methods are strong: given recorded acuity and resource needs, Dual pricing (bounded) reaches 0.513 UTA (Table 7). Three characteristics of the medical setting go beyond that formulation.
- The type is not revealed. Online allocation assumes that the arriving item's type, or a calibrated signal of it, is observed. Here acuity and resource needs are observed only through a noisy, language-based pre-arrival interview. The modular baseline (W3) measures the cost: the same routing rules that reach 0.50–0.51 with recorded acuity reach 0.44–0.47 when an LLM infers acuity and resources from the inputs SICL sees, while SICL reaches 0.540 on the same inputs.
- The routed patient's interest is an objective, not a cost. A dispatcher maximizes a system objective; a navigator also owes each patient a capable, close and timely site. We therefore score patient utility beside system outcomes, and destinations are chosen in the patient's interest (§4).
- The system objective is thresholded, delayed and set by runs of decisions. Whether an urgent patient reaches a bed within 30 minutes depends on placements made earlier for other patients. Dual pricing handles this well when types are revealed (Table 7). When the router is a language policy reading interviews, the open problem is to credit a delayed, shared outcome to the decisions that shaped it, so we see the learning problem mainly as one of credit assignment rather than dispatch.
We will add a related-work paragraph on online matching and resource allocation, network revenue management, and patient scheduling and ambulance dispatch, organized around these three differences.
W3. Same-input modular baseline.
We agree this is an important comparison and have run it (Table R1 in the general response). An LLM receives exactly the inputs SICL receives, predicts ESI and the resource set, and passes them to the two strongest classical routing rules of Table 7, Shortest-delay and Dual pricing (bounded), each in its own simulator run. We use two LLMs: a prompted Qwen3-32B, and a Qwen2.5-1.5B (SICL's backbone) fine-tuned on our training labels. The 32B model's prompt was chosen among three versions on 150 training-side cases by a rule fixed in advance, and frozen before any evaluation run; the fine-tuned model uses its own training prompt.
- With recorded acuity and resources, the two rules reach 0.504 and 0.513 UTA (Table 7). With the clinical state inferred by Qwen3-32B, they reach 0.441 ± 0.029 and 0.456 ± 0.065. SICL reaches 0.540 ± 0.030 on the same interviews (our reply to Reviewer MxN4, W4), with higher patient utility (0.679 vs 0.656 and 0.666) and lower capacity efficiency, the trade-off reported in §6.2.
- The 32B model finds most urgent patients (urgent recall 0.71) but over-triages: it labels 41% of arrivals ESI 1–2, against 13% in the records, and rarely assigns ESI 4–5. The rules then hold capacity for patients who do not need it, and incomplete resource predictions raise the transfer rate to 4–6%, against 1% for SICL.
- The fine-tuned 1.5B model finds more urgent patients (urgent recall 0.86) but over-triages further, labeling 54% of arrivals ESI 1–2. With it the rules reach 0.436 ± 0.049 and 0.470 ± 0.010 UTA, with patient utility 0.651 and 0.661, again below their results with recorded acuity and below SICL.
These results suggest that SICL's gain is not reproduced by pairing a prompted or fine-tuned LLM with a strong routing rule, and that learning the routing from system outcomes matters.
Response to Reviewer vj8Z (1/4)
Thank you for the careful review. You named two concerns as decisive: the analysis of the benchmark and the comparison with related work. We will add a main-text section on how TriageRoute is built and validated (case funnel, label construction, clinician review, care network), a related-work paragraph on online allocation, and a comparison with existing clinical LLM benchmarks. We also ran the modular baseline you suggested. We answer W1 and Q1–Q3 first, then related work (W2) and the modular baseline (W3), and finally the clinician review (Q4) and Q5.
W1. Benchmark construction in the main text.
We agree this material belongs in the main text. We will move it into a new section, using the additional page, that covers case selection (the funnel in Q2), label construction (Q1), the clinician review (Q4), the care network (Q1) and the simulator's checks (the arrival profile against NHAMCS, and the capacity variants in Q1). Two choices you flag need a direct justification.
Q1. Six sites, other configurations, resource validation.