- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
MOSAIC combines specialty prompts with small trainable adapters. The backbone remains frozen. GEPA discovers natural language guidance for eight specialties. TinyLoRA adapters learn additional adjustments through reinforcement learning. Patient diagnosis frequencies determine the adapter mixture. Training updates only 78.4K parameters across all specialties.
The method predicts diagnoses at the next visit. Code F1 improves from 0.099 to 0.124 on MIMIC. The paper also reports results on eICU.
Critical strengths
The adapter composition is mathematically coherent. The reported improvements over the Table 1 baselines have paired bootstrap support. The parameter-matched comparisons are useful.
Critical weaknesses
The method combines several established techniques. Its novelty mainly lies in their clinical integration.
The experiments do not isolate the central personalization mechanism. Several evaluation details also remain unclear. M1 and M2 concern the paper's central contribution. M5 concerns whether the comparisons can be reliably interpreted. M3 and M4 mainly require stronger evidence or narrower claims. These concerns lead me towards weak rejection.
M1. Missing Routing Ablations
Table 2 does not isolate patient-dependent routing. There is no global TinyLoRA baseline. Uniform mixtures and hard routing are also missing. A global adapter could learn the same dataset statistics. Specialty routing might therefore be unnecessary. Existing LoRA comparisons do not resolve this question. They change the adapter parameterization alongside the proposed structure.
The granularity claim has another possible explanation. GEPA optimizes group F1, while adapters use a graded code-matching reward that gives the highest reward to exact matches. Their different objectives could explain their different improvements. The paper should separate objective effects from architectural effects.
These comparisons should match prompts, rewards, and training budgets. An adapter-only variant would also clarify GEPA's contribution.
M2. Unclear Diagnostic Metrics
Section 4.4 defines chapters using the first ICD letter. However, first letters do not define official ICD chapters. The letter H spans two official chapters. Conversely, A and B share one chapter. This affects metric interpretation and the adapter reward.
Twelve clinical groups are presented after 22 body systems. Their definitions and nesting relationships are not provided. This weakens the interpretation of granularity-specific improvements. CCSR can also assign codes to multiple categories. The paper should specify how these assignments are handled.
Code F1 measures three-character categories, not complete codes. This distinction should remain explicit throughout the paper.
M3. Temporal Personalization Is Not Established
Section 2.2 routes patients using chapter frequencies. Different chronological histories can share identical chapter frequencies. Such histories receive identical adapter mixtures. The backbone still receives chronological context. However, the router itself ignores order and recency.
Figure 2 also compares patients with different history lengths. This does not isolate the benefit of additional history. A recency-aware router would provide a useful comparison.
M4. Coverage Does Not Establish Rare Disease Accuracy
Figure 2 reports 585 distinct predicted codes. This measures output diversity, not accuracy on rare diagnoses. However, the paper claims accurate prediction across the diagnostic long tail. Aggregate F1 can improve through frequent diagnoses alone.
Also, new diagnoses should be separated from previously observed diagnoses. Figure 3 should report disease prevalence and uncertainty.
M5. Incomplete Evaluation Details
Several details are necessary to interpret the reported comparisons. These include encounter construction, cutoff selection, and eligible patient counts. The requested prediction count also needs justification. This count must be selected without future target information. F1 averaging and output thresholds also need specification.
Baseline training and tuning budgets are insufficiently described. Some baselines predict only 28 to 53 distinct codes. Threshold choices could contribute to this narrow coverage. These details matter when interpreting the claimed performance advantage.
The eICU transfer claim introduces another ambiguity. Training on eICU would demonstrate replication, rather than frozen transfer. The paper should identify every component adapted using eICU. These omissions prevent reliable interpretation.
Questions for authors
The suggested fixes corresponding to the concerns above are:
M1. Can you isolate routing benefits using matched controls, including a global TinyLoRA baseline, uniform mixtures, and hard routing? These comparisons should match prompts, rewards, and training budgets. An adapter-only variant would also clarify GEPA's contribution.
M2. Can you release validated mappings and correct affected evaluations?
M3. Can you compare routers while truncating the same patients' histories?
M4. Can you report precision and recall across diagnosis-frequency groups?
M5. Can you release complete protocols, including explicit eICU adaptation details?
2 comments
Dear authors,
Thank you for the efforts you have made in this rebuttal. However, the evidence for the paper’s central personalization claim remains modest. On balance, I am maintaining my weak-reject rating.
Wishing you well in the last stretch of reviews.
We thank the reviewer for a careful and specific review. Changes in the revised manuscript are in blue; the analyses below are in Appendix D (Table 7), E (Tables 8–9), F (Table 10), G (Table 11), H (Table 12) and I. Numbers are macro F1 (Chapter / CCSR / Clinical Group / Code).
M1: routing ablations
All adapter rows match MOSAIC's 78K budget, prompts, GRPO objective and reward; the global adapter is one 78K adapter trained on all patients, no routing; hard routing uses the stored specialty label's adapter; uniform mixture averages the eight adapters equally.
MOSAIC exceeds every other row of the table at every granularity. No single specialist reaches the mixture at Clinical Group or Code (0.359 to 0.371 and 0.107 to 0.120), and the best is within 0.001 at Chapter and CCSR. With the objective held fixed, the gain from composing the adapters is architectural. Table 2's LoRA rows use MOSAIC's reward with a generic instruction; with the evolved prompts (last row) the 78K LoRA gains at most 0.014 and stays below MOSAIC everywhere.
M2: metric definitions and mappings
The paper's Chapter F1 groups 3-character categories by their first letter; re-scored by the official ICD-10-CM chapters, MOSAIC reaches 0.291 against 0.278 for the best baseline, and all 18 baselines stay below under both definitions. The 12 clinical groups are our own, not nested in the 22 CCSR body systems: unions of whole ICD-10-CM blocks by the clinical service that acts on the prediction, and categories outside them score as a residual Other label (31% of targets). A category whose codes span CCSR body systems takes the AHRQ default system of the majority of its codes. Code F1 scores the 3-character category, as Section 4.4 and the revised caption of Table 1 now state. Both mapping tables will be released.
M3: temporal information
The router uses only the chapter composition of the history and is order-free. We permuted each patient's event order, keeping events, routing weights and prompt unchanged.
The loss (0.028 to 0.039, paired difference in Appendix G) exceeds that from removing the adapters entirely, and its 95% interval excludes zero everywhere: the predictions depend on the ordered trajectory. Two routers fed truncated histories of the same patients, each applying the single adapter it weights highest:
Neither exceeds MOSAIC at any granularity: it already reads the ordered trajectory, so an explicit recency-aware router is unnecessary.
M4: long tail, prior versus new diagnoses
In three equal groups of the 357 categories by target frequency (74% / 18% / 8% of targets), MOSAIC's macro F1 is 0.138 / 0.120 / 0.113 on frequent / medium / rare codes, highest in every group against its two ablation points, the frozen backbone (0.116 / 0.095 / 0.086) and the 78K LoRA of Table 2 (0.109 / 0.107 / 0.094), and against TransformEHR (0.127 / 0.046 / 0.016), the strongest deep EHR baseline in Table 1; MOSAIC's widest margins are on medium and rare codes. Pooled precision / recall (Table 9): MOSAIC 0.179 / 0.175, 0.151 / 0.123, 0.118 / 0.117, balanced in each group; TransformEHR 0.199 / 0.188, 0.179 / 0.055, 0.148 / 0.012, precise but recovering only 5.5% and 1.2% of medium and rare targets. On targets split by whether they already appear in the record (43% prior, 57% new; Table 12), MOSAIC has the highest Clinical Group F1 on both (0.385, 0.224), above every baseline on new diagnoses.
M5: evaluation protocol
Section 4.2 and Appendix C state the full protocol. Every system is scored on its top three predictions, with no tuned threshold, macro F1 over a fixed 357-category vocabulary. The deep EHR baselines were retrained from scratch, following the original configuration where one is specified at a comparable scale. On eICU (Appendix C), prompts and adapters are refit with backbone, router (MIMIC-IV prototypes) and hyperparameters unchanged, ICU stays in unit-visit order, so Table 3 is a replication; paired tests put MOSAIC significantly above 16 of 18 baselines at Chapter, CCSR and Clinical Group and 18 of 18 at Code.
We hope these results address your concerns and would be glad to answer any further questions.