acceptodds
Under review as a conference paper at ICLR 2027

Guideline-Anchored Multimodal Fusion with Prototype Mining for Reliable EHR Analysis

Abstract

Multimodal Electronic Health Records (EHRs), which integrate multivariate physiological time-series data with free-text clinical notes, provide rich temporal features for healthcare analytics. Capturing the corresponding temporal dynamics across different modalities is crucial for enhancing the accuracy of clinical outcome prediction. However, existing approaches predominantly capture superficial statistical correlations. Furthermore, the absence of evidence-based clinical guidelines in representation learning leads to suboptimal predictive performance and inadequate reliability in safety-critical scenarios.To address these limitations, we propose the **G**uideline-**A**nchored **M**ultimodal Fusion with **P**rototype Mining (**GAMP**) framework, which leverages medical guidelines to efficiently extract clinically meaningful cross-modal temporal features from multimodal EHRs. Specifically, GAMP initializes shared prototype representations using prior knowledge derived from medical guidelines. We introduce a guideline-anchored alignment loss, enabling the model to comprehensively learn cross-modal temporal semantics while constraining the latent space distance between the learned representations and the clinical guideline knowledge. Finally, a dual-branch Transformer decoder is employed to fuse standardized medical evidence with individual patient characteristics, yielding predictions that are both accurate and clinically plausible. Our experiments on MIMIC-III for 48h in-hospital mortality prediction and 24h phenotype classification verify that our method surpasses current state-of-the-art baseline models.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Multimodal Representation Learning, Knowledge-Guided Learning, Prototype Learning, Cross-Modal Alignment, Healthcare AI

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_j4A5OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

GAMP predicts in-hospital mortality and 25 phenotypes from physiological time series and clinical notes. Synthesized medical statements initialize shared prototype queries, calibrate patient-specific prototypes through attention, and provide a latent alignment target. Reconstruction retains record information, and a dual-branch decoder combines guideline and patient-note queries. On two MIMIC-III tasks, the paper reports five-run improvements over fourteen baselines, component ablations, and qualitative prototype analyses. The reported predictive gains are promising. The evidence for medically valid guideline semantics as their source, and for the resulting clinical guarantees, is substantially less complete.

Critical strengths
  1. The method gives shared medical text a concrete role inside representation learning. It integrates guideline initialization, statement-level calibration, and decoding with patient-specific prototypes, while reconstruction and residual pathways retain record information. Relative to the manuscript's description of CTPD, this is an identifiable and potentially useful extension of cross-modal prototype learning.

  2. The empirical results are encouraging across two different tasks. Compared with CTPD, Table 1 reports mortality AUROC and AUPRC gains of 2.084 and 5.901 percentage points, and phenotype gains of 8.256 and 4.622 points. Five-run summaries, a relevant prototype-based comparator, and performance decreases after each tested component removal provide meaningful initial evidence for the complete system.

Critical weaknesses
  1. The present controls do not establish the central guideline-specific interpretation. The reference combines Wikipedia, PubMed, textbooks, and 45,000 unspecified guidelines, then uses an LLM to synthesize statements. Appendix F supplies a prompt and one sourced example, but not the actual statement bank, its full provenance, or a documented fidelity check. Calling it the sole gold standard therefore assumes a status not established in the paper. The ablations also change more than knowledge content. Removing G-Init replaces a guideline mean plus noise with standard deviation 0.01 by N(0, I); removing G-Calib deletes an attention operation. These comparisons do not isolate correct medical semantics from initialization geometry or additional computation, and Table 2 does not independently remove the alignment objective. The authors need matched reference-content controls and a documented check of the synthesized bank to support the claimed mechanism. Similarity to selected concepts from the same reference and attention on one patient do not establish that spurious associations are avoided or predictions conform to guidelines. Those claims, and the conclusion's assurance of trustworthy real-world deployment, should be narrowed unless directly validated (Sections 3.3, 4.3 to 4.5, and Appendix F).

  2. The benchmark advantage is difficult to assess without a more explicit common evaluation protocol. A patient-level split is reported, but the paper does not define the task-specific measurement and note cutoffs, note-availability timestamps, precise cohort exclusions, or the training-only fitting of preprocessing statistics. Causal masking inside interpolation does not establish that all inputs respect the prediction window. The phenotype metric averaging and F1 threshold-selection rules are also unspecified. Baseline adaptation is particularly important here. Appendix D describes DAFT and MedFuse in image-related settings, whereas the present inputs are notes and time series; the replacements and tuning procedures are not given. Some baseline AUROCs are near or below chance, including SeFT on mortality and DGM-O on phenotyping, which warrants implementation and convergence checks rather than an assumption that these methods are intrinsically weak. These observations do not prove leakage or unfairness, but they prevent a reliable assessment of the large reported gains. Please provide reproducible cohort, input, metric, and baseline configurations, together with the paired evaluation details for the main comparisons (Section 4.1, Table 1, and Appendices A, D, and E).

  3. Equation 10 leaves the core alignment objective underspecified. The positive and negative scores are named but not defined mathematically. Although Section 3.2 identifies the pooled summary as the alignment target, the loss still needs the exact similarity function, source prototypes, normalization, temperature, negative set, and handling of j = k. Please also state whether gradients update the guideline encoder. With a common positive reference and prototype negatives, these choices determine the mechanism being proposed. This is a substantive reproducibility issue, not a typographical detail (Sections 3.2 and 3.3, Equation 10).

  4. The phenotype AUPRC improvement in Section 4.2 is incorrect. Table 1 gives 53.555 for GAMP and 48.933 for CTPD, so the gain is 4.622 percentage points, approximately 4.62, rather than 6.62. Please correct the prose. The correction does not change the ranking in the table.

  5. Figure 2 needs a clearer and more faithful account of the computation. Dense diagonal connections inside the Transformer blocks, long connectors crossing module boundaries, and small rotated labels obscure the main data flow. Replace decorative connectivity with compact blocks, route inter-module arrows consistently, and define the concatenation and addition symbols. More importantly, distinguish the statement bank from the pooled guideline summary and explicitly show statement-level calibration, the shared patient representation in Equation 12, and the weighted prediction fusion in Equation 15. The current two Transformer blocks and branch-specific outputs leave these relationships ambiguous. Please also clarify whether the two blocks share parameters and how they produce the common representation used by both prediction branches.

Questions for authors
  1. Can you provide the final statement bank, source and version mapping, and the procedure used to verify synthesized statements? Do the gains persist with initialization scale and model capacity held fixed while reference content is replaced by unrelated or semantically altered medical text? Please also isolate the alignment term.

  2. What are the exact information cutoffs and note-availability rules for each task? Please specify cohort exclusions, per-split counts, preprocessing fit boundaries, and how repeated stays remain grouped by patient. How were the image-related baselines adapted and tuned on the same inputs?

  3. What exact formulas and gradient paths implement Equation 10? Please identify the clinical Bert-tiny checkpoint and its training or freezing policy, and clarify which quantities enter the positive and negative scores.

  4. Which macro or micro averaging conventions and F1 thresholds produced the phenotype results? Were the five seeds paired across methods on one fixed split, and can you report patient-level paired uncertainty for the main differences? These clarifications would materially affect my confidence in the comparison.

2 comments
AuthorsOpenReview ↗

Response to Comments on Critical Weaknesses

W1. On the failure of the control experiments to establish guideline specificity and the lack of full provenance and fidelity checks for the synthesized statement bank.

We agree the "sole gold standard" wording lacked verification and the original ablations confounded medical semantics with initialization geometry and extra computation. With architecture, initialization scale (σ=0.01\sigma=0.01), capacity, split, and training all fixed, we added three matched reference-content controls (Section 4.3, Table 2)—mismatched guidelines, irrelevant medical text, and non-semantic learnable vectors. All degrade clearly below full GAMP (48-IHM AUROC 81.872/81.245/80.936; 24-PHE 78.246/77.658/77.124 vs. 85.567/87.579), confirming the gain comes from clinically relevant content, not conditioning or regularization; construction details are in Appendix E. Appendix F now documents the bank (1,284 disease-specific statements with unique IDs), source/version mapping, conflict resolution, and two checks: pre-LLM quality screening (removing duplicate/outdated/low-quality references) and post-LLM independent cross-checking with consistency voting by several medical-domain LLMs (no human clinicians available), with non-consensus statements flagged, revised, or discarded.

W2. On the lack of a clear unified evaluation protocol, baseline adaptation, and convergence checks for low-performing baselines.

We reorganized the first appendix into "Experimental Details" with a new "Task Definition and Evaluation Protocol" subsection: 48-IHM is binary on the first 48h after ICU admission; 24-PHE is 25-label multilabel on the first 24h. Notes are timestamp-filtered, excluding late entries beyond the cutoff and discharge summaries. The cohort keeps adults (≥18) with valid series and notes; repeated stays are grouped and split patient-level (7:1:2), with normalization/imputation fitted on training only. Phenotype AUROC/AUPRC are macro-averaged over 25 labels; F1 sweeps thresholds on validation and fixes the validation-F1-maximizing one for test (per label in PHE, same for IHM), not a fixed 0.5. Appendix D details adaptations: DAFT's image–tabular affine conditioning is transferred to series/text and reimplemented; MedFuse's image branch is replaced by series/text, keeping its LSTM fusion and missing-modality handling. SeFT's low IHM AUROC reflects permutation-invariant set aggregation under sparse, high-missingness inputs; DGM-O's near-chance PHE reflects a generative clustering objective mismatched to multilabel prediction—convergence, hyperparameter, and multi-initialization checks confirm neither is an implementation error. All baselines share GAMP's split, windows, and tuning budget.

W3. On the insufficient description of the core alignment objective in Equation 10.

We revised Eq. 10 to make the matching explicit and exclude self-pairs from the negatives. The positive score is the scaled dot-product between a guideline-calibrated prototype and the pooled guideline representation (the shared positive reference); the negative score is the scaled dot-product between that prototype and each of the other K−1K−1 same-modality prototypes, with self-pairs$ (j≠k)$ excluded. The guideline encoder is fine-tuned end-to-end, so alignment gradients flow through the calibrated prototypes, the pooled reference, and the negatives to update the shared BERT. This specifies the exact alignment form and improves reproducibility.

W4. On the numerical error in the phenotype AUPRC improvement reported in Section 4.2.

Confirmed: per Table 1, GAMP 53.555 and CTPD 48.933 give 53.555−48.933=4.62253.555-48.933=4.622 (≈4.62), not 6.62. We corrected the text; the table ranking and all other conclusions are unchanged. We apologize for this.

W5. On the unclear illustration of the computation in Figure 2 and the ambiguous parameter relationship between the two Transformer blocks.

The two Transformer blocks are marked "Shared": one parameter set, drawn separately only to show the two modality-specific prototype streams. Parameter sharing is distinct from the common patient representation: Equation 12 forms the shared representation H\mathbf{H} from the time-series and text features/prototypes, and both prediction branches query this same H\mathbf{H}, fusing outputs by the weighted sum in Equation 15. The layout follows Section 3's three cascaded modules (temporal encoding, guideline-anchored prototype alignment, dual-branch prediction), with arrows tracing the actual data flow (Equation 12 concatenation, residual calibration, Equation 15 fusion) rather than decorative connectivity. The statement bank Z\mathbf{Z} and its pooled summary zˉ\bar{\mathbf{z}} are distinct because their roles differ (statement-level attention keys/values versus shared initialization, alignment target, and guideline query). We respectfully believe this design faithfully reflects the computation, and added the "Shared" annotation and these equation references to resolve the ambiguity.

AuthorsOpenReview ↗

Response to Comments on Questions

Q1. Statement-bank release, provenance, verification, and matched controls

We thank the reviewer for the follow-up and appreciate the importance of reproducibility. Copyright and licensing restrictions currently prevent release of the full statement bank. After publication, we plan to release a controlled-access or de-identified version, subject to resolving the relevant licensing requirements. Appendix F documents the pipeline: guideline sources, MedCPT retrieval, Qwen2.5-72B synthesis, source/version mapping, conflict resolution, and two quality checks. Each statement has a unique identifier linked to document IDs, retrieval scores, publication dates, and prompt versions. Verification includes pre-LLM screening and independent cross-checking with consistency voting by multiple medical-domain LLMs; disputed statements are revised or discarded.

While keeping the other configurations fixed, we added three matched reference-content controls in Table 2, replacing the original guidelines with mismatched guidelines, irrelevant medical text, and learnable vectors, respectively. The newly added control data are as follows:

Reference content48-IHM24-PHE
AUROCAUPRCF1AUROCAUPRCF1
w/ Mismatched G.81.87243.25643.51878.24646.83245.924
w/ Irrelevant Med. Text81.24542.68343.12677.65846.34745.413
w/ Learnable Vectors80.93642.21442.78277.12445.92644.968

These results support the contribution of clinically relevant reference content beyond conditioning or regularization alone. Regarding alignment, Appendix C varies λ1\lambda_1 while holding other components fixed. Performance declines below the optimal weight, supporting the objective’s contribution within the tested range. Table 2 additionally removes guideline calibration (w/o G-Calib), which yields the largest performance drop among all ablation variants. The matched controls further assess reference semantics under fixed initialization and computation. We retain these complementary analyses, while acknowledging that neither removing calibration nor testing small positive weights is equivalent to setting λ1=0\lambda_1=0. We therefore interpret them as supporting evidence.

Q2. Evaluation windows, cohort, splits, preprocessing, and baseline adaptations

48-IHM uses the first 48 hours after ICU admission with one sigmoid output for binary classification; 24-PHE uses the first 24 hours with 25 independent sigmoid outputs. Notes are filtered using charting and storage timestamps. Notes with ambiguous timing, entries stored after the cutoff, and discharge summaries are excluded to prevent information outside the observation window from entering the inputs.

The cohort includes adults aged ≥18 with valid time series and eligible notes; incomplete or insufficient records are excluded. We use a patient-level 7:1:2 split, keeping all ICU stays from each patient in the same partition. For 19,316 IHM records, ratio-based train/validation/test estimates are 13,521/1,932/3,863; for 38,645 PHE records, they are 27,052/3,864/7,729.Normalization statistics and imputation parameters are fitted exclusively on training data and applied unchanged to validation and test data.

Appendix D explains the baseline adaptations. DAFT originally uses tabular information to scale and shift image feature maps; we transfer this affine conditioning mechanism to time-series/text features. MedFuse originally fuses clinical time series and chest X-rays through an LSTM; we replace its image branch with text/time-series features while retaining LSTM fusion and missing-modality handling. All baselines use GAMP’s input windows, split, and tuning budget.

Q3. Equation 10, BERT-tiny, and gradient paths

Clinical notes and guidelines share the BERT-tiny, with a maximum input length of 512 tokens. The encoder is fine-tuned end to end. Other trainable components include the projection, positional encodings, prototype slots, calibration and interaction Transformers, time-series encoder, and prediction heads.

We have clarified the quantities entering the alignment scores. The positive score measures the dot product between each calibrated prototype and the pooled guideline representation, which serves as the shared positive reference. Negative scores measure dot products between that prototype and the remaining K−1K-1 prototypes from the same modality, explicitly excluding self-pairs. All scores are divided by D\sqrt{D}, with no additional normalization or temperature.

Alignment gradients propagate through the calibrated prototypes, negative prototypes, and pooled guideline representation to their upstream trainable components, including the shared BERT encoder. Thus, the guideline reference is optimized jointly with the model.

Reviewer_4eUgOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces GAMP to combine physiological time series and clinical notes for clinical prediction. The main idea is to use clinical guidelines to guide what the model learns from each patient. Guideline embeddings initialize prototype queries, refine the resulting prototypes, and help align their representations. A dual-branch decoder then combines guideline-based and patient-specific information. The authors evaluate GAMP on mortality prediction and phenotype classification in MIMIC-III and report improvements over all evaluated baselines. The idea is appealing, and the gains over CTPD are substantial.

Critical strengths

The paper tackles a worthwhile problem. Models can perform well on EHR benchmarks while relying on associations that have little clinical meaning. Using medical knowledge to guide representation learning is a sensible direction, and this paper makes a concrete attempt to do that.

I also like that the model leaves room for patient information that is not covered by the guidelines. The prototypes are updated using each patient’s record, and the residual connections and reconstruction losses are intended to preserve that information. This is a reasonable design choice given that guidelines cannot describe every patient or clinical situation.

The reported results are good. Compared with CTPD, mortality AUPRC increases from 43% to 49%, while phenotype AUROC increases from 79% to 87%. The authors also include repeated runs, component ablations, and sensitivity analyses. These give a useful starting point for understanding the model.

Critical weaknesses

My main concern is the misalignment between the results and the claims. The ablations remove parts of the model, but they also change the architecture or initialization. What happens if the authors keep everything else the same and replace the guidelines with unrelated medical text or learned vectors?

Question regarding trust to put in the guideline collection. The authors combine several types of sources and use an LLM to summarize them. That seems reasonable as a way to collect medical information, but calling the result a gold standard feels too strong without showing how it was checked; e.g., were clinicians involved? What happened when sources disagreed? More details are required.

Interpretability. In Figure 4, the model highlights pneumothorax in a passage about ruling it out, and airway injury in a passage about preventing it. Those terms are relevant, but highlighting them does not tell us whether the model understood what the note actually said. An example would be more useful if the authors showed the prediction and how it changed when these passages were edited or removed. The claims about avoiding spurious associations and being ready for real-world use go beyond what the experiments show.

Questions for authors

How exactly does the model decide what information it sees before making a prediction? The patient-level split is reassuring, but the handling of late-entered notes and discharge summaries needs to be spelled out. The paper also seems to switch between predicting death during hospitalization using the first 48 hours and predicting death within 48 hours. These are different tasks, so this needs to be clarified.

Modeling choices explanation. Equation 10 does not fully define the matching scores, and I do not quite see why bringing every prototype closer to the same pooled guideline embedding should give the prototypes different clinical meanings. Can you explian why the chosen fusion weight gives the guideline branch only 5% of the final prediction? Guidelines still affect the earlier layers, so this does not mean they are unimportant, but it does leave me wondering what the second prediction branch adds.

The paper should also explain how the phenotype metrics were averaged and how the F1 thresholds were chosen. There is a small numerical error as well. The phenotype AUPRC improvement over CTPD is 4.622 percentage points, not 6.62.

3 comments
AuthorsOpenReview ↗

Response to Comments on Critical Weaknesses

W1. On the ablations changing the architecture or initialization while removing components and the request to fix other conditions before replacing the guidelines.

We thank you for pointing out this key issue. We agree that the original ablations changed the initialization or computation while removing components, making it impossible to separate medical semantics from initialization geometry or extra computation. While keeping the architecture, initialization scale (σ=0.01\sigma=0.01), model capacity, data split, and all training configurations completely fixed, we added three matched reference-content controls in Table 2, replacing the original guidelines with mismatched guidelines, irrelevant medical text, and learnable vectors, respectively. The newly added control data are as follows:

Reference content48-IHM24-PHE
AUROCAUPRCF1AUROCAUPRCF1
w/ Mismatched G.81.87243.25643.51878.24646.83245.924
w/ Irrelevant Med. Text81.24542.68343.12677.65846.34745.413
w/ Learnable Vectors80.93642.21442.78277.12445.92644.968

All three clearly degrade and none approaches the full model: the 48-IHM AUROC values are 81.872, 81.245, and 80.936, and the 24-PHE values are 78.246, 77.658, and 77.124, compared with 85.567 and 87.579 for the full GAMP. This shows that the improvement specifically comes from meaningful clinical knowledge rather than from extra conditioning or regularization.

W2. On the credibility of the guideline collection, the verification methods and source-conflict handling, and the overly strong "gold standard" wording.

We agree that calling the synthesized results a gold standard without describing the verification procedure is too absolute. We expanded Appendix F, clarifying that when sources conflict, authoritative guidelines and textbooks are prioritized with preference for newer versions, other sources are assessed based on quality and cross-source agreement, and unresolved conflicts are handled conservatively and flagged. We also provide two reliability measures: before feeding the LLM, the literature is quality-screened to remove duplicate, outdated, and low-quality references; after LLM refinement, because no human clinical personnel are available, several advanced medical-domain LLMs independently cross-check and perform consistency voting, with disputed statements returned for revision or discarded. We accordingly position the statement bank as a traceable shared semantic reference rather than a clinical gold standard, thereby avoiding absolute wording.

W3. On highlighting terms not being sufficient to demonstrate understanding, the suggestion to show how predictions change when passages are edited or removed, and the real-world deployment claims exceeding the evidence.

We thank you for this suggestion, which is exactly the focus of our additions. We conduct counterfactual text interventions on 500 randomly sampled test cases, removing clinically relevant sentences, flipping negation wording (e.g., changing "pneumothorax ruled out" to "pneumothorax present"), and removing irrelevant sentences, and record the absolute change in predicted probability relative to the original note. The results show that clinically relevant edits produce markedly larger prediction shifts: removing clinically relevant sentences changes the prediction by 0.184 on average and negation flips by 0.147, whereas removing irrelevant sentences changes it by only 0.023, close to the re-encoding noise, indicating that the model responds to contextualized clinical content rather than mere keyword matching.

AuthorsOpenReview ↗

Response to Comments on Questions

Q1. On the handling of late-entered notes and discharge summaries and the ambiguity in the 48-IHM task definition.

We thank you for the confirmation request. Regarding note availability, the appendix task definition already clarifies that notes are selected by their charting and storage timestamps, and notes with ambiguous timing, late-entered notes beyond the prediction cutoff, and discharge summaries are excluded, ensuring that no information outside the window enters the input. Regarding the task definition, the appendix already states that 48-IHM is a single-sigmoid binary classification that uses clinical observations collected within the first 48 hours after ICU admission to predict whether the patient dies during the entire hospitalization; the 48 hours refers to the input observation window rather than the prediction interval, and the target remains in-hospital mortality over the full stay, not mortality within the first 48 hours. The same applies to 24-PHE, where 24 hours is only the input window.

Q2. On the definition of the matching scores in Equation 10, why the shared pooled embedding can still assign distinct meanings to the prototypes, and the fusion weight w=0.05 and the role of the second branch.

The matching scores are completed in Equation 10; both the positive and negative scores use scaled dot-products divided by D\sqrt{D} with no additional normalization or temperature, the negatives are the remaining K−1K-1 prototypes within the same modality with self-pairs excluded (j≠kj\neq k), and the guideline encoder is fine-tuned end-to-end during training. On why the shared pooled embedding does not make the prototypes converge, the paper already distinguishes the roles of the pooled summary and the statement bank: the pooled summary is used only where a single shared reference is needed, for initialization, the alignment target, and the guideline query, whereas the statement bank retains individual clinical criteria as separate attention keys and values, allowing different prototypes to weight different statements. The prototype diversity analysis also shows that the pairwise cosine similarities of the 16 prototypes are clearly heterogeneous and do not collapse, and different prototypes selectively align with different clinical concepts, indicating that the shared anchor provides a common semantic coordinate while the differentiated roles are produced by data-driven attention allocation and inter-prototype negative repulsion. On the fusion weight, the hyperparameter analysis already states that ww is selected on the validation set with an optimal value of w=0.05w=0.05, i.e., at the final linear fusion the individual-specificity branch carries the main individualization signal. However, the guidelines already shape the shared representation used by both branches at earlier stages through prototype initialization, statement-level calibration, and the alignment objective; therefore a small end-point weight does not mean the guidelines are unimportant, and the guideline-evidence branch provides standardized, traceable calibration. The w/o Dual-Branch result in Table 2 (48-IHM AUROC 80.752, 24-PHE 76.890) also confirms the joint value of the dual branches.

Q3. On how the phenotype metrics are averaged, how the F1 thresholds are selected, and the numerical error in the phenotype AUPRC improvement.

The phenotype AUROC, AUPRC, and F1 are computed separately for the 25 labels and then macro-averaged; no micro averaging is used, while 48-IHM is a single label and is computed directly. For F1, a fixed 0.5 threshold is not used; instead, candidate thresholds are swept over the validation set and the one maximizing the validation F1 is selected, with an independent threshold for each label in PHE and the same logic for IHM; the thresholds are then fixed for the test set and the threshold optimization does not touch test labels. The numerical error you point out is real: the correct phenotype AUPRC improvement over CTPD is 53.555−48.933=4.62253.555-48.933=4.622 percentage points, and we corrected the 6.62 in the text to 4.62. This correction does not change the ranking in the table or other conclusions, and we sincerely apologize for the oversight.

I thank the authors for their response to my review comments. Your response has helped me better understand the paper. After considering it carefully, I believe that my current overall rating still reflects my assessment of the paper.

Reviewer_WDe9OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GAMP, a guideline-anchored multimodal representation learning framework for clinical outcome prediction using physiological time series and clinical notes. The core idea is to incorporate clinical guideline knowledge into patient-specific prototype learning, encouraging learned representations to capture clinically meaningful patterns rather than relying solely on statistical correlations.

GAMP initializes shared prototype queries from guideline embeddings and iteratively extracts patient-specific prototypes from time series and clinical notes using Slot Attention. Statement-level guideline attention calibrates the prototypes, while alignment and reconstruction objectives encourage clinical consistency and preservation of patient information. A dual-branch prediction module combines guideline-based evidence and patient-specific representations.

The method is evaluated on MIMIC-III for 48-hour in-hospital mortality prediction and 24-hour phenotype classification. GAMP outperforms the evaluated baselines across AUROC, AUPRC, and F1. Ablation studies, prototype analysis, and a qualitative case study provide additional evidence for the contributions of guideline anchoring and multimodal representation learning.

Critical strengths

1. Clinically motivated integration of external knowledge into multimodal representation learning.

The paper addresses an important limitation of purely data-driven EHR models by incorporating guideline knowledge directly into prototype initialization, calibration, and representation alignment. Rather than simply concatenating clinical knowledge with patient features, the proposed approach uses guideline information throughout the representation learning process. This is a meaningful design choice with potential benefits for clinical prediction and interpretability.

2. Strong empirical performance with reasonably comprehensive ablations.

GAMP consistently outperforms the evaluated baselines on both clinical prediction tasks. In particular, compared with the prototype-based CTPD baseline, GAMP improves AUROC from 83.483 to 85.567 on mortality prediction and from 79.323 to 87.579 on phenotype classification. The ablation studies further show substantial performance degradation when guideline initialization or calibration is removed, providing evidence that guideline anchoring contributes to the observed improvements.

Critical weaknesses

1. Limited validation of the claimed clinical interpretability and guideline alignment.

A central motivation of GAMP is that guideline-anchored prototypes should capture clinically meaningful patterns and reduce spurious correlations. However, the supporting evidence mainly consists of prototype similarity heatmaps, assignment entropy distributions, and a qualitative case study involving a single patient.

These analyses demonstrate changes in learned representations, but they do not establish whether the prototypes correspond to clinically valid concepts or whether the resulting predictions are more clinically faithful. For example, similarity between prototypes and guideline embeddings does not necessarily imply that the underlying patient evidence supports the associated clinical concepts. Stronger evaluation using expert annotations, concept-level agreement, or controlled perturbations of guideline knowledge would be needed to substantiate the interpretability claims.

2. Insufficient isolation of the contribution and robustness of guideline knowledge.

The proposed method introduces several components, including guideline-based initialization, statement-level calibration, alignment loss, reconstruction, and a dual-branch decoder. While the reported ablations demonstrate the contributions of several architectural components, the experiments do not fully isolate the benefit of clinically meaningful guideline knowledge itself.

In particular, there is no direct comparison against randomly selected, shuffled, or irrelevant guideline statements using the same architecture. Such controls would help determine whether the observed improvements arise from meaningful clinical knowledge or from additional conditioning and regularization.

Furthermore, the guidelines are constructed using retrieval from heterogeneous sources followed by Qwen2.5-72B summarization. The paper provides limited evidence regarding the clinical correctness, consistency, and robustness of these automatically generated guidelines. Since the proposed method depends heavily on guideline quality, this is an important limitation.

Questions for authors

Q1. Contribution of clinically meaningful guideline knowledge: Could the authors evaluate GAMP using shuffled, random, or unrelated guideline statements while keeping the same architecture and training procedure? This would help establish whether the improvements arise specifically from clinically relevant knowledge rather than additional conditioning or regularization.

Q2. Clinical validity of learned prototypes: How do the authors verify that guideline-aligned prototypes represent clinically meaningful patient patterns? Could they provide quantitative evaluation using expert-annotated clinical concepts, guideline-evidence agreement, or another independent measure of clinical faithfulness?

Q3. Guideline construction and robustness: How are the retrieved and LLM-generated guidelines validated for factual correctness and consistency? How sensitive is performance to the choice of guideline sources, retrieval results, or the LLM used for summarization?

Q4. Role of the guideline prediction branch: Appendix C reports that the optimal fusion weight for the guideline-evidence branch is w=0.05w=0.05. Does this suggest that most predictive information is provided by the patient-specific branch? Could the authors report separate branch performance and clarify the contribution of guideline-guided representation learning versus direct guideline-based prediction?

2 comments
AuthorsOpenReview ↗

Response to Comments on Critical Weaknesses

W1. On the limited verification of clinical interpretability and guideline alignment, with evidence mostly being heatmaps and a single-patient case study.

We thank you for your rigorous comments. We agree that similarity heatmaps, assignment entropy distributions, and a single-patient case study can only show that the representations have changed and are insufficient to establish that the prototypes correspond to clinically valid concepts. To address this, we added two stronger types of evidence. The first is controlled perturbation: counterfactual text interventions on 500 test cases, where removing clinically relevant sentences changes the prediction by 0.184 on average and negation flips by 0.147, while removing irrelevant sentences changes it by only 0.023, indicating that the model responds to contextualized clinical content rather than keywords. The second is a quantitative concept-level alignment agreement: the dominant patient evidence each prototype binds through Slot Attention is matched against its highest-weight guideline concept, and the average agreement across the 16 prototypes is 0.81 on 48-IHM and 0.79 on 24-PHE, far above the 0.05 chance level for a 20-way choice, indicating that the guideline-aligned prototypes are indeed supported by corresponding patient evidence rather than merely lying near guideline embeddings. We honestly note that, since no human expert annotation resources are available, the factual correctness and consistency of the statement bank are guaranteed by the quality filtering, cross-voting among multiple medical LLMs, and conflict-resolution pipeline described in Appendix F. These automated workflows cannot fully replace expert clinical validation.

W2. On the insufficient isolation of the contribution of guideline knowledge and the lack of a comparison with random or irrelevant statements under the same architecture.

While keeping the architecture, initialization scale (σ=0.01\sigma=0.01), model capacity, data split, and training configurations completely fixed, we added three matched reference-content controls in Table 2, replacing the original guidelines with mismatched guidelines, irrelevant medical text, and learnable vectors, respectively. The newly added control data are as follows:

Reference content48-IHM24-PHE
AUROCAUPRCF1AUROCAUPRCF1
w/ Mismatched G.81.87243.25643.51878.24646.83245.924
w/ Irrelevant Med. Text81.24542.68343.12677.65846.34745.413
w/ Learnable Vectors80.93642.21442.78277.12445.92644.968

All three clearly degrade and none approaches the full model: the 48-IHM AUROC values are 81.872, 81.245, and 80.936, and the 24-PHE values are 78.246, 77.658, and 77.124, compared with 85.567 and 87.579 for the full GAMP. This shows that the improvement specifically comes from meaningful clinical knowledge rather than from extra conditioning or regularization.

W3. On the limited evidence for the clinical correctness, consistency, and robustness of the automatically generated guidelines.

For correctness and consistency, Appendix F provides the complete pipeline: quality screening before feeding the LLM to remove duplicate, outdated, and low-quality references, and after LLM refinement independent cross-checking and consistency voting by multiple medical-domain LLMs, with source conflicts resolved by prioritizing authoritative guidelines and textbooks and preferring newer versions. We also candidly note that, owing to resource constraints, the paper has not yet systematically assessed the sensitivity of performance to the choice of guideline sources, retrieval depth, or summarization model; this robustness analysis is a limitation of the current work, and we leave it as a direction for future research. The present conclusions are based on the fixed MedCPT retrieval and Qwen2.5-72B synthesis pipeline and should not be extrapolated to other construction configurations.

AuthorsOpenReview ↗

Response to Comments on Critical Questions

Q1. On evaluating GAMP with shuffled, random, or irrelevant statements under the same architecture and training pipeline.

This control is already completed in Table 2. The three groups of mismatched guidelines, irrelevant medical text, and learnable vectors all replace the original guidelines with the architecture, σ\sigma, capacity, split, and training configurations completely fixed, and all results are significantly below the full model: the 48-IHM AUROC values are 81.872, 81.245, and 80.936, and the 24-PHE values are 78.246, 77.658, and 77.124, compared with 85.567 and 87.579 for the full GAMP. It can be confirmed that the gains specifically come from clinically relevant knowledge rather than from conditioning or regularization.

Q2. On how to verify that the guideline-aligned prototypes represent clinically meaningful patient patterns and whether an independent clinical fidelity measure can be provided.

We added an automated concept-level alignment agreement as an independent measure, comparing the dominant patient evidence each prototype binds against its highest-weight guideline concept and recording agreement when both point to the same clinical concept. The agreement across the 16 prototypes is 0.81 on 48-IHM and 0.79 on 24-PHE, far above the 0.05 chance level. Combined with the clear prediction changes from clinically relevant edits in the counterfactual interventions (0.184, 0.147) and the very small change from irrelevant edits (0.023), this supports the clinical validity of the prototypes from both concept-consistency and controlled-perturbation perspectives. We also honestly state that this is an automated proxy measure and cannot yet replace expert annotation of clinical concepts.

Q3. On how the retrieved and LLM-generated guidelines are verified for factual correctness and consistency and the sensitivity of performance to the source and model choices.

Factual correctness and consistency are ensured by the quality screening, multi-LLM cross-voting, and conflict-resolution procedures in Appendix F. Regarding the sensitivity you ask about, we honestly state that the current paper has not yet conducted systematic performance-perturbation experiments across different guideline sources, retrieval results, or summarization LLMs, and therefore cannot provide quantitative sensitivity intervals; we recognize this as a limitation. Under the fixed construction pipeline, the cross-model consistency of the statements is already examined by the multi-LLM voting, while a more complete robustness analysis will be left to future work, and we avoid making claims about stability under other configurations that go beyond the evidence.

Q4. On whether w=0.05 means the patient branch carries the main information, the branch-only performance, and the respective contributions of guideline-guided representation learning and direct guideline-based prediction.

We first report branch-only performance, then explain w=0.05w=0.05.

Setting48-IHM24-PHE
AUROCAUPRCAUROCAUPRC
Guideline branch only (w=1w=1)80.21444.12882.36749.742
Patient branch only (w=0w=0)84.92548.73687.14352.918
Fused model (w=0.05w=0.05)85.56749.51587.57953.555

Crucially, ww is the relative weight of the two query perspectives at the output, not the total contribution of guideline knowledge. Both decoders query the same guideline-anchored memory HPat\mathbf{H}_{Pat} and differ only in query source: the guideline branch uses the pooled guideline embedding to retrieve standardized cross-patient knowledge, while the patient branch uses the patient's note embedding to retrieve individual atypical manifestations and comorbidities. Since diagnosis concerns a specific individual, the discriminative signal comes from the patient's own observations, whereas shared guideline knowledge carries no individual discriminative information; patient-specific evidence thus naturally dominates the final fusion (w=0.05w=0.05). More importantly, the guidelines are already internalized into the shared memory via initialization, statement-level calibration, and alignment before retrieval, so the patient branch retrieves guideline-purified representations. This mirrors Bayesian inference: the posterior is data-driven, but the prior defines the solution space and regularizes the estimate. The fused model outperforming either branch confirms this complementarity, and ww is selected on validation rather than fixed by hand.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.