- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
GAMP predicts in-hospital mortality and 25 phenotypes from physiological time series and clinical notes. Synthesized medical statements initialize shared prototype queries, calibrate patient-specific prototypes through attention, and provide a latent alignment target. Reconstruction retains record information, and a dual-branch decoder combines guideline and patient-note queries. On two MIMIC-III tasks, the paper reports five-run improvements over fourteen baselines, component ablations, and qualitative prototype analyses. The reported predictive gains are promising. The evidence for medically valid guideline semantics as their source, and for the resulting clinical guarantees, is substantially less complete.
Critical strengths
-
The method gives shared medical text a concrete role inside representation learning. It integrates guideline initialization, statement-level calibration, and decoding with patient-specific prototypes, while reconstruction and residual pathways retain record information. Relative to the manuscript's description of CTPD, this is an identifiable and potentially useful extension of cross-modal prototype learning.
-
The empirical results are encouraging across two different tasks. Compared with CTPD, Table 1 reports mortality AUROC and AUPRC gains of 2.084 and 5.901 percentage points, and phenotype gains of 8.256 and 4.622 points. Five-run summaries, a relevant prototype-based comparator, and performance decreases after each tested component removal provide meaningful initial evidence for the complete system.
Critical weaknesses
-
The present controls do not establish the central guideline-specific interpretation. The reference combines Wikipedia, PubMed, textbooks, and 45,000 unspecified guidelines, then uses an LLM to synthesize statements. Appendix F supplies a prompt and one sourced example, but not the actual statement bank, its full provenance, or a documented fidelity check. Calling it the sole gold standard therefore assumes a status not established in the paper. The ablations also change more than knowledge content. Removing G-Init replaces a guideline mean plus noise with standard deviation 0.01 by N(0, I); removing G-Calib deletes an attention operation. These comparisons do not isolate correct medical semantics from initialization geometry or additional computation, and Table 2 does not independently remove the alignment objective. The authors need matched reference-content controls and a documented check of the synthesized bank to support the claimed mechanism. Similarity to selected concepts from the same reference and attention on one patient do not establish that spurious associations are avoided or predictions conform to guidelines. Those claims, and the conclusion's assurance of trustworthy real-world deployment, should be narrowed unless directly validated (Sections 3.3, 4.3 to 4.5, and Appendix F).
-
The benchmark advantage is difficult to assess without a more explicit common evaluation protocol. A patient-level split is reported, but the paper does not define the task-specific measurement and note cutoffs, note-availability timestamps, precise cohort exclusions, or the training-only fitting of preprocessing statistics. Causal masking inside interpolation does not establish that all inputs respect the prediction window. The phenotype metric averaging and F1 threshold-selection rules are also unspecified. Baseline adaptation is particularly important here. Appendix D describes DAFT and MedFuse in image-related settings, whereas the present inputs are notes and time series; the replacements and tuning procedures are not given. Some baseline AUROCs are near or below chance, including SeFT on mortality and DGM-O on phenotyping, which warrants implementation and convergence checks rather than an assumption that these methods are intrinsically weak. These observations do not prove leakage or unfairness, but they prevent a reliable assessment of the large reported gains. Please provide reproducible cohort, input, metric, and baseline configurations, together with the paired evaluation details for the main comparisons (Section 4.1, Table 1, and Appendices A, D, and E).
-
Equation 10 leaves the core alignment objective underspecified. The positive and negative scores are named but not defined mathematically. Although Section 3.2 identifies the pooled summary as the alignment target, the loss still needs the exact similarity function, source prototypes, normalization, temperature, negative set, and handling of j = k. Please also state whether gradients update the guideline encoder. With a common positive reference and prototype negatives, these choices determine the mechanism being proposed. This is a substantive reproducibility issue, not a typographical detail (Sections 3.2 and 3.3, Equation 10).
-
The phenotype AUPRC improvement in Section 4.2 is incorrect. Table 1 gives 53.555 for GAMP and 48.933 for CTPD, so the gain is 4.622 percentage points, approximately 4.62, rather than 6.62. Please correct the prose. The correction does not change the ranking in the table.
-
Figure 2 needs a clearer and more faithful account of the computation. Dense diagonal connections inside the Transformer blocks, long connectors crossing module boundaries, and small rotated labels obscure the main data flow. Replace decorative connectivity with compact blocks, route inter-module arrows consistently, and define the concatenation and addition symbols. More importantly, distinguish the statement bank from the pooled guideline summary and explicitly show statement-level calibration, the shared patient representation in Equation 12, and the weighted prediction fusion in Equation 15. The current two Transformer blocks and branch-specific outputs leave these relationships ambiguous. Please also clarify whether the two blocks share parameters and how they produce the common representation used by both prediction branches.
Questions for authors
-
Can you provide the final statement bank, source and version mapping, and the procedure used to verify synthesized statements? Do the gains persist with initialization scale and model capacity held fixed while reference content is replaced by unrelated or semantically altered medical text? Please also isolate the alignment term.
-
What are the exact information cutoffs and note-availability rules for each task? Please specify cohort exclusions, per-split counts, preprocessing fit boundaries, and how repeated stays remain grouped by patient. How were the image-related baselines adapted and tuned on the same inputs?
-
What exact formulas and gradient paths implement Equation 10? Please identify the clinical Bert-tiny checkpoint and its training or freezing policy, and clarify which quantities enter the positive and negative scores.
-
Which macro or micro averaging conventions and F1 thresholds produced the phenotype results? Were the five seeds paired across methods on one fixed split, and can you report patient-level paired uncertainty for the main differences? These clarifications would materially affect my confidence in the comparison.
2 comments
Response to Comments on Questions
Q1. Statement-bank release, provenance, verification, and matched controls
We thank the reviewer for the follow-up and appreciate the importance of reproducibility. Copyright and licensing restrictions currently prevent release of the full statement bank. After publication, we plan to release a controlled-access or de-identified version, subject to resolving the relevant licensing requirements. Appendix F documents the pipeline: guideline sources, MedCPT retrieval, Qwen2.5-72B synthesis, source/version mapping, conflict resolution, and two quality checks. Each statement has a unique identifier linked to document IDs, retrieval scores, publication dates, and prompt versions. Verification includes pre-LLM screening and independent cross-checking with consistency voting by multiple medical-domain LLMs; disputed statements are revised or discarded.
While keeping the other configurations fixed, we added three matched reference-content controls in Table 2, replacing the original guidelines with mismatched guidelines, irrelevant medical text, and learnable vectors, respectively. The newly added control data are as follows:
| Reference content | 48-IHM | 24-PHE | ||||
|---|---|---|---|---|---|---|
| AUROC | AUPRC | F1 | AUROC | AUPRC | F1 | |
| w/ Mismatched G. | 81.872 | 43.256 | 43.518 | 78.246 | 46.832 | 45.924 |
| w/ Irrelevant Med. Text | 81.245 | 42.683 | 43.126 | 77.658 | 46.347 | 45.413 |
| w/ Learnable Vectors | 80.936 | 42.214 | 42.782 | 77.124 | 45.926 | 44.968 |
These results support the contribution of clinically relevant reference content beyond conditioning or regularization alone. Regarding alignment, Appendix C varies while holding other components fixed. Performance declines below the optimal weight, supporting the objective’s contribution within the tested range. Table 2 additionally removes guideline calibration (w/o G-Calib), which yields the largest performance drop among all ablation variants. The matched controls further assess reference semantics under fixed initialization and computation. We retain these complementary analyses, while acknowledging that neither removing calibration nor testing small positive weights is equivalent to setting . We therefore interpret them as supporting evidence.
Q2. Evaluation windows, cohort, splits, preprocessing, and baseline adaptations
48-IHM uses the first 48 hours after ICU admission with one sigmoid output for binary classification; 24-PHE uses the first 24 hours with 25 independent sigmoid outputs. Notes are filtered using charting and storage timestamps. Notes with ambiguous timing, entries stored after the cutoff, and discharge summaries are excluded to prevent information outside the observation window from entering the inputs.
The cohort includes adults aged ≥18 with valid time series and eligible notes; incomplete or insufficient records are excluded. We use a patient-level 7:1:2 split, keeping all ICU stays from each patient in the same partition. For 19,316 IHM records, ratio-based train/validation/test estimates are 13,521/1,932/3,863; for 38,645 PHE records, they are 27,052/3,864/7,729.Normalization statistics and imputation parameters are fitted exclusively on training data and applied unchanged to validation and test data.
Appendix D explains the baseline adaptations. DAFT originally uses tabular information to scale and shift image feature maps; we transfer this affine conditioning mechanism to time-series/text features. MedFuse originally fuses clinical time series and chest X-rays through an LSTM; we replace its image branch with text/time-series features while retaining LSTM fusion and missing-modality handling. All baselines use GAMP’s input windows, split, and tuning budget.
Q3. Equation 10, BERT-tiny, and gradient paths
Clinical notes and guidelines share the BERT-tiny, with a maximum input length of 512 tokens. The encoder is fine-tuned end to end. Other trainable components include the projection, positional encodings, prototype slots, calibration and interaction Transformers, time-series encoder, and prediction heads.
We have clarified the quantities entering the alignment scores. The positive score measures the dot product between each calibrated prototype and the pooled guideline representation, which serves as the shared positive reference. Negative scores measure dot products between that prototype and the remaining prototypes from the same modality, explicitly excluding self-pairs. All scores are divided by , with no additional normalization or temperature.
Alignment gradients propagate through the calibrated prototypes, negative prototypes, and pooled guideline representation to their upstream trainable components, including the shared BERT encoder. Thus, the guideline reference is optimized jointly with the model.
Response to Comments on Critical Weaknesses
W1. On the failure of the control experiments to establish guideline specificity and the lack of full provenance and fidelity checks for the synthesized statement bank.
We agree the "sole gold standard" wording lacked verification and the original ablations confounded medical semantics with initialization geometry and extra computation. With architecture, initialization scale (σ=0.01), capacity, split, and training all fixed, we added three matched reference-content controls (Section 4.3, Table 2)—mismatched guidelines, irrelevant medical text, and non-semantic learnable vectors. All degrade clearly below full GAMP (48-IHM AUROC 81.872/81.245/80.936; 24-PHE 78.246/77.658/77.124 vs. 85.567/87.579), confirming the gain comes from clinically relevant content, not conditioning or regularization; construction details are in Appendix E. Appendix F now documents the bank (1,284 disease-specific statements with unique IDs), source/version mapping, conflict resolution, and two checks: pre-LLM quality screening (removing duplicate/outdated/low-quality references) and post-LLM independent cross-checking with consistency voting by several medical-domain LLMs (no human clinicians available), with non-consensus statements flagged, revised, or discarded.
W2. On the lack of a clear unified evaluation protocol, baseline adaptation, and convergence checks for low-performing baselines.
We reorganized the first appendix into "Experimental Details" with a new "Task Definition and Evaluation Protocol" subsection: 48-IHM is binary on the first 48h after ICU admission; 24-PHE is 25-label multilabel on the first 24h. Notes are timestamp-filtered, excluding late entries beyond the cutoff and discharge summaries. The cohort keeps adults (≥18) with valid series and notes; repeated stays are grouped and split patient-level (7:1:2), with normalization/imputation fitted on training only. Phenotype AUROC/AUPRC are macro-averaged over 25 labels; F1 sweeps thresholds on validation and fixes the validation-F1-maximizing one for test (per label in PHE, same for IHM), not a fixed 0.5. Appendix D details adaptations: DAFT's image–tabular affine conditioning is transferred to series/text and reimplemented; MedFuse's image branch is replaced by series/text, keeping its LSTM fusion and missing-modality handling. SeFT's low IHM AUROC reflects permutation-invariant set aggregation under sparse, high-missingness inputs; DGM-O's near-chance PHE reflects a generative clustering objective mismatched to multilabel prediction—convergence, hyperparameter, and multi-initialization checks confirm neither is an implementation error. All baselines share GAMP's split, windows, and tuning budget.
W3. On the insufficient description of the core alignment objective in Equation 10.
We revised Eq. 10 to make the matching explicit and exclude self-pairs from the negatives. The positive score is the scaled dot-product between a guideline-calibrated prototype and the pooled guideline representation (the shared positive reference); the negative score is the scaled dot-product between that prototype and each of the other K−1 same-modality prototypes, with self-pairs$ (j≠k)$ excluded. The guideline encoder is fine-tuned end-to-end, so alignment gradients flow through the calibrated prototypes, the pooled reference, and the negatives to update the shared BERT. This specifies the exact alignment form and improves reproducibility.
W4. On the numerical error in the phenotype AUPRC improvement reported in Section 4.2.
Confirmed: per Table 1, GAMP 53.555 and CTPD 48.933 give 53.555−48.933=4.622 (≈4.62), not 6.62. We corrected the text; the table ranking and all other conclusions are unchanged. We apologize for this.
W5. On the unclear illustration of the computation in Figure 2 and the ambiguous parameter relationship between the two Transformer blocks.
The two Transformer blocks are marked "Shared": one parameter set, drawn separately only to show the two modality-specific prototype streams. Parameter sharing is distinct from the common patient representation: Equation 12 forms the shared representation H from the time-series and text features/prototypes, and both prediction branches query this same H, fusing outputs by the weighted sum in Equation 15. The layout follows Section 3's three cascaded modules (temporal encoding, guideline-anchored prototype alignment, dual-branch prediction), with arrows tracing the actual data flow (Equation 12 concatenation, residual calibration, Equation 15 fusion) rather than decorative connectivity. The statement bank Z and its pooled summary zˉ are distinct because their roles differ (statement-level attention keys/values versus shared initialization, alignment target, and guideline query). We respectfully believe this design faithfully reflects the computation, and added the "Shared" annotation and these equation references to resolve the ambiguity.