acceptodds
Under review as a conference paper at ICLR 2027

PORTER: Portable Representations for Transferable Electronic Health Records

Abstract

Clinical prediction models often degrade across sites because differences in clinical practice and documentation induce site-specific joint distributions over observed records and outcomes. These shifts arise from the underlying data-generating process and therefore cannot be fully addressed by increasing downstream model capacity alone. We instead focus on constructing representations that support transferable prediction across sites. We propose Portable Representations for Transferable Electronic Health Records (PORTER), a summarize-then-embed pipeline built using a clinical schema designed to reduce site-specific variation, alongside a fine-tuning method to addresses variation unresolvable by schema alone. We design the schema to guide the LLM to preserve predictive information while ignoring site-specific noise. However, a site-independent representation alone cannot capture how predictions should be adapted to a new site. To address this limitation, we build on the site-invariant representation and further adapt its predictions by distilling target-site policies from a small support set. Across three ICU datasets, six transfer directions, and five clinical prediction tasks, schema-only PORTER achieves the best average rank among all 13 methods in zero-shot transfer. Its target-site distillation procedure also achieves the best average rank in the few-shot setting, with the largest gains on tasks that transfer most poorly under zero-shot setting. We also experiment with PORTER on longitudinal EHR data where we find PORTER improves with increasing context length. Our work demonstrates that LLMs can be used not only to encode clinical text, but to generate representations explicitly designed to improve cross-site clinical prediction.

Primary area:
learning on time series and dynamical systems
Keywords:
ML for Healthcare, Representation Learning, Transfer Learning, Large Language Model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_4ZoDOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes PORTER, a summarize-then-embed pipeline that uses a clinical schema to guide an LLM to produce textual summaries stripped of site-specific variation, followed by site-adaptive fine-tuning. Transfer is evaluated on cross-site laboratory test prediction and forecasting tasks, and a few-shot experiment is used to argue for the effect of policy distillation.

Critical strengths

The paper correctly attributes cross-site performance degradation to shifts in the data-generating process rather than to downstream model capacity, and the resulting summarize-then-embed framework is a clearly motivated and reasonable design direction.

Critical weaknesses

The transfer evaluation is organized over the thirty task columns of the six transfer directions, and the paper does not verify that the compared quantities share clinical semantics across sites, nor does it provide any same-site (source=target) reference against which the transfer cost could be judged; the central transfer claim therefore cannot be assessed.

Moreover, the forecasting task includes the values to be predicted in its own input, making it a reconstruction exercise rather than forecasting, and the gains in the few-shot experiment cannot be separated from ordinary supervised target adaptation.

Questions for authors

What is the clinical justification for the column-position alignment across sites? Does the forecasting input contain the target window? Can you report a same-site (source=target) upper bound to quantify the true cost of transfer?

4 comments
AuthorsOpenReview ↗

Response to Review

[W2] "forecasting task ... adaptation"

We appreciate the reviewer pointing this out. To clarify, our actual pipeline does not perform reconstruction, and no target information is leaked into the forecasting input. In both our code and all reported experiments, the model takes the embedding of the first 48 hours and forecasts the subsequent, non-overlapping 24-hour window (24×F). The description resembling a reconstruction setup in Appendix D.1.3 was strictly a manuscript writing mistake. All experimental numbers remain unaffected, and we have corrected Appendix D.1.3 and Table 3 in the revised manuscript.

[Q1] What is the clinical justification for the column-position alignment across sites?

We thank the reviewer for this question. Comparing the targets by position is an evaluation choice rather than a clinical claim. Each site keeps its own variable list, as a deployed model would face at a new hospital, and the evaluation does not have to decide which site-specific variables count as the same; recent work likewise bridges differences between sites' vocabularies in the representation, reducing the need to harmonize them (Hegselmann et al., 2026; Kirchler et al., 2026; Guo et al., 2026). The same targets are used for every method, so the comparison between methods stays fair. We further repeated the evaluation with the targets matched by name on the variables shared by all three sites. The conclusions hold: in the setting of Table 1, the lab ranking is unchanged, and on forecasting the 24 hours after the input window, PORTER moves from second to first.

[Q2] Does the forecasting input contain the target window?

We thank the reviewer for catching this writing error in our manuscript. No, the forecasting input does not contain the target window. The input is the embedding of a stay's first 48 hours, and the output and target are all F charted variables for the next 24 hours, a 24×F24 \times F matrix that does not overlap with the input. The MSE is computed over the observed values. Our experiments, results and code all follow this definition, and the error was only in the writing of Appendix D.1.3, which described the target as the input window. We have corrected Appendix D.1.3 and Table 3.

[W1 Q3] "transfer evaluation ... cannot be assessed. Can you report a same-site (source=target) upper bound to quantify the true cost of transfer?"

We thank the reviewer for raising this. We checked each target against the data at every site. Drugs and labs are defined the same way at all three sites. The lab and forecasting targets are, however, aligned by column position, so we also scored them with targets matched by variable name across the three sites. The conclusions do not change, as shown in the new Appendix H.

For the same-site upper bound, we trained the same pipeline with each site as both source and target, using the same settings as the transfer runs, and tested it on the same test rows as every transfer into that site. On mortality and drug, where the two settings use the same targets, every method except the controls stays below this bound after transfer, averaged over the target sites. In the setting of Table 1, transfer costs PORTER about 4% of its same-site AUROC on mortality and 12% on drug, against 9% to 10% and 18% for Template and Markdown, the strongest methods within a site. Within a site and averaged over the three sites, PORTER is the best method on drug and forecasting in that setting, and on drug in twelve of the sixteen summarizer and head settings. The new Appendix I reports these in-distribution results.

Please let us know if you have any follow-up questions to our response. We will be glad to clarify before the discussion closes on Nov. 18.

Thank you for your time and effort!

Thank you for the response, which addresses most of my original concerns. However, the magnitude and consistency of the gains under name-matched evaluation remain unclear, as does whether the few-shot gains specifically reflect learned site policies. I have raised my rating from 1 to 2, although I still find the evidence insufficient to support acceptance.

Thank you so much for taking the time to follow up and for raising your score, we truly appreciate your reassessment and support! We are glad to clarify these two remaining points:

Re: name-matched evaluation

We’ve already put the name-matched transfer result in Appendix H in the revised manuscript. For the name-matched (feature aligned) evaluation results:

On the forecasting task: PORTER's MSE is lower than all other prior methods. Compared with the three text-based (Record2Vec, Template and Markdown) baselines, PORTER’s MSE is lower than them by a median of 1% to 4%. Compared with other non-text baselines, PORTER’s MSE is lower than them by 2% to 25% taken over all summarizers, heads and directions. This advantage holds under any choice of summarizing LLM which shows that PORTER consistently outform prior works under the name-matched experiment setup.

On the lab task: name-matching experiment setup does not affect transfer results for all the methods which also shows the consistent performance of PORTER compared with other methods.

The above clarification is shown in our appendix and shows that PORTER provides consistent gain for transfer learning tasks.

Re: few-show gains

On the few-shot gains, we tested different ablations of the policy vector and found that our distilled policy works better than all other ablations (random, false site, learned constant). This suggests that the distilled policy vector captures latent site policy affecting the outcome. We included these results in Figure 4 and Table 24 of the manuscript.

  • We tested a random vector, a learned constant vector (requested by Reviewer Qzt7), and a false policy vector for few-shot ablations.
  • We found that the distilled policy helps the most on the lab task, and using the other three ablations resulted in significantly decreases in performance.

Again, we sincerely thank the reviewer’s feedback and comment on how we have addressed your original concerns and raised your rating accordingly.

Reviewer_8MeFOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The authors propose a method that addresses the impact of the heterogeneous nature of electronic health records in the context of representation learning. They first conduct a site normalization of each recording (which in practice, means some institution, i.e. a hospital), then a representation is extracted using an LLM grounded on a data generating process of the data. The results are promising, but regrettably the text is dense and the analysis too broad: both from a fundamental Machine Learning, but also from an application standpoint. The paper is more suited to a more specialized venue or track, where a more thorough analysis can be presented.

Critical strengths

The method seems to have promising performance and addresses a very relevant and timely challenge within the everyday medical domain.

Critical weaknesses

Since authors’ are targeting ICLR, they leave a lot of the nuances of the application undressed and simply enumerate them. Even so, the main text also is lacking from technical rigour due to the space delegated to application specific technicalities. This makes it an odd fit for ICLR, and I believe a more targeted venue would allow the authors to produce a more digestible, or at least, direct themselves towards their actual target audience. The notation is too dense for the complexity of the results it presents.

Questions for authors

It is not clear to me why the related work (e.g., Markdown) does not satisfy sufficiency. What I_Y represents is not clear. I am a bit confused w.r.t. The purpose of the RANDOM and ADVERSARIAL analysis you perform. Could you kindly clarify?

2 comments
AuthorsOpenReview ↗

Response to Review

[W1] "I believe a more targeted venue ... actual target audience."

We respectfully disagree. Machine learning for healthcare is an established and active area of research. ICLR 2027 Call for Papers explicitly lists healthcare as a subject area, and representation learning as relevant topics. Papers that try to solve the problem that we address – generating representations from electronic health records – have been published at ICLR. The following papers have been published at ICLR 2024, 2025, 2026 respectively.

[1] Poulain, Raphael, and Rahmatollah Beheshti. "Graph transformers on EHRs: Better representation improves downstream performance." International Conference on Learning Representations (ICLR), 2024.

[2] Wornow, Michael, et al. "Context clues: evaluating long context models for clinical prediction tasks on EHR data." International Conference on Learning Representations (ICLR), 2025.

[3] Ji, Zongliang, et al. “Can We Generate Portable Representations for Clinical Time Series Data Using LLMs?” International Conference on Learning Representations (ICLR), 2026.

[Q1] Why doesn’t related work satisfy sufficiency?

Sufficiency in general is difficult to satisfy: it requires the representation to capture all information, in the treatments and the unobserved true patient state, that are related to the outcome. Consider Markdown [1], also a text-based representation. In Markdown, due to the context length of the embedding model, the authors chose to keep only the most recent value of each measurement. Therefore, important information about the trajectory of the patient’s condition is lost, and measurements at different timesteps are conflated.

Crucially, we do not claim that PORTER is sufficient. The intuition is that PORTER contains more information about the true patient state than Markdown or other related work.

[1] Hegselmann, Stefan, et al. "Large language models are powerful electronic health record encoders." npj Digital Medicine (2026).

[Q2] "What I_Y represents is not clear."

Thank you for pointing out the need for clarification. As we stated in Figure 2, (I_Y) represents the site-specific outcome mechanism, capturing how institutional policies and clinical practices influence patient outcomes. For example, differences in nurse staffing levels may affect inpatient mortality, while different hospitals may discharge patients at different stages of recovery, affecting length-of-stay prediction. We expanded the explanation of (I_Y) in the manuscript with these concrete examples to improve clarity at Line 122-126.

[Q3] "The purpose of the RANDOM and ADVERSARIAL analysis you perform"

RANDOM replaces each record's representation with an independent noise vector of the same size. The heads, training and metrics are unchanged, so it measures the floor of the evaluation itself. ATTACK (adversarial) keeps the whole PORTER pipeline but instructs the summarizer to ignore the record and always output the same sentence ("The patient is an adult in the intensive care unit receiving supportive care."). Every patient therefore receives essentially the same representation. This tests whether PORTER's gains come from the patient content of its summaries rather than from the language model or the embedding model. We have added these descriptions to Section 4.

Please let us know if you have any follow-up questions to our response. We will be glad to clarify before the discussion closes on Nov. 18.

Thank you for your time and effort!

Reviewer_Qzt7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses cross-site degradation in clinical prediction. The authors model site-specific measurement/treatment policies (IXI_X) and outcome policies (IYI_Y) via a causal graph, deriving sufficient conditions for Y⊥S∣ZY \perp S \mid Z. Their method, PORTER, constructs ZXZ_X by embedding an LLM summary of a 48-hour ICU window structured around a 13-section clinical schema, and ZYZ_Y by summarizing a site policy from 16 labeled target records. Across three ICU datasets, six transfer directions, and five tasks, schema-only PORTER achieves the best mean rank in zero-shot transfer (under the headline head), while PORTER + policy ranks first in the 16-shot setting. Additional experiments on EHRSHOT$\leftrightarrow$INSPECT show benefits from longer context.

Critical strengths
  1. Thorough empirical evaluation: The experimental grid is broad and fully reported in the appendix (Tables 7–23), covering 13–16 representations, four summarizers, four prediction heads, all site pairs, sensible sanity checks (ATTACK, RANDOM), and fold standard deviations.

  2. Clear problem framing and candid reporting: Separating patient state (ZXZ_X) from site-level outcome policy (ZYZ_Y) is an intuitive formulation. The authors also transparently report negative and countervailing results: direct text embedding is a strong, cheap baseline; SOFA wins on the drug task; and two ablated prompts outperform the full PORTER prompt across all arms (Table 28).

Critical weaknesses
  1. Flawed multi-label evaluation on lab, forecast, and drug: For lab and forecast, target vectors are truncated to the first min⁡(Ls,Lt)\min(L_s, L_t) columns and aligned by index, even though the authors note that matching indices "need not hold the same variable at both sites" (App. D.1.3). Evaluating predictions against mismatched targets likely explains why most methods get below-chance AUROC (<0.5<0.5) on HiRID$\to$MIMIC lab (Table 7). Furthermore, the ATTACK control—which feeds an identical representation for every patient—achieves 0.66–0.70 AUROC on drug. Because a constant representation must yield 0.5 macro-AUROC per label, the multi-label AUROC appears to be micro-averaged (pooled), conflating discrimination with label prevalence.

  2. Unexplained mechanism for ZYZ_Y in few-shot fine-tuning: Because ZYZ_Y is fixed for all patients at a given target site, it acts as a constant vector during single-site fine-tuning. For a linear head, concatenating a constant only shifts the logits' intercepts and cannot change within-site patient rankings on any single label. Yet adding ZYZ_Y boosts lab AUROC from 0.652 to 0.783 (Table 25)—suggesting either a pooled-metric artifact (shifting per-label thresholds relative to one another) or an interaction with the optimizer/regularization. The random and false-policy controls merely swap one constant vector for another, so they do not isolate the mechanism. Separately, Table 24's caption contradicts itself on whether cells are averaged over support sets or source folds.

  3. Triviality of the causal guarantee: Proposition 1 assumes common support for IYI_Y across sites, even though IYI_Y is defined as a site-specific policy. Worse, because ZYZ_Y is constant per site, it nearly identifies SS, making the conditional independence Y⊥S∣(ZX,ZY)Y \perp S \mid (Z_X, Z_Y) close to vacuous. Meanwhile, whether ZXZ_X itself is site-invariant is never tested empirically.

  4. Overstated headline claims and confounded baselines: PORTER's top zero-shot rank holds only for the linear and xLSTM-Mixer heads; with MLP and LSTM heads, SOFA ranks first (Tables 8–12, 15, 18, 21). Rank margins are narrow and reported without significance tests. In the long-context (LEHR) experiments, direct embedding is truncated at 32.7k tokens compared to 120k for PORTER, confounding representation quality with context length.

Questions for authors
  1. How is multi-label AUROC computed (micro/pooled vs. macro across labels)? If macro-averaged, how does a constant-input baseline (ATTACK) exceed 0.5 on drug?

  2. How do lab and forecast results change when restricted to the 47 semantically matched shared variables rather than positional truncation?

  3. With a linear head, how does a site-constant ZYZ_Y improve within-site AUROC? Does fine-tuning plain PORTER with a learned constant vector (or equivalent bias parameterization) match the "+ policy" gains?

  4. Can site identity SS be predicted from ZXZ_X? Reporting a site-classification probe across PORTER, Markdown, and Record2Vec would directly test whether schema summarization removes site signatures.

  5. How do the authors reconcile the common-support assumption on IYI_Y with its definition as a deterministic site property, and what guarantees remain if conditioned on ZXZ_X alone?

6 comments
AuthorsOpenReview ↗

Response to Review

[W2] "Unexplained mechanism ... Z_Y improve within-site AUROC?"

We clarify that PORTER + policy retrains the entire prediction head using the concatenated representation, rather than adding a constant bias to a fixed predictor. Although the policy embedding is constant within a target site, its inclusion can change the optimization dynamics and learned weights on patient representations, particularly under few-shot training and regularization. Therefore, the resulting predictions are not necessarily related by a constant shift, and within-site AUROC can change. Thanks to the reviewer for pointing out the Table 24’s caption contradiction. We clarify that: we report the mean across all combinations of five independently sampled 16-record target-site support sets and five source-trained folds, with one standard deviation across these combinations, using an MLP prediction head.

AuthorsOpenReview ↗

Response to Review 2

[W4] "Overstated headline claims ... context length."

Rank significance tests We thank the reviewer and agree that our headline claim was too strong. We have now tested whether the rank differences are statistically significant, and we revised the claim to match the results.

Pooled over all four prediction heads, PORTER is significantly better than every non-LLM baseline: the clinical sequence encoders (CLMBR-T, GARLIC, KEDGN), the time-series and tabular models (MIRA, TabICL), MEWS, and the random and adversarial controls. Against each of these methods, PORTER wins 74–107 of the 120 settings (6 transfer directions × 5 tasks × 4 heads), all with p < 0.001.

PORTER has the best mean rank across all heads (4.62) and ranks first in the largest share of bootstrap resamples (35%). However, it is not significantly different from SOFA, Record2Vec, or the two methods that embed the record text without summarization (Markdown and Canonicalization). Together these methods form a top tier. Which of them ranks first depends on the head, as the reviewer noted. We have therefore replaced "PORTER ranks first" with "PORTER is among the top-performing methods and significantly outperforms all non-LLM baselines."

Table A. Mean rank of each method among 13 methods (1 = best), by prediction head. Brackets: 95% bootstrap confidence interval. Bold: best mean rank in each column. Last column: fraction of bootstrap resamples in which the method ranks first.

MethodLinearMLPLSTMxLSTM-MixerAll headsP(rank 1), all heads
PORTER4.53 [3.5–5.6]4.90 [3.6–6.2]5.20 [4.0–6.6]3.87 [2.9–4.9]4.62 [3.6–5.7]0.35
SOFA4.83 [3.8–5.9]4.53 [3.6–5.7]4.40 [3.3–5.6]5.00 [3.8–6.3]4.69 [3.7–5.8]0.24
Canonicalization4.83 [3.6–6.2]4.83 [3.6–6.2]6.10 [4.6–7.4]3.37 [2.5–4.4]4.78 [3.7–5.9]0.27
Record2Vec4.83 [4.0–5.7]5.50 [4.5–6.5]5.67 [4.6–6.7]3.80 [3.2–4.8]4.95 [4.2–5.8]0.06
Markdown4.97 [4.0–6.3]5.20 [4.0–6.4]5.90 [4.6–7.2]4.42 [3.5–5.5]5.12 [4.1–6.2]0.08
CLMBR-T6.13 [5.2–7.3]6.10 [5.0–7.2]5.90 [4.9–7.0]7.53 [6.1–8.5]6.42 [5.5–7.3]0.00
MIRA6.70 [5.7–7.7]6.70 [5.7–7.5]7.10 [6.0–8.1]7.63 [6.6–8.7]7.03 [6.1–7.9]0.00
MEWS7.20 [6.2–8.2]7.20 [6.3–8.2]6.63 [5.7–7.6]7.55 [6.7–8.3]7.15 [6.4–8.0]0.00
TabICL8.50 [7.3–9.6]8.10 [6.8–9.3]7.60 [6.3–8.8]7.33 [6.4–8.4]7.88 [6.8–8.9]0.00
GARLIC8.00 [6.9–9.1]8.17 [7.0–9.3]7.83 [7.0–8.9]8.97 [8.0–9.7]8.24 [7.3–9.1]0.00
KEDGN9.67 [8.2–10.9]9.17 [7.6–10.5]8.53 [7.1–10.1]10.87 [10.1–11.6]9.56 [8.4–10.7]0.00
Attack9.93 [8.6–11.0]10.17 [9.0–11.2]10.10 [8.9–11.1]9.50 [8.3–10.9]9.93 [8.7–11.0]0.00
Random10.87 [9.8–11.6]10.43 [9.3–11.5]10.03 [8.7–11.2]11.17 [10.5–11.6]10.62 [9.7–11.4]0.00

Table B. PORTER head-to-head against each method: wins–losses across settings (30 per head, 120 for all heads), with Holm-corrected Wilcoxon signed-rank p-values. Bold: p < 0.05.

PORTER vs.LinearMLPLSTMxLSTM-MixerAll heads
SOFA18–12 (p=1.000)15–15 (p=1.000)16–14 (p=1.000)19–11 (p=0.336)68–52 (p=0.856)
Canonicalization15–15 (p=1.000)14–16 (p=1.000)19–11 (p=1.000)13–17 (p=0.840)61–59 (p=0.856)
Record2Vec15–15 (p=1.000)19–11 (p=0.333)18–12 (p=1.000)18–12 (p=1.000)70–50 (p=0.097)
Markdown15–15 (p=1.000)16–14 (p=1.000)16–14 (p=1.000)15–15 (p=1.000)62–58 (p=0.856)
CLMBR-T20–10 (p=0.125)17–13 (p=0.333)15–15 (p=0.886)22–8 (p=0.003)74–46 (p<0.001)
MIRA20–10 (p=0.236)19–11 (p=0.207)20–10 (p=0.255)23–7 (p=0.002)82–38 (p<0.001)
MEWS25–5 (p=0.008)24–6 (p=0.014)20–10 (p=0.248)27–3 (p<0.001)96–24 (p<0.001)
TabICL28–2 (p<0.001)25–5 (p<0.001)23–7 (p=0.020)26–4 (p<0.001)102–18 (p<0.001)
GARLIC21–9 (p=0.061)22–8 (p=0.061)20–10 (p=0.255)27–3 (p<0.001)90–30 (p<0.001)
KEDGN25–5 (p<0.001)22–8 (p=0.009)19–11 (p=0.089)29–1 (p<0.001)95–25 (p<0.001)
Attack24–6 (p<0.001)25–5 (p<0.001)24–6 (p<0.001)25–5 (p<0.001)98–22 (p<0.001)
Random28–2 (p<0.001)25–5 (p<0.001)24–6 (p<0.001)30–0 (p<0.001)107–13 (p<0.001)

We tested whether methods differ overall with the Friedman test, and compared PORTER with each baseline using paired Wilcoxon signed-rank tests with Holm correction. We added a critical-difference diagram to the appendix (Figure 7).

AuthorsOpenReview ↗

Response to Review 3

[W4] "Overstated headline claims ... context length." (continued)

Long-context experiments In the long-context experiments, we argue that representation quality is related to context length. PORTER leverages the longer context-window of the summarizer model, enabling it to incorporate longer-context information and produce representations that are more useful for certain tasks.

datasetcontext_lengthCLMBRDirect embeddingSummarize then embed
ehrshot302.8%3.2%0.3%
1809.0%11.5%1.2%
36513.7%17.2%2.4%
all27.0%33.8%8.8%
inspect302.4%2.9%0.1%
1807.6%10.0%1.0%
36512.1%17.4%1.9%
all25.4%37.0%8.8%

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to All Reviewers

We sincerely thank all reviewers for their time and constructive feedback, which have helped us significantly strengthen the paper. We are encouraged that the reviewers found our clinical motivation timely, our summarize-then-embed formulation well-grounded, and our empirical evaluation thorough and promising.

We carefully read and considered and responded to each weakness, questions and concerns raised by reviewers. These concerns are mostly in:

  • Clarification of theory (8MeF, Qzt7),
  • Clarification of venue fitness (8MeF),
  • Clarification of target-site policy embedding (Qzt7, 8MeF)
  • Clarification of the metrics, results and experiment setup (Qzt7, 8MeF, 4ZoD)
  • Asking more results (Qzt7, 4ZoD)

We agree with the reviewers on the following issues and have fixed them in the revised manuscript, with all changes marked in blue:

  • Scoring of lab and drug. Our submission used micro AUROC. We now score both tasks by macro AUROC, the AUROC of each label averaged over the labels, and re-scored all zero-shot tables with the models unchanged. The two controls now sit at chance, and PORTER keeps the best mean rank in Table1. (Qzt7)
  • Forecasting target. Our code and results already follow the correct forecasting definition, predicting the 24 hours after the input window. The appendix described the task as “reconstruction” because of a writing error, which we have corrected. (4ZoD)
  • We added a new critical differences diagram in the Appendix F to show the (lack of) statistical significance of our ranking results. (Qzt7)
  • We updated the statement of proposition 1 to restrict the scope. The new proposition is also no longer vacuous. (Qzt7)

We clarify the following points and support them with new results:

  • Alignment of targets across sites. In the main results each site keeps its own variable set. The new Appendix H matches the lab and forecasting targets by variable name, under every summarizer and head. The lab ranking barely changes, and on forecasting PORTER has the best mean rank in twelve of the sixteen settings. (Qzt7, 4ZoD)
  • Same-site upper bound. The new Appendix I trains and tests every method on the same site, under every summarizer and head. On mortality and drug, every method except the controls stays below this bound after transfer, and within a site, averaged over the three sites, - PORTER is the best method on drug in twelve of the sixteen settings. ( 4ZoD)
  • Role of the controls. Section 4 now explains what RANDOM and ATTACK test. (8MeF)
  • One of the reasons why PORTER generates better representations on LEHR is precisely because it can incorporate more context while other methods fail to do so. (Qzt7)
  • The site policy vector, while fixed for a site, affects the learned predictor through optimization dynamics that change the AUROC. Adding a learned constant vector does not match the “+ policy” gain. (Qzt7)

We respectfully disagree with some claims:

  • “A more targeted venue would allow the authors to produce a more digestible, or at least, direct themselves towards their actual target audience.” Our paper has good venue-fit at ICLR, as seen by previously published papers at ICLR on similar subjects and areas. (8MeF)

We hope these clarifications and revisions address the reviewers' concerns, and we welcome any further questions or feedback. If our responses have resolved your main concerns, we kindly ask that you consider updating your evaluation accordingly.

AuthorsOpenReview ↗

manuscript change summary

We’ve applied all the revisions to our manuscript. Here are the changes we made:

  • L227 - L234, Table 1: we re-scored lab and drug by macro AUROC and recomputed the marks, mean ranks and top-k counts, to address W1/Q1 raised by Reviewer Qzt7.
  • L253 - L254, Section 4: we now score lab and drug by macro AUROC, to address W1/Q1 raised by Reviewer Qzt7.
  • L266 - L268 and L301 - L303, Section 4: we describe what the RANDOM and ATTACK controls test, to address Q3 raised by Reviewer 8MeF.
  • L340 - L341, Section 5.1: we corrected the sentence on SOFA's drug results to match the macro AUROC, to address W1/Q1 raised by Reviewer Qzt7.
  • L1195 - L1196 and L1215 - L1217, Appendix D.1.3 and Table 3: we corrected the description of forecasting, which did not match Section 4, and updated the forecasting results in Table 1 and Appendix F to this target, to address W2/Q2 raised by Reviewer 4ZoD.
  • L1454 and Tables 7 to 23, Appendix F.1: we re-scored lab and drug by macro AUROC in Tables 7 to 23, recomputed the marks and rank summaries, and updated the text accordingly, to address W1/Q1 raised by Reviewer Qzt7.
  • L1831 - L1835, L1925 - L1926, Tables 33 to 48, new Appendix H: we evaluate lab and forecasting with targets matched by variable name over the variables recorded at all three sites, under every summarizer and head, to address Q2 raised by Reviewer Qzt7 and W1/Q1 raised by Reviewer 4ZoD.
  • L1928 - L1936, Tables 49 to 64, new Appendix I: we report the same-site, in-distribution results under every summarizer and head, to address W1/Q3 raised by Reviewer 4ZoD.
  • new Figure 7 in Appendix: we added a new critical difference diagram showing the ranks of each method to address W4 raised by Reviewer Qzt7
  • new Figure 9, 10 in Appendix: we added plots showing token length of the input text before and after summarization, as well as comparison of context window exceedance across three different methods (CLMBR-T, direct embedding, summarize then embed) to address W4 raised by Reviewer Qzt7
  • new Figure 8 In Appendix: we added plots showing the UMAP of the embeddings generated by each method. It is clear that PORTER, MIRA, and Record2Vec generate embeddings that contain mostly site-agnostic information to address Q4 raised by Reviewer Qzt7
  • updated Proposition 1 in Appendix B: we added an additional assumption such that the conditional independence is no longer vacuous and the site is no longer identifiable from I_Y. The proposition no longer applies to the fine tuning part, but remains informative of the zero-shot method to address W3/Q5 raised by Reviewer Qzt7
  • L133 - L134 updated to match the new statement of the proposition 1 to address W3/Q5 raised by Reviewer Qzt7
  • L122 - L126 updated caption to describe what I_Y is in more detail to address Q2 raised by Reviewer 8MeF
  • L2442 - L2446 Table 24 updated caption to clarify the mean is calculated over the sets and folds to address W2 raised by Reviewer Qzt7

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.