- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
DISCERN aims to separate disease evidence from shortcuts caused by differences in cohort enrollment. Under a shared Gaussian latent-coordinate model, it identifies a selection statistic from cross-cohort changes and removes its contribution to the state likelihood ratio. The proposed model combines this correction with a multimodal encoder and an outcome predictor. Experiments show strong state recovery under reversed enrollment when the true latent coordinates are supplied, and improved cross-database ICU mortality prediction over a matched ERM baseline.
Critical strengths
- The paper defines a precise target—conditional evidence about an unobserved disease state—and states the assumptions under which it is identifiable.
- The synthetic reversal experiment and random-direction control give convincing evidence that the proposed correction works when the required latent coordinates are available.
- The ICU study includes matched-backbone ablations and reports gains in AUROC, AUPRC, and NLL in both transfer directions.
Critical weaknesses
- The theoretical and practical components are not connected end to end. The identification result assumes aligned Gaussian latent coordinates, shared state-specific covariances, fixed acquisition, and target shifts contained in the source-identified subspace. These conditions are not established by the proposed encoder. Indeed, when the true latent coordinates are replaced by learned multimodal representations, state recovery under reversed enrollment falls below chance and DISCERN does not outperform the matched lattice ERM.
- The clinical experiment does not validate the interpretation of the learned quantities as disease evidence or enrollment effects. eICU and MIMIC-IV differ in both patient selection and measurement practices, while the latent states have no clinical anchors and are influenced by outcome supervision through joint training. The learned state components and selection direction may therefore capture database, measurement, or label-related variation rather than the intended disease and enrollment mechanisms.
- The clinical evidence remains limited and exploratory. The evaluation uses one prediction task, three seeds, and targets that were inspected during earlier development. Several historical baselines use different backbones and training protocols, while the matched rich ERM is weaker than some simpler ERM variants. In addition, the deployed rank-one correction is not justified empirically, although the first direction explains only 37.1–57.6% of the fitted shift energy.
Questions for authors
- Can the authors provide an end-to-end synthetic experiment in which raw multimodal observations are used, acquisition is held fixed, and only enrollment changes? This would clarify whether the failure of learned-view state recovery is caused by the encoder coordinates or by the additional measurement shift.
- Is there any clinical evidence that the learned mixture components correspond to meaningful disease states and that the estimated selection direction captures enrollment rather than measurement or database identity? Associations with independent clinical variables or clinically defined subgroups would help.
- How do state recovery and ICU prediction change when the full estimated selection subspace, or several ranks, are used instead of a single direction?
- Can the authors evaluate the final method on a previously unseen clinical target and compare it with baselines using the same backbone, inputs, training budget, and model-selection protocol?
6 comments
2. Response to Reviewer xCxA
W3: clinical evidence and matched comparisons
Replication. Table 2 retains the original three-seed results and adds three random directions per training seed across five seeds. Each control trains separately from the shared warm-up checkpoint; Appendix D.6 gives the direction-generation and training protocol. The controlled studies use 10 to 20 enrollment/target replicates, with paired comparisons in Appendix G.2.
Comparison scope. The rich ERM and original mechanism ablations share the backbone, inputs, training budget and source-only checkpoint selection. Historical baselines use different backbones/protocols and remain contextual comparisons. In particular, matched rich ERM is weaker than some simpler historical models; DISCERN is slightly below SWAD-style into MIMIC-IV (0.818 vs 0.820) and above the historical baselines into eICU.
Controlled-study methods share inputs and source-only selection, but GroupDRO/V-REx use MLPs while the recorded-state estimator and ERM use logistic models. Their architectures and budgets are reported in Appendix G.1; the random-direction control holds the estimator fixed to compare the conditioning choice directly.
On eICU age enrollment (Table 11), AUROC is:
| Method | Reversed | Mean of four laws |
|---|---|---|
| Oracle, target labels | 0.879 | 0.881 |
| ERM | 0.588 | 0.794 |
| IPW | 0.726 | 0.804 |
| GroupDRO | 0.629 | 0.800 |
| V-REx | 0.629 | 0.797 |
| IRMv1 | 0.690 | 0.786 |
| Conditional label shift | 0.780 | 0.794 |
| Random direction | 0.576 | 0.784 |
| DISCERN | 0.772 | 0.795 |
Bold marks the best non-oracle value. DISCERN improves robustness under reversal. The unlabeled-target gate adapts to the target selection law, raising its four-law mean to 0.870 (Table 12).
Development history. The original cross-database comparisons remain exploratory because targets were inspected earlier. Controlled designs were fixed before their results; the first natural-parameter-statistic run failed, and the subsequent differential-statistic studies used new replicates/cohorts. Appendix G also reports the ADNI conversion and MIMIC-IV→eICU gate results, where adaptation does not consistently improve on ERM.
Q3: several ranks versus rank one
The 37.1 to 57.6% energy share measures fitted shift variation, while useful rank also depends on evidence retained after conditioning. The original clinical implementation uses a rank-one approximation. Conditioning can remove state evidence: on supplied synthetic coordinates, increasing the rank from one to two reduces additive-regime AUROC from 0.879 to 0.705 (Appendix C.7). At the full-coordinate limit, conditioning gives constant state evidence.
Table 13 reports the recorded-state eICU age-family comparison:
| Statistic | First-direction energy | Age loading | AUROC, ranks 1 / 2 / 3 |
|---|---|---|---|
| Original natural-parameter | 0.460 | 0.360 | 0.624 / 0.768 / 0.771 |
| Differential | 0.850 | 0.990 | 0.772 / 0.773 / 0.775 |
| Random, mean of three draws | 0.100 | 0.270 | 0.587 / 0.618 / 0.636 |
The original statistic improves when more directions are included, while the differential statistic concentrates the relevant shift in one direction. Under common covariance, Theorem 2 preserves pairwise evidence ranking up to constants, with prior/gate adjustment for calibration. If sources enroll on age but the target on sex, correcting the source direction gives 0.765 versus ERM 0.757; using the whole background gives 0.788, illustrating the importance of the source-subspace assumption.
Q4: a previously unseen target
We added eICU leave-one-region-out evaluation: each US census region is held out in turn, with five replicates per region (20 total), under a design fixed before results. Reversal AUROC is 0.779 for DISCERN, 0.586 for random conditioning and 0.593 for ERM; DISCERN exceeds both in all 20 replicates. With an unlabeled-target gate, its four-law mean is 0.875. MIMIC-IV and ADNI additionally test the differential statistic beyond the eICU cohort on which it was developed.
These additions extend the evaluation to held-out geographic targets and additional clinical state definitions. The region study uses new partitions of eICU; an untouched external database with all baselines matched in backbone and budget would provide a further independent evaluation.
Thank you for the additional experiments and clarifications. They address several of my questions, though two concerns remain.
-
The successful raw-input results use recorded states and observed backgrounds. The original lattice still fails with fixed acquisition, and the full joint model shows little benefit from conditioning under imposed enrollment. It therefore remains unclear whether the original model learns the stable evidence required by the theory.
-
Could you clarify which results support the original model and which support the recorded-state estimator and subsequent extensions? Theorem 2 assumes common covariance across states, so it does not directly cover the state-dependent covariances used in the original interaction setting.
1. Response to Reviewer xCxA
We thank the reviewer. S is a latent disease-related processes underlying physiology and clinical manifestations. It is not defined as a pre-existing diagnosis class with a corresponding ground-truth annotation. Clinical observations and recorded outcomes provide partial information from which S is inferred. Thus outcome supervision contributes to learning without directly labeling S. The identification target is conditional evidence about this latent state under selection.
W1 / Q1: end-to-end recovery with fixed acquisition
The new study holds acquisition fixed and changes only enrollment, using raw multimodal observations from the §5 generator (Appendix C.7, Table 6). State AUROC under far reversal, over five generated studies per mechanism, is:
Bold identifies the raw-observation estimator. Synthetic S is specified by the generator, enabling evaluation across these state/coordinate information settings.
The lattice fitted without state annotations remains below chance with acquisition fixed, showing that measurement change alone does not explain its behavior. With recorded source states and observed background, the additional estimator averages joint log odds relative to reference state 0 minus the corresponding log odds conditioned on the estimated background statistic. It gives 0.831 to 0.877 across four target laws with fixed acquisition and 0.815 to 0.872 with changing acquisition; its direction has cosine 1.000 with the generator direction. This provides a direct raw-input test of the correction with a recorded operational state.
We also tested the full jointly trained clinical model under imposed enrollment within eICU and across eICU→MIMIC-IV (ten replicates each; Appendix G.5). Its state branch receives weights below 0.010, and conditional, uncorrected and random variants stay within 0.002 AUROC: DISCERN reaches 0.605 / 0.603 versus matched ERM 0.614 / 0.597. State-evidence AUROC is at most 0.680 in the more favorable orientation. In this imposed-mortality-enrollment setting, the learned composite states give little correction benefit.
With source states recorded, applying the additional estimator to the learned representation gives 0.695 under reversal versus random conditioning 0.552 and the network head 0.556. Thus the same representation supports a useful correction when the operational state and background information are supplied.
W2 / Q2: clinical interpretation of states and direction
Clinical meaning of the composite states. In eICU→MIMIC-IV, one component consistently has mortality of 22 to 34% and mean APACHE IV of 72 to 80, versus 6 to 8% and 56 to 61 in other components (Appendix D.6). These associations support a clinically meaningful severity organization. The normalized mutual information with diagnosis is at most 0.016: the composite states span diagnosis categories, as the model permits. Demographics, physiology and, in the reverse direction, vital-sign measurement frequency influence assignments. Outcome supervision is intentional, because Y is an observed clinical view informing the composite representation; it is not a direct annotation of latent S.
Direction attribution. Table 2 now includes random conditioning. DISCERN exceeds it in AUROC (0.818 vs 0.793; 0.812 vs 0.786), AUPRC and NLL. The variants train separately from the same warm-up checkpoint with matched training settings and source-only selection. Random directions remain fixed, whereas DISCERN updates its estimated direction each epoch (Appendix D.6). This supports the direction choice in the jointly trained model.
Known enrollment and recorded states. New §5.4 instead imposes enrollment tied to mortality or CSF amyloid positivity. The differential statistic loads 0.980 to 0.990 on the enrolled factor, versus 0.360 to 0.500 for the original natural-parameter direction (Appendix G.3), and outperforms the mean random direction in all 80 replicates. These operational instances test whether the correction follows enrollment; mortality and amyloid labels are not ground-truth annotations of the original latent composite.
The guarantee addresses selection under the stated coordinate, covariance, alignment and source-subspace assumptions; Proposition 3 accounts separately for measurement and endpoint mismatch. Appendix G.5 tests shortcut sensitivity by including or excluding background from the encoder and applying the correction at the representation level.