acceptodds
Under review as a conference paper at ICLR 2027

M2M-FSurv: Multimodal Deep Functional Modeling for Competing Risks Survival Analysis

Abstract

Modern clinical datasets contain rich multimodal information, including clinical narratives, longitudinal physiological measurements, and baseline patient characteristics. At the same time, patients may experience multiple competing outcomes, such as discharge or in-hospital death. Integrating these heterogeneous modalities to model competing events over time remains a fundamental challenge in survival analysis. Existing classical and deep learning–based survival models remain limited in jointly modeling multimodal inputs, competing risks, and interpretable continuous-time event predictions. To address this limitation, we propose M2M-FSurv, a deep learning framework for multimodal competing-risks survival analysis that incorporates principles from functional data analysis to estimate smooth, continuous-time cause-specific density incidence functions. M2M-FSurv employs an attention-based shared encoder that integrates inputs and adjusts modality contributions across individuals, along with cause-specific prediction heads that estimate smooth, cause-specific event-time densities approximated using B-spline expansions. We evaluate the framework on the MIMIC-III and METABRIC datasets. Experimental results demonstrate that M2M-FSurv achieves improved predictive performance while providing interpretable insights into modality contributions and cause-specific event dynamics.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Multimodal learning, competing risks, survival analysis, functional data analysis, B-splines, electronic health records

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_WD4kOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes M2M-FSurv, a multimodal framework for continuous-time competing-risks survival prediction. It integrates clinical notes, longitudinal physiological measurements, and structured covariates through modality-specific encoders and attention-based fusion, followed by B-spline prediction heads that estimate cause-specific event-time distributions. The authors provide a theoretical consistency analysis and evaluate the method on MIMIC-III and METABRIC.

Critical strengths
  1. The paper addresses a clinically relevant problem by combining heterogeneous patient information with continuous-time competing-risks prediction.
  2. The B-spline formulation provides smooth, continuous-time event predictions while ensuring valid cumulative incidence functions by construction.
Critical weaknesses
  1. The multimodal architecture largely follows established encoder–fusion paradigms. The B-spline prediction head appears to be the more distinctive contribution, yet its motivation remains underdeveloped. In particular, the paper does not clearly establish why B-spline representations are preferable to existing parametric or discrete-time survival models, or which properties are particularly valuable for the target clinical setting.
  2. The consistency result applies only to a restricted feedforward B-spline estimator, rather than the implemented M2M-FSurv architecture. Moreover, it does not establish any specific statistical advantages of B-spline representations over alternative continuous-time survival models, leaving its relevance to the proposed method unclear.
  3. The evaluation omits SAMVAE, a closely related multimodal continuous-time competing-risks method explicitly discussed in the paper. Furthermore, the experiments do not isolate the contribution of the B-spline prediction head from the multimodal architecture.
  4. METABRIC involves scalar-only, single-event prediction and therefore cannot validate the proposed multimodal fusion or competing-risks modeling. While it offers additional evidence of general survival prediction performance, it does not isolate the benefits of B-splines. Moreover, the justification that classical survival methods cannot handle multimodal inputs is questionable: Methods such as Cox regression and DeepSurv can be combined with multimodal representations and could also serve as baselines on MIMIC-III.
Questions for authors
  1. What specific advantages of B-spline representations motivate their use over existing parametric and discrete-time survival models?
  2. What insights does the consistency analysis provide about the implemented architecture?
  3. Why is SAMVAE excluded from the comparisons? Could the authors provide controlled comparisons using the same multimodal encoder with alternative survival prediction heads (e.g., parametric or discrete-time) to isolate the contribution of B-splines?
7 comments
AuthorsOpenReview ↗

Response to Weakness 1 (part 1)

Thank you for highlighting the need to better motivate our B-spline formulation and clarify its methodological contribution. We agree that the modality-specific encoders and Transformer-based fusion build upon established architectures. Our primary contribution lies in the construction of continuous-time cause-specific event distributions, rather than in introducing new multimodal encoding components.

Motivation for B-spline event-time modeling. In our ICU competing-risks setting, we aim to estimate the timing of two mutually exclusive outcomes—alive ICU discharge and in-ICU death—following the 24-hour landmark. The corresponding event-time distributions may exhibit distinct temporal patterns across patients and competing outcomes, motivating a flexible representation that can accommodate locally varying density shapes.

Existing survival formulations offer different modeling trade-offs. Discrete-time approaches represent event probabilities over predefined time intervals, with temporal resolution determined by the chosen discretization. Parametric continuous-time approaches provide event-time predictions without discretization but impose distributional structure through their selected parametric families. Although flexible parametric mixtures can represent complex distributions, their density shapes are governed by the chosen distributional components.

In contrast, B-splines provide a locally supported basis representation of smooth event-time densities, allowing different regions of the prediction horizon to vary without imposing a particular global parametric distributional shape.

Relevance to the ICU setting. The local support of B-splines is particularly useful for representing event-time variation across different post-landmark periods. Moreover, the spline degree, basis dimension, and knot placement provide explicit control over representation flexibility and temporal resolution. In our implementation, we use non-uniform knots, allocating greater resolution to the earlier post-landmark period and a coarser representation to the sparsely observed tail. This allows the event-time representation to accommodate temporal variation while retaining a structured, smooth functional form.

Our formulation also directly represents continuous-time cause-specific event densities, from which cumulative incidence functions can be evaluated at arbitrary times through integration. Furthermore, patient-specific nonnegative B-spline coefficients and joint probability normalization ensure nonnegative cause-specific densities, nondecreasing cumulative incidence functions, and valid competing-risks probabilities by construction.

Empirical validation of the modeling choice. To assess whether these representational properties translate into predictive benefits, we conducted additional controlled head-replacement experiments comparing B-spline, discrete-time probability, and continuous-time log-normal mixture prediction heads. All configurations retained the same multimodal encoders, Transformer fusion architecture, input representations, data partitions, and prediction horizon.

With ranking loss, the B-spline head achieved the highest CtdC^{td}-index for both alive ICU discharge (0.788) and in-ICU death (0.866), compared with the discrete-time head (0.729 and 0.775) and the log-normal mixture head (0.751 and 0.782). The B-spline head also maintained the highest observed concordance without ranking loss, indicating that its discrimination advantage is not solely attributable to the ranking objective.

These findings provide empirical support for the B-spline modeling choice in terms of cause-specific discrimination, beyond the use of continuous-time prediction alone. We acknowledge that alternative heads achieve lower mortality IBS in some configurations; therefore, our results support an advantage in cause-specific discrimination under the evaluated setting rather than universal superiority across all prediction metrics.

Prediction accuracy at clinically relevant horizons. To compare short-horizon mortality probability-prediction accuracy, we evaluated cause-specific Brier scores (BS) at 24 and 72 hours after the landmark, using the same held-out test data and evaluation protocol across all three prediction heads.

Table 1: Comparison of mortality Brier scores (BS) at clinically relevant prediction horizons after the 24-hour landmark.

Prediction HeadBS (24h)BS (72h)Overall IBS
B-spline0.0230.0490.109
Discrete-time0.0290.0590.064
Log-normal mixture0.0250.0520.077
AuthorsOpenReview ↗

Response to Weakness 1 (part 2)

Although the B-spline head did not achieve the lowest mortality IBS over the entire prediction horizon, it achieved lower Brier scores at both 24 and 72 hours compared with the discrete-time and log-normal mixture heads (Table 1 ).

These findings suggest that the B-spline representation provides benefits not only in cause-specific discrimination but also in short-horizon prediction accuracy. The difference between the horizon-specific Brier scores and the overall IBS highlights the importance of evaluating prediction quality at clinically relevant time points rather than relying exclusively on an integrated metric over the full prediction horizon.

Overall findings. Our motivation for using B-splines is not that alternative survival formulations are incapable of modeling complex event-time distributions, but that B-splines offer a locally supported, smoothly varying representation with explicit control over temporal resolution through knot placement. Coupled with joint competing-risks probability normalization, this provides a structured formulation for continuous-time cause-specific event-density prediction. The additional controlled experiments demonstrate that this design choice makes a measurable contribution to cause-specific discrimination beyond the shared multimodal architecture.

We will strengthen this motivation and clarify its relevance to the ICU competing-risks setting in the revised manuscript.

AuthorsOpenReview ↗

Response to Weakness 2

We thank the reviewer for highlighting the need to clarify the relationship between our theoretical analysis and the implemented M2M-FSurv architecture. To address this concern, we have revised the theoretical analysis to provide two complementary results covering the Transformer-based prediction architecture, the normalized B-spline heads, and the role of the ranking objective.

Fixed-model theoretical guarantee. Revised Theorem 4.1 establishes a fixed-model population-risk guarantee for a finite Transformer architecture with a fixed B-spline basis and a fixed positive ranking weight. Under the stated bounded-input, compact-parameter, masking, and sampling conditions, the analysis derives uniform complexity bounds covering masked attention, residual connections, LayerNorm, pooling, and both softmax prediction heads.

For i.i.d. patient clusters with uniformly bounded numbers of stays, the theorem establishes convergence to the optimal value of the combined population objective induced by the specified sampler and numerical loss reductions within the fixed model class. This requires the uniform deviations and the global minimization gap for the conditional expected training criterion to vanish. Importantly, this result accommodates a fixed positive ranking weight and does not require the true event-time distribution to belong to the fixed model class.

Approximation and complexity of normalized Transformer models. We further establish a constructive approximation result for an explicitly specified normalized Transformer family. This result accounts for the Transformer operations and the event-probability and B-spline mixture softmax heads. Under the stated smoothness, positivity, and architectural conditions, we show that the approximation error for cause-specific event-time densities can vanish while the model complexity remains controlled as the network dimensions and spline basis size increase.

These results extend the original analysis beyond the restricted feedforward coefficient class and provide a theoretical foundation for the normalized neural B-spline formulation.

Scope and limitations. We distinguish fixed-model population-risk optimality from consistency for the true cause-specific event distributions. A nonvanishing ranking objective may alter the population optimum; therefore, truth-consistency additionally requires the stated censoring conditions, vanishing optimization error, and a vanishing ranking perturbation. The growing-family approximation result does not establish universal truth consistency for the fixed architecture and spline dimension used in our experiments. Application of the fixed-model guarantee to the reported runs remains conditional on matching the analyzed computational graph, preprocessing, sampler, and numerical loss reductions. Moreover, the results do not guarantee global convergence of AdamW.

Overall findings. The revised analysis provides theoretical support for the Transformer-based normalized B-spline formulation under explicit conditions, while distinguishing its fixed-model optimization properties from asymptotic recovery of the true event-time distributions. Together with the controlled prediction-head experiments, these results strengthen the theoretical and empirical justification for our proposed modeling framework without claiming unconditional consistency or theoretical superiority over alternative survival models.

Reviewer_EVXhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes M2M-FSurv, a multimodal deep learning framework for competing-risks survival analysis. M2M-FSurv integrates clinical text, physiological time series, and static patient features using a shared Transformer to model cross-modal interactions, and employs B-splines to model continuous-time event distributions for different outcomes, enabling predictions of event times and cumulative incidence probabilities. On MIMIC-III, M2M-FSurv outperforms the compared methods in both discrimination and integrated Brier score (IBS) for discharge and mortality prediction. Additional experiments on METABRIC evaluate the model in a single-modality, single-event setting.

Critical strengths
  • Conventional discrete-time methods typically partition follow-up time into predefined intervals. In contrast, M2M-FSurv uses B-splines to directly model continuous-time event densities and derives cumulative incidence functions (CIFs) through integration, enabling predictions at arbitrary time points within a specified time horizon. Furthermore, it employs a shared Transformer architecture to integrate clinical text, physiological time series, and static patient features, thereby modeling cross-modal interactions for competing-risks survival analysis.

  • On MIMIC-III, the proposed method outperforms the compared approaches in discrimination and integrated Brier score (IBS) for discharge and mortality prediction, supporting the effectiveness of the proposed method.

Critical weaknesses
  • Insufficient experimental validation: The related work acknowledges that SAMVAE (arXiv preprint, 2025) has already addressed continuous-time multimodal competing-risks prediction, yet no meaningful empirical comparison with this method is provided. For multimodal competing-risks prediction, the evaluation is limited to comparisons with SurvTRACE (ACM BCB 2022) and FCRN (arXiv preprint, 2025) on MIMIC-III. METABRIC, in contrast, involves scalar feature inputs and single-event survival analysis, and therefore cannot be considered external validation for multimodal competing-risks prediction.

  • Limited novelty: M2M-FSurv is primarily a combination of mathematical modeling tailored to a specific survival analysis task and established deep learning components. Clinical text encoding, physiological time-series encoding, static feature encoding, and Transformer self-attention are all well-established techniques. Although their integration for ICU patient risk prediction has practical value, the methodological novelty is limited. Moreover, the B-spline prediction head, a central component of the proposed method, lacks dedicated ablation studies to demonstrate its specific contribution.

Questions for authors
  • Could the authors evaluate M2M-FSurv on additional multimodal competing-risks datasets? If such datasets are unavailable, could they compare the proposed method with more recent approaches, particularly the paper acknowledges SAMVAE, to better establish its advantages over existing methods?

  • The authors acknowledge in the Limitations section that they do not isolate the effects of the spline head. However, B-spline-based event-density modeling is a central component of the proposed method. Could the authors explain why its contribution has not been independently evaluated and provide ablation studies against alternative prediction heads to demonstrate its effectiveness?

4 comments
AuthorsOpenReview ↗

Response to Weakness 1

Thank you for highlighting the need for stronger empirical validation. We agree that the original submission lacked a direct comparison with SAMVAE, a closely related method for continuous-time multimodal competing-risks prediction. To address this concern, we conducted an additional comparison with SAMVAE on the MIMIC-III dataset, using matched input modalities, patient-level data partitions, competing-event definitions, prediction horizons, and evaluation metrics.

Additional comparison with SAMVAE. We evaluated both methods using the same underlying input modalities, patient-level data partitions, competing-event definitions, prediction horizons, and evaluation metrics. For the SAMVAE implementation, we retained its variational reconstruction objective, KL-divergence regularization, and Weibull-based survival decoder. SAMVAE received the same clinical-text embeddings as M2M-FSurv, while physiological measurements were flattened into a 125-dimensional vector. Scalar covariates were processed using the same preprocessing methods as M2M-FSurv, and the resulting feature vector was provided to the SAMVAE encoder. SAMVAE followed the same training-set sampling strategy as M2M-FSurv, with minority death-event oversampling applied only to the training folds.

M2M-FSurv used the sampling-probability correction and validation-fitted calibration described in our response to Reviewer 1. SAMVAE used the same training-only minority-event oversampling strategy, with an expected 1:2 ratio of in-ICU deaths to alive ICU discharges. For SAMVAE, sampling-probability correction was applied to the survival likelihood using pk/qkp_k/q_k, and post-training calibration was performed using temperature scaling, with calibration parameters fitted on the validation set while keeping the trained model parameters fixed.

Hyperparameters were selected using validation data, and checkpoints were selected by maximizing the validation time-dependent C-index minus IBS, consistent with the M2M-FSurv evaluation protocol. Early stopping was based on validation loss.

Under this setting, M2M-FSurv achieved higher CtdC^{td}-index for discharge (0.788 vs.\ 0.709) and death (0.866 vs.\ 0.703), and lower discharge IBS (0.020 vs.\ 0.121). SAMVAE had a numerically lower mortality IBS (0.104 vs.\ 0.109) for M2M-FSurv. These results provide additional empirical evidence for the effectiveness of M2M-FSurv, particularly in cause-specific discrimination, and strengthen the comparison with existing continuous-time multimodal competing-risks methods.

Clarification of the METABRIC evaluation. As explicitly stated in Section 5.3.2 of the original manuscript, METABRIC was included as a complementary scalar-only, single-event survival experiment to enable comparisons with baseline methods in a standard tabular-input setting. It was not intended to serve as external validation of multimodal competing-risks prediction. We nevertheless acknowledge that MIMIC-III remains the only multimodal competing-risks dataset evaluated in this study.

Overall findings. The additional SAMVAE comparison directly addresses the missing closely related baseline and provides stronger empirical support for the proposed framework under the evaluated ICU setting. The METABRIC experiment continues to serve its originally stated purpose of examining the framework in a scalar-only, single-event survival setting. We will update the IBS conclusions in Section 5.3.1 to reflect the expanded comparison set and the outcome-specific results. We will incorporate the SAMVAE comparison and its implementation details into the revised manuscript, while acknowledging that evaluation on additional independent multimodal competing-risks datasets remains an important direction for future work.

AuthorsOpenReview ↗

Response to Weakness 2

We thank the reviewer for highlighting the need to better distinguish our methodological contribution from established multimodal encoding and attention mechanisms.

Clarification of methodological novelty. We agree that clinical-text encoding, physiological time-series encoding, and Transformer-based fusion are established techniques. Our methodological contribution concerns the event-time representation: patient-dependent probability masses over fixed, locally supported B-spline density components, jointly normalized across competing events and survival beyond the analysis horizon.

This representation combines fixed local density components with smooth continuous-time subdensities. The spline coefficients determine the event-time densities used in the training likelihood, and the corresponding CIFs are obtained by integrating the same basis. Thus, smoothness is part of the fitted model rather than a post-processing step. Temporal resolution is controlled by the knots and basis dimension. Degree-zero splines recover a uniform-within-bin continuous-time extension of a bin-probability representation, with matching cumulative probabilities at bin boundaries; cubic splines replace these uniform components with overlapping smooth components.

The distinction from the existing baselines concerns how event time is represented. SurvTRACE uses an interval-based hazard parameterization, while the reported SAMVAE implementation uses a conditional Weibull event-time decoder. M2M-FSurv instead directly represents cause-specific event-time subdensities through patient-dependent masses over fixed, locally supported spline density components, and obtains the corresponding CIFs by integrating the same basis. This provides a specific smooth density-based representation whose temporal resolution is controlled by the spline basis. Our contribution concerns this output formulation and its predictive value assessed through controlled head-replacement experiments, rather than novelty in the encoders, attention, or softmax normalization individually.

New controlled prediction-head ablation. To directly address the reviewer's concern regarding the contribution of the B-spline head, we compared three event-time prediction heads while retaining the same multimodal encoder and Transformer fusion architecture, input representations, patient-level data partitions, and prediction horizon. The alternatives were a discrete-time head with 32 equally spaced bins and a continuous-time log-normal mixture head with 16 components per cause. For each configuration, the encoders, fusion module, and prediction head were jointly retrained. Each head used its corresponding likelihood and was evaluated both with and without ranking loss, with configuration-specific hyperparameters selected using validation data.

With ranking loss, the B-spline head achieved discharge and death CtdC^{\mathrm{td}}-index of 0.788 and 0.866, compared with 0.729 and 0.775 for the discrete-time head, and 0.751 and 0.782 for the continuous-time log-normal mixture head. Without ranking loss, the corresponding values were 0.782 and 0.808 for B-splines, 0.709 and 0.750 for the discrete-time head, and 0.706 and 0.780 for the log-normal mixture head. The B-spline head therefore achieved higher observed concordance for both outcomes in both ranking settings, including against another continuous-time representation.

These comparisons support the predictive value of the B-spline head configuration within the evaluated architecture and training protocol. They do not separately identify the contributions of smoothness and local support. We also acknowledge that alternative prediction heads achieve lower mortality IBS under some configurations. We have therefore clarified that the observed advantage concerns discrimination, rather than claiming superiority across all evaluation metrics.

Summary. Our contribution is a specific continuous-time competing-risks output formulation that combines fixed, locally supported spline density components with jointly normalized event probabilities. The controlled head-replacement experiments directly address the missing ablation and support its discrimination benefits within the evaluated multimodal framework, while identifying a trade-off in mortality IBS. We will revise the contribution statement to reflect this scope and incorporate these experiments into the manuscript.

AuthorsOpenReview ↗

Response to Q1

Thank you for this valuable suggestion. We agree that SAMVAE is a particularly relevant baseline, as it also addresses continuous-time multimodal competing-risks prediction. Although we have not identified an additional suitable multimodal competing-risks dataset, we conducted a new comparison with a MIMIC-III adaptation of SAMVAE to strengthen the empirical evaluation.

Comparison with SAMVAE. SAMVAE employs a variational framework for multimodal representation learning and continuous-time survival prediction, using parametric event-time decoders based on Weibull distributions. In contrast, M2M-FSurv represents cause-specific event-time densities through patient-dependent masses over fixed B-spline density components, allowing locally varying density shapes with temporal resolution controlled by the knots and basis dimension.

Experimental Setup and Implementation Fairness. We additionally evaluated a MIMIC-III adaptation of SAMVAE, a closely related multimodal continuous-time competing-risks framework. To ensure a fair comparison, both methods used identical patient-level data partitions, a 24-hour landmark, input modalities, competing-event definitions, prediction horizons, and evaluation metrics.

The adaptation preserved SAMVAE's core methodological components, including its variational latent representation, reconstruction objective, KL-divergence regularization, and Weibull-based survival decoder. SAMVAE received the same clinical-text embeddings as M2M-FSurv, while physiological measurements were flattened into a 125-dimensional vector. Scalar covariates were processed using the same preprocessing methods as M2M-FSurv, and the resulting feature vector was provided to the SAMVAE encoder. SAMVAE followed the same training-set sampling strategy as M2M-FSurv, with minority death-event oversampling applied only to the training folds. For SAMVAE, sampling-probability correction was applied to the survival-likelihood term using pk/qkp_k/q_k, and post-training temperature scaling was fitted on validation data with the trained model parameters held fixed.

Hyperparameters were selected using validation data, and checkpoints were selected by maximizing the validation time-dependent C-index minus IBS, consistent with the M2M-FSurv evaluation protocol. Early stopping was based on validation loss. The original reconstruction and parametric survival modeling framework were retained rather than replaced with components from M2M-FSurv.

SAMVAE hyperparameters were independently optimized using validation data, with model selection and early stopping performed without access to held-out test outcomes. Both methods were evaluated using the same implementations of cause-specific time-dependent concordance and integrated Brier scores.

Experimental Results. Under this setting, M2M-FSurv achieved higher CtdC^{td}-index for both alive ICU discharge (0.788 vs. 0.709) and in-ICU death (0.866 vs. 0.703), together with lower discharge IBS (0.020 vs. 0.121). Mortality IBS was comparable (0.109 vs. 0.104).

Overall Findings. The additional comparison provides stronger empirical evidence for M2M-FSurv against an existing continuous-time multimodal competing-risks framework. We emphasize that this experiment evaluates the complete modeling frameworks rather than isolating the B-spline prediction heads. The controlled head-replacement experiments provide complementary evidence regarding the specific contribution of the proposed event-time density parameterization. We will incorporate the SAMVAE adaptation details and hyperparameter optimization protocol into the revised manuscript to improve reproducibility.

Table 1: Comparison of M2M-FSurv and SAMVAE on MIMIC-III. Results are averaged over ten held-out test folds, with three training seeds per fold. Parenthesized ranges denote 95% bootstrap confidence intervals for the CtdC^{\mathrm{td}}-index.

ModelInputDischarge CtdC^{{td}}-indexDeath CtdC^{{td}}-indexDischarge IBSDeath IBS
SAMVAET + F + S0.709 (0.701, 0.717)0.703 (0.701, 0.706)0.1210.104
M2M-FSurvT + F + S0.788 (0.774, 0.802)0.866 (0.821, 0.902)0.0200.109
AuthorsOpenReview ↗

Response to Q2

Thank you for this constructive suggestion. We agree that the original submission did not independently isolate the contribution of the B-spline prediction heads. To address this limitation, we conducted additional controlled head-replacement experiments comparing the proposed B-spline formulation with alternative event-time prediction heads.

Experimental design. Since the B-spline basis defines the cause-specific event-time densities, CIFs, and corresponding likelihood, simply removing it would not produce a valid prediction model. We therefore replaced the event-time prediction formulation while keeping the remaining architecture unchanged.

Specifically, we compared three prediction heads: (1) the proposed B-spline head, (2) a discrete-time probability head, and (3) a continuous-time log-normal mixture head. All configurations retained the same multimodal encoders, Transformer fusion architecture, input representations, patient-level data partitions, and prediction horizon. Each head was trained using its corresponding likelihood formulation, with ranking loss included across all three configurations.

Experimental results. The B-spline head achieved the highest time-dependent C-index for both alive ICU discharge (0.788) and in-ICU death (0.866), compared with the discrete-time head (0.729 and 0.775) and the log-normal mixture head (0.751 and 0.782). The B-spline formulation also achieved the lowest discharge IBS (0.020), although alternative heads achieved lower mortality IBS. Thus, the results support its advantage in cause-specific discrimination rather than universal superiority across all prediction metrics. We additionally evaluated all three prediction heads without ranking loss, as detailed in our response to Reviewer 1, Q1. The B-spline head maintained the highest observed cause-specific concordance for both outcomes, indicating that its discrimination advantage is not solely attributable to the ranking objective.

Interpretation. These controlled comparisons assess the predictive contribution of the B-spline head configuration within the evaluated multimodal architecture and training protocol. Compared with the continuous-time log-normal mixture head, the B-spline head achieved higher observed concordance for both competing events. These results support the usefulness of the chosen head configuration, without separately identifying the contributions of smoothness and local support.

Overall findings. The additional head-replacement experiments directly address the missing ablation identified by the reviewer and provide empirical support for the contribution of the proposed B-spline formulation. We will incorporate these results into the revised manuscript to more clearly distinguish the effects of the event-time representation from those of the multimodal architecture.

Table 2: Comparison of prediction heads, using the same multimodal encoders and fusion architecture.

Prediction headDischarge CtdC^{td}-index ↑Death CtdC^{td}-index ↑Discharge IBS ↓Death IBS ↓
B-spline0.7880.8660.0200.109
Discrete-time probability0.7290.7750.0750.064
Log-normal mixture0.7510.7820.0230.077
Reviewer_YEqmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes M2M-FSurv, a multimodal deep learning framework for continuous-time competing-risks survival analysis. The model integrates clinical notes, longitudinal physiological measurements, and scalar patient covariates using modality-specific encoders and a shared attention-based Transformer. It adopts a 24-hour landmark design and employs event-specific prediction heads based on normalized B-spline mixtures to estimate continuous-time density incidence functions and cumulative incidence functions while satisfying competing-risks probability constraints.The training objective combines a censored negative log-likelihood with a cause-specific ranking loss. The authors also establish a conditional consistency result for a restricted feedforward B-spline coefficient estimator. Experiments on MIMIC-III demonstrate improvements over the evaluated baselines in cause-specific discrimination and integrated Brier scores, while an additional scalar-only, single-event evaluation is conducted on METABRIC. The paper further provides modality and architectural ablation studies to examine the contributions of the input modalities and selected model components.

Critical strengths
  1. A principled continuous-time formulation for competing-risks prediction. A notable strength is the use of normalized B-spline mixtures to parameterize cause-specific event-time subdensities. By jointly normalizing the event probabilities and the spline mixture weights, the proposed formulation guarantees nonnegative event-time densities, nondecreasing cumulative incidence functions, and a valid overall survival probability over the analysis horizon. This offers a flexible alternative to discrete-time predictions and restrictive parametric event-time distributions. The resulting continuous-time predictions are also potentially useful for visualizing patient-specific event dynamics.
  2. Encouraging empirical performance on a clinically relevant multimodal task. The paper addresses a meaningful clinical prediction problem and evaluates the model using complementary clinical modalities. On MIMIC-III, the proposed method improves the mortality C-index from 0.801 for SurvTRACE to 0.866 and reduces the corresponding IBS from 0.165 to 0.109. The patient-level cross-validation protocol, training-only oversampling, and inclusion of modality and component ablations further strengthen the empirical evaluation. These results suggest that the overall framework has practical potential for competing-risks prediction.
Critical weaknesses
  1. The contribution of the proposed B-spline parameterization is insufficiently isolated, and the theoretical result provides limited support for the implemented model. The central methodological contribution is the continuous-time B-spline formulation for cause-specific event-time densities. However, the component ablations in Table 1 focus on temporal encoding, attention heads, padding masks, and the functional encoder, without directly evaluating the proposed density head. Consequently, it remains unclear whether the reported improvements mainly arise from the B-spline parameterization or from the multimodal representation and fusion architecture. Comparisons against alternative continuous-time density heads under the same encoder would be particularly important.Furthermore, the paper fails to include empirical comparisons against SAMVAE, the most closely related existing multimodal continuous‑time competing‑risks method. This missing comparison weakens the argument for the novelty and practical advantages of the proposed spline based modeling strategy. In addition, Theorem 4.1 establishes consistency only for a restricted feedforward coefficient class with an asymptotically vanishing ranking weight, rather than for the implemented Transformer and softmax-based prediction heads. Its approximation and complexity assumptions are not verified for the implemented architecture. These limitations weaken the evidence supporting the paper's central methodological contribution.
  2. Important questions remain regarding the reliability of absolute risk prediction and the evaluation protocol. The model aims to estimate cause-specific cumulative incidence probabilities, yet minority-event oversampling is applied during training without a clearly described correction for its possible effect on absolute risk calibration. The experiments report integrated Brier scores but do not provide time-specific calibration analyses. Furthermore, the IPCW formulation in Appendix A.6 uses a marginal censoring survival function, whereas the theoretical assumptions allow censoring to depend on patient covariates. The validity of this evaluation requires further clarification. Finally, Eq. (5) appears to contain an inconsistency in the cause-specific comparable-pair condition, which should be checked against the actual evaluation implementation. Together, these issues make it difficult to fully assess the reliability of the reported probability predictions and discrimination results.
Questions for authors

Q1. Contribution of the B-spline prediction heads. The B-spline density parameterization is a central methodological contribution of this work. However, the current ablation studies do not directly isolate its effect. Could the authors compare the proposed B-spline heads with alternative event-time parameterizations, such as discrete-time probability heads or parametric continuous-time mixtures, while keeping the multimodal encoders and fusion architecture unchanged? Additionally, an ablation without the ranking loss would help clarify the individual contributions of the proposed density representation and training objective. Q2. Clinical-note preprocessing and temporal information preservation. The model uses Kimi K2.6 to rewrite lengthy clinical notes into approximately 1,800–2,200 tokens, followed by Clinical Longformer and masked mean pooling to produce a single text token. Could the authors provide a comparison with directly encoding the original clinical notes, for example using chunk-based processing without Kimi rewriting? Furthermore, since clinical notes contain timestamps and sequential clinical events, could aggregating the entire narrative into a single token limit fine-grained temporal alignment with physiological measurements? Have the authors considered retaining multiple text-segment tokens for cross-modal attention, and how would this affect predictive performance and computational cost? Q3. Oversampling and probability calibration. The model directly estimates cause-specific event probabilities, while minority-event oversampling is applied during training. Could the authors clarify the exact oversampling ratio and whether any sampling-probability correction or post-training calibration was performed? Since oversampling may affect absolute risk estimation, it would be helpful to report calibration results at clinically relevant prediction horizons, or compare the current approach with training without oversampling.

9 comments
AuthorsOpenReview ↗

Response to Weakness 1: Isolation and Empirical Validation of the B-spline Contribution ( Part 1)

Thank you for highlighting the need to more directly establish the contribution of our B-spline event-time parameterization and strengthen comparisons with closely related continuous-time competing-risks methods. We agree that the original component ablations did not sufficiently isolate the effect of the prediction heads. To address these concerns, we conducted two complementary sets of additional experiments: controlled prediction-head replacements and a comparison with SAMVAE.

  1. Isolating the contribution of the B-spline parameterization.We conducted controlled experiments comparing three event-time parameterizations: the proposed B-spline head, a discrete-time probability head, and a continuous-time log-normal mixture head. All configurations retained the same multimodal encoders, Transformer fusion architecture, input representations, patient-level data partitions, and prediction horizon. Each head was evaluated both with and without ranking loss, using its corresponding likelihood formulation.The discrete-time head used 32 time bins, with boundaries defined by equally spaced points over the analysis horizon. The log-normal mixture head used 16 components per cause. The encoder and fusion modules were trained separately for each configuration, jointly with the corresponding prediction head. The learning rate and, when ranking loss was included, λ\lambda and ρ\rho were tuned separately for each configuration using validation data. The oversampling strategy, likelihood weighting, and model-selection protocol were kept consistent across all three prediction-head configurations and followed the same procedures as in the original experiments. The same post-training calibration protocol was applied to all three prediction-head configurations. Calibration parameters were optimized using validation data, with the trained model parameters kept fixed.

With ranking loss, the B-spline head achieved the highest time-dependent C-index for both alive ICU discharge (0.788) and in-ICU death (0.866), compared with the discrete-time head (0.729 and 0.775) and the log-normal mixture head (0.751 and 0.782). Without ranking loss, B-spline also achieved the highest observed C-index for both outcomes. These comparisons provide direct evidence that the B-spline event-time representation contributes to cause-specific discrimination beyond the shared multimodal encoding and fusion architecture. In particular, the comparison with the log-normal mixture head demonstrates that the observed advantage is not attributable solely to continuous-time prediction. We note that the B-spline head does not achieve the lowest integrated Brier score for every outcome, and therefore do not claim uniform superiority across all prediction metrics.

  1. Comparison with SAMVAE. We additionally evaluated a MIMIC-III adaptation of SAMVAE, a closely related method for multimodal continuous-time competing-risks prediction. Both methods used the same underlying clinical-text, longitudinal physiological, and scalar input modalities, patient-level data partitions, competing-event definitions, prediction horizons, and evaluation metrics. The SAMVAE adaptation retained its variational reconstruction objective, KL-divergence regularization, and Weibull-based survival decoder. SAMVAE received the same clinical-text embeddings as M2M-FSurv, while the physiological measurements were flattened into a 125-dimensional vector. Scalar covariates were processed using the same preprocessing methods as M2M-FSurv, and the resulting feature vector was provided as input to the SAMVAE encoder.SAMVAE used the same training-set sampling strategy as M2M-FSurv, with minority death-event oversampling applied only to the training folds. For SAMVAE, sampling-probability correction was applied to the survival-likelihood term using pk/qkp_k/q_k, and post-training temperature scaling was fitted on validation data with the trained model parameters held fixed. Hyperparameters were selected using validation data, and checkpoints were selected by maximizing the validation CtdC^{td}-index minus IBS, consistent with the M2M-FSurv evaluation protocol. Early stopping was based on validation loss. Under the reported training protocols, M2M-FSurv achieved higher CtdC^{td}-index for discharge (0.788 vs.\ 0.709) and death (0.866 vs.\ 0.703), and lower discharge IBS (0.020 vs.\ 0.121). SAMVAE had a numerically lower mortality IBS (0.104 vs.\ 0.109) for M2M-FSurv.

Table : Comparison of M2M-FSurv and SAMVAE on MIMIC-III. Results are averaged over ten held-out test folds, with three training seeds per fold. Parenthesized ranges denote 95% bootstrap confidence intervals for the (C^{\mathrm{td}})-index.

ModelInputDischarge CtdC^{td}-indexDeath CtdC^{td}-indexDischarge IBSDeath IBS
SAMVAET + F + S0.709 (0.701, 0.717)0.703 (0.701, 0.706)0.1210.104
M2M-FSurvT + F + S0.788 (0.774, 0.802)0.866 (0.821, 0.902)0.0200.109
AuthorsOpenReview ↗

Response to Weakness 1: Isolation and Empirical Validation of the B-spline Contribution ( Part 2)

The SAMVAE comparison benchmarks the full M2M-FSurv framework against a closely related method with a Weibull-based decoder on the same ICU prediction task. Table 2 complements this benchmark by comparing prediction heads within the same M2M-FSurv encoder and fusion architecture. In this controlled comparison, the B-spline head achieved higher observed time-dependent concordance than the continuous-time log-normal mixture head for both alive ICU discharge and in-ICU death, with and without ranking loss. These results support the contribution of the B-spline head to discrimination within the evaluated architecture.

Overall findings. These additional experiments address two limitations of the original evaluation. The controlled head-replacement experiments directly assess the contribution of the B-spline representation under a fixed multimodal architecture, while the SAMVAE comparison provides evidence of the full framework's predictive performance against a closely related method. Together, they strengthen the empirical support for our central methodological contribution: flexible, continuous-time cause-specific event-density modeling through patient-specific B-spline representations. We will incorporate these experiments and clarify the scope of our contribution in the revised manuscript. 3. Revised theoretical analysis for the Transformer and normalized prediction heads. We have replaced the restricted feedforward analysis with two complementary results addressing the architecture, approximation, and ranking-weight issues raised by the reviewer. First, revised Theorem 4.1 provides a fixed-model guarantee with a fixed positive ranking weight. For a fixed finite Transformer graph and spline basis, under the stated bounded-input, compact-parameter, mask, and sampling conditions, we derive complexity bounds through masked attention, residual connections, LayerNorm, pooling, and both softmax heads. For i.i.d. patient clusters with uniformly bounded numbers of stays, the theorem gives, with high probability,

Jλ,γ(θ^)−min⁡θ∈ΘJλ,γ(θ)≤2UN+2λVN+εN.J_{\lambda,\gamma}(\widehat{\theta})-\min_{\theta\in\Theta}J_{\lambda,\gamma}(\theta)\leq 2U_N+2\lambda V_N+\varepsilon_N.

Here �0��0� is the population objective induced by the specified sampler and loss reductions; UNU_N and VNV_N are uniform likelihood and ranking deviations; and εN\varepsilon_N is the global minimization gap for the conditional expected training criterion. For a fixed class and fixed ranking settings, the deviations vanish. Thus, a fixed positive λ\lambda permits convergence to the best combined-risk value within the class when εN→0\varepsilon_N\to0, without requiring the true distribution to belong to that class. Application to the reported runs remains conditional on matching the analyzed graph, sampler, and numerical loss reductions.

Second, the neural approximation premise is now proved for an explicit normalized Transformer family. The constructive proposition uses a one-layer post-norm/ReLU Transformer with a fixed text representation and at least five fixed attention heads. It handles both LayerNorm operations and the event-probability and spline-mixture softmax heads. For continuous, mask-compatible conditional densities with bounded first time derivative and the stated strict density and residual-probability margins, it constructs approximants as widths, parameter bounds, and spline dimension grow. With fixed spline degree, fixed positive LayerNorm epsilons, and bounded knot mesh ratio, the admissible coefficient class satisfies

log⁡N(η,GN,∥⋅∥∞,coef)≤PNlog⁡(1+C(BNwN)cη)\log \mathcal{N}\left(\eta,\mathcal{G}_N,\|\cdot\|_{\infty,\mathrm{coef}}\right)\leq P_N\log\left(1+\frac{C(B_Nw_N)^c}{\eta}\right)

Here PNP_N is the parameter count, BNB_N bounds the parameters, and wNw_N is the maximum of the network widths and spline dimension. The same constructive sequence achieves density sup-norm approximation error $ AN→0A_N\to0$ while ensuring �10��10�, where NN counts independent patients. Approximation and complexity are therefore verified simultaneously for this specified Transformer family.

The revised analysis distinguishes fixed-model risk optimality from recovery of the true event probabilities. A nonvanishing ranking term can shift the population optimum. The truth-consistency corollaries therefore additionally require the stated censoring conditions, vanishing optimization error, and vanishing ranking perturbation �12��12�, where �13��13� bounds the penalty’s oscillation.

The fixed experimental graph and M=32M=32 are addressed by the fixed-class result subject to its conditions; the growing-family construction does not establish universal truth consistency at those dimensions. Neither result asserts global convergence of AdamW. Complete statements and proofs are provided in the revised theory appendix.

AuthorsOpenReview ↗

Response to Weakness 2: Reliability of Absolute Risk Prediction and Evaluation Protocol (part 1)

Thank you for highlighting the importance of reliable probability estimation and a clearly defined evaluation protocol. We appreciate the opportunity to clarify the oversampling and calibration procedures, the IPCW formulation, and the cause-specific concordance definition. We also conducted an additional controlled experiment to assess the effect of oversampling.

  1. Oversampling and probability calibration. As detailed in our response to Q3, minority-event oversampling was applied exclusively to the training subsets, with an expected 1:2 ratio of in-ICU deaths to alive ICU discharges. To account for the altered event proportions, we applied importance weighting to the negative log-likelihood using pk/qkp_k/q_k, where pkp_k and qkq_k denote the original and expected oversampled event proportions, respectively. We additionally performed post-training temperature scaling using validation data, while keeping the trained model parameters fixed.

We assessed mean calibration for in-ICU death and alive ICU discharge at 24 and 72 hours after the 24-hour landmark. Outcome status was observed through both horizons, so we estimated observed cumulative incidence using empirical event proportions. Table 1 compares these proportions with the mean predicted CIFs from Set 1 and Set 3 on the same held-out ICU stays.

Table 1: Mean calibration at 24 and 72 hours after the landmark. Set 1 uses training-only oversampling and Set 3 uses training without oversampling. The Set 1 and Set 3 columns report mean predicted CIFs. All CIF values are percentages.

OutcomeHorizonObserved CIFSet 1Set 3
In-ICU death24 h1.41%2.03%3.64%
In-ICU death72 h2.99%3.27%5.56%
Alive ICU discharge24 h3.32%3.74%8.52%
Alive ICU discharge72 h13.53%14.34%11.82%

At 72 hours after the landmark, observed in-ICU mortality was 2.99%, compared with mean predicted risks of 3.27% in Set 1 and 5.56% in Set 3. Set 1 (with oversampling) showed closer agreement with the observed mortality risk, whereas Set 3 (without oversampling) exhibited greater overestimation.

  1. Clarification of the IPCW evaluation. We thank the reviewer for identifying the apparent discrepancy between the marginal censoring survival function in Appendix A.6 and the conditional censoring assumption in our theoretical analysis. We agree that conditional independent censoring alone does not justify the marginal IPCW formulation in Appendix A.6. We will revise the general formulation to use covariate-conditional censoring probabilities and explicitly state the conditional independence, positivity, and consistent censoring-estimation conditions required for validity.

We clarify that all ICU stays included in the MIMIC-III evaluation have observed competing-event outcomes (alive ICU discharge or in-ICU death), with the maximum post-landmark event time below the selected 5,000-hour analysis horizon. Therefore, no right censoring occurs in the evaluated cohort. With fully observed competing-event states throughout the evaluation interval, the cause-specific Brier scores can be calculated directly without estimating censoring probabilities. Observed competing events are treated as known negative outcomes for the cause of interest. Our implementation and numerical verification confirm that the direct calculation reproduces the reported discharge and mortality IBS values of 0.020 and 0.109, respectively.

The conditional censoring assumption in the theoretical analysis accommodates a more general setting, whereas the MIMIC-III evaluation corresponds to the special case without right censoring. We will explicitly clarify this distinction in Section 5.3.1 and Appendix A.6. For METABRIC, the dataset contains 1,093 right-censored observations among 1,981 patients. Since this experiment focuses on single-event survival discrimination and Table 2 of the original submission reports only the CtdC^{td}-index, checkpoint selection was based on maximizing the validation CtdC^{td}-index without incorporating IBS. Right censoring was accounted for in the time-dependent concordance evaluation. 3. Clarification of the cause-specific concordance definition.

We thank the reviewer for identifying the notational inconsistency in Eq. (5). The condition Di=kiD_i=k_i is a typographical error and should read k(i)=kk^{(i)}=k, where kk denotes the cause of interest. Specifically, for $ i \ne j $, define

Ak,ij=1{k(i)=k,  y(i)<y(j)},A_{k,ij}=\mathbf{1}\{k^{(i)}=k,\;y^{(i)}<y^{(j)}\},

=\widehat{F}_k(y^{(i)}\mid W^{(i)}) -\widehat{F}_k(y^{(i)}\mid W^{(j)}).$$ Subject $j$ must remain event-free and under observation beyond $y^{(i)}$, but may subsequently experience any cause or be censored. This pair definition excludes earlier competing events, earlier censoring, and tied observed times. Both CIFs are evaluated for the same cause at the same reference time.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.