acceptodds
Under review as a conference paper at ICLR 2027

RosFormer: Robust Stochastic Continuous-Time Transformer for Irregular Multivariate Time Series

Abstract

Irregular multivariate time series (IMTS) forecasting is a challenging problem due to the distribution shift incurred by changes in the observation density and missingness patterns. A stochastic continuous-time Transformer incorporates the linear-noise stochastic differential equations (LNSDEs) with a continuous-time attention to attenuate the influence of input perturbations and capture dependencies across observations, which, however, requires a costly numerical calculation of the integration. Patch-based methods, on the other hand, can reduce computation by compressing the local observations into patch embeddings, while the patch compression alone does not ensure robustness to such a shift. This thus motivates us to incorporate the patch-based modeling with a stochastic continuous-time Transformer for a robust and efficient forecasting under distribution shifts. Such an incorporation is non-trivial, as it requires adapting the discrete patch representations to continuous-time modeling, while preserving robustness to input perturbations across both the trajectory evolution and attention aggregation. In this paper, we propose RosFormer, a robust stochastic continuous-time Transformer for an efficient IMTS forecasting under distribution shifts. RosFormer captures the inter-variable dependencies within each patch through a relative-time-aware graph, and uses the resulting embeddings to initialize the LNSDE trajectories at their temporal anchors. These trajectories are then aggregated through compactness-aware attention, with the queries, keys and values sharing one trajectory per variable-level patch to reduce numerical integration costs. We further theoretically analyze the stability of the attention output under bounded perturbations of the initial patch embeddings. Experiments demonstrate the robustness of RosFormer to distribution shifts in the controlled test-time missingness and hospital-disjoint eICU-CRD settings, along with a competitive forecasting and event prediction performance and lower computational cost than the other continuous-time baselines.

Primary area:
learning on time series and dynamical systems
Keywords:
irregular multivariate time series, time series modeling, neural stochastic differential equations

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_WQ6QOpenReview ↗
Rating
2
Confidence
3(−1)
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RosFormer, a stochastic continuous-time Transformer for irregular multivariate time series forecasting under observation-process distribution shifts. The method constructs patch-level representations and uses a relative-time-aware graph to model cross-variable dependencies. These representations initialize linear-noise stochastic differential equations, with queries, keys, and values derived from shared stochastic trajectories. A compactness-aware continuous-time attention mechanism then aggregates the trajectories for prediction. The paper further provides a theoretical analysis of attention-output stability under bounded perturbations. Experiments evaluate robustness under controlled test-time missingness and hospital-disjoint distribution shifts, alongside standard forecasting and event prediction benchmarks, ablation studies, and computational efficiency analyses.

Critical strengths

The paper provides a comprehensive empirical evaluation across forecasting, event prediction, and distribution-shift settings. RosFormer achieves competitive predictive performance, particularly under severe test-time missingness on certain datasets. The efficiency analysis also demonstrates substantial computational savings while largely preserving predictive accuracy, supporting the practical effectiveness of the overall framework.

Critical weaknesses

1. Limited Methodological Novelty

The paper presents the integration of patch-based graph representations with LNSDE-driven continuous-time attention, along with three associated integration challenges, as its central methodological contribution. While the paper acknowledges the relevant foundations, the novelty of this integration appears limited. In particular, SDEFormer (Wang et al., 2025) has already incorporated stable Neural SDEs, including an LNSDE variant, into continuous-time Transformer attention for irregular time series modeling. This closely related work is not discussed, despite directly addressing a central component of RosFormer's proposed framework. On the patch side, the proposed approach builds on established mechanisms in T-PatchGNN (Zhang et al., 2024), Hi-Patch (Luo et al., 2025), and VIMTS (Hu et al., 2025), which the paper appropriately cites. Although the three integration challenges motivate specific design choices, their solutions primarily involve adapting existing patch representations to stochastic trajectories, extending attention with temporal weighting and stability analysis, and reducing computational overhead through patch-level evolution and shared Q/K/V trajectories. These are meaningful architectural refinements, but they do not appear to introduce a substantially distinct modeling mechanism. Consequently, the methodological contribution seems to lie mainly in a particular integration and localized extensions of established techniques, rather than a major methodological advance.

2. Insufficient Validation of the Distinctive Contributions

Although the experiments demonstrate the effectiveness of the overall framework, they do not sufficiently isolate the benefits of RosFormer's specific integration and design choices. The comparisons with existing models and the component-removal ablations do not clearly establish how much of the improvement arises from the proposed adaptations, rather than the established mechanisms themselves. This limits the empirical support for the paper's methodological contributions.

Questions for authors

The paper presents the non-trivial integration of patch-based representations with LNSDE-driven continuous-time attention as its main methodological contribution. Could the authors provide convincing evidence that RosFormer's specific integration offers substantial robustness benefits over reasonable alternative combinations of these established techniques? Such comparisons should incorporate both patch-based modeling and LNSDE-driven attention, rather than relying on component-removal ablations. While Appendix B evaluates a ContiFormer + LNSDE variant in terms of computational efficiency, this variant does not incorporate patch-based representations, and its predictive performance and robustness are not reported. More direct evidence would help clarify the methodological value of RosFormer's specific designs beyond combining existing mechanisms.

9 comments

Weakness 1.1: Methodological Novelty

Reviewer comment:

1. Limited Methodological Novelty

The paper presents the integration of patch-based graph representations with LNSDE-driven continuous-time attention, along with three associated integration challenges, as its central methodological contribution. While the paper acknowledges the relevant foundations, the novelty of this integration appears limited. In particular, SDEFormer (Wang et al., 2025) has already incorporated stable Neural SDEs, including an LNSDE variant, into continuous-time Transformer attention for irregular time series modeling. This closely related work is not discussed, despite directly addressing a central component of RosFormer's proposed framework.

Response:

Thank you for highlighting SDEFormer [C1]. We will include it in the related work and clarify its relationship to RosFormer in the revised manuscript. Both methods use neural SDEs within continuous-time attention. We summarize their differences and the additional designs in RosFormer below.

1). Different motivations.

RosFormer focuses on forecasting robustness to distribution shifts in the observation process, including changes in sampling density and missingness between training and deployment. SDEFormer-LN focuses on modeling stochastic continuous-time dynamics in irregular time series. RosFormer additionally introduces time-aware patch modeling, compactness-aware attention, and LNSDE-based query dynamics to improve forecasting robustness under these shifts.

2). Different model architectures.

RosFormer constructs continuous-time attention from variable-level patch representations. Each patch evolves along a latent LNSDE trajectory, from which Q, K, and V are obtained through separate projections. Thus, all three representations follow learned stochastic dynamics. SDEFormer-LN constructs Q by linear interpolation between observations and models K and V using separate LNSDEs.

The difference in query construction is particularly relevant when observations become sparse. Larger observation gaps constrain the interpolated Q to vary linearly over longer intervals. RosFormer allows Q to follow learned nonlinear dynamics over these intervals, providing greater flexibility in modeling queries under changes in observation patterns.

3). Improved computational efficiency.

Compared with SDEFormer-LN, which evolves K and V using two separate LNSDEs, RosFormer generates Q, K, and V from a single shared LNSDE trajectory per variable-level patch. SDEFormer-LN obtains Q through linear interpolation, and introducing a separate LNSDE for Q would require a third trajectory. RosFormer's shared construction enables stochastic modeling of all three representations without this additional integration cost.

4). Additional theoretical analysis.

RosFormer and SDEFormer-LN both incorporate LNSDEs into continuous-time attention. RosFormer further analyzes how perturbations to initial patch representations propagate through the latent trajectories and attention weights to affect the attention output. The analysis holds temporal anchors and compactness scores fixed under the stated assumptions.

5). Experimental validation.

RosFormer achieves lower MSE and MAE than SDEFormer-LN across all four datasets and evaluated missingness levels. Additional experiments evaluate the query modeling and trajectory sharing designs discussed above. The comparison with RosFormer (Linear Q) shows that LNSDE-based query modeling improves forecasting performance under test-time missingness. Shared and independent trajectories achieve comparable predictive performance, while sharing reduces training time, inference time, and GPU memory usage by 66.2%, 59.4%, and 30.5%, respectively, on Human Activity.

The experimental settings and detailed results are presented below.

Forecasting under controlled test-time missingness

We adapted SDEFormer-LN to our forecasting experiments and compared it with RosFormer and RosFormer (Linear Q), a patch-based variant that constructs Q by linear interpolation. Full denotes the original test inputs. The other settings randomly remove 30%, 50%, or 70% of the observed test inputs while keeping the prediction targets unchanged. Each entry reports MSE / MAE, with the metric scales shown alongside each dataset.

Table C.1. Forecasting performance under controlled test-time missingness.

DatasetModelFull30%50%70%
Human Activity (×10⁻³ / ×10⁻²)SDEFormer-LN2.77 / 3.343.11 / 3.453.64 / 4.024.80 / 4.94
RosFormer (Linear Q)2.67 / 3.112.94 / 3.393.34 / 3.804.49 / 4.58
RosFormer2.54 / 3.052.71 / 3.242.89 / 3.383.37 / 3.68
PhysioNet (×10⁻³ / ×10⁻²)SDEFormer-LN5.06 / 3.966.21 / 4.856.86 / 5.158.72 / 5.45
RosFormer (Linear Q)4.70 / 3.615.06 / 3.965.65 / 4.266.96 / 4.78
RosFormer4.55 / 3.465.00 / 3.865.61 / 4.086.73 / 4.60
USHCN (×10⁻¹ / ×10⁻¹)SDEFormer-LN4.96 / 3.005.10 / 3.135.17 / 3.265.25 / 3.40
RosFormer (Linear Q)4.90 / 2.954.94 / 3.025.00 / 3.085.04 / 3.11
RosFormer4.84 / 2.924.86 / 2.924.92 / 2.954.96 / 2.98
MIMIC-III (×10⁻² / ×10⁻²)SDEFormer-LN1.79 / 7.511.94 / 8.202.07 / 8.562.12 / 8.66
RosFormer (Linear Q)1.44 / 6.561.52 / 6.741.57 / 6.941.67 / 7.16
RosFormer1.30 / 6.321.43 / 6.681.49 / 6.781.53 / 7.12

RosFormer maintains lower prediction errors than both alternatives across all four datasets. The comparison with Linear Q supports the value of trajectory-based query construction under sparse inputs. On Human Activity, for example, MSE increases by 32.7% from Full to 70% missingness for RosFormer, compared with 68.2% for Linear Q, providing additional evidence of reduced sensitivity to observation removal on this dataset.

Efficiency and predictive performance of trajectory sharing

To evaluate trajectory sharing, we compare RosFormer with RosFormer (Independent LNSDEs for Q/K/V). The independent variant retains patch-based representations but solves three separate LNSDE trajectories for Q, K, and V. The shared variant generates all three through projections of one latent trajectory.

The efficiency comparison uses Human Activity with a batch size of 32. Training and inference times are measured per epoch.

Table C.2. Computational efficiency of independent and shared trajectories on Human Activity.

ModelTraining time (s/epoch)Inference time (s/epoch)GPU memory (MB)
RosFormer (Independent LNSDEs for Q/K/V)25.329.265828
RosFormer (Shared trajectory)8.553.764048

We also evaluate both variants under the controlled test-time missingness protocol to examine whether these efficiency gains affect forecasting performance.

Table C.3. Forecasting performance of independent and shared trajectories under controlled test-time missingness. Each entry reports MSE / MAE, with the lowest confirmed MSE in bold.

DatasetModelFull30%50%70%
Human Activity (×10⁻³ / ×10⁻²)RosFormer (Independent LNSDEs for Q/K/V)2.52 / 3.052.73 / 3.202.95 / 3.383.42 / 3.76
RosFormer (Shared trajectory)2.54 / 3.052.71 / 3.242.89 / 3.383.37 / 3.68
PhysioNet (×10⁻³ / ×10⁻²)RosFormer (Independent LNSDEs for Q/K/V)4.53 / 3.594.99 / 3.885.47 / 4.096.77 / 4.63
RosFormer (Shared trajectory)4.55 / 3.465.00 / 3.865.61 / 4.086.73 / 4.60
USHCN (×10⁻¹ / ×10⁻¹)RosFormer (Independent LNSDEs for Q/K/V)4.86 / 2.944.88 / 2.954.93 / 2.975.00 / 3.01
RosFormer (Shared trajectory)4.84 / 2.924.86 / 2.924.92 / 2.954.96 / 2.98
MIMIC-III (×10⁻² / ×10⁻²)RosFormer (Independent LNSDEs for Q/K/V)1.32 / 6.361.44 / 6.701.52 / 7.021.57 / 7.23
RosFormer (Shared trajectory)1.30 / 6.321.43 / 6.681.49 / 6.781.53 / 7.12

The two variants achieve similar forecasting errors across the evaluated missingness levels. Together with the efficiency results, this supports trajectory sharing as a way to reduce computation while retaining comparable predictive performance under changing observation patterns. We will include these additional evaluations in the revised manuscript.

Weakness 1.2: Methodological Novelty

Reviewer comment:

1. Limited Methodological Novelty

On the patch side, the proposed approach builds on established mechanisms in T-PatchGNN (Zhang et al., 2024), Hi-Patch (Luo et al., 2025), and VIMTS (Hu et al., 2025), which the paper appropriately cites. Although the three integration challenges motivate specific design choices, their solutions primarily involve adapting existing patch representations to stochastic trajectories, extending attention with temporal weighting and stability analysis, and reducing computational overhead through patch-level evolution and shared Q/K/V trajectories. These are meaningful architectural refinements, but they do not appear to introduce a substantially distinct modeling mechanism. Consequently, the methodological contribution seems to lie mainly in a particular integration and localized extensions of established techniques, rather than a major methodological advance.

Response:

RosFormer builds on established patch encoders and LNSDE dynamics. Its methodological contribution lies in jointly accounting for observation-process shifts when constructing initial patch states and aggregating their stochastic trajectories. Temporal anchors determine both the relative-time graph corrections and the starting times of continuous evolution, while compactness modulates both these corrections and attention weights. These connections make the temporal structure of the observed data an explicit part of the modeling process from cross-variable interaction to trajectory aggregation.

1). Observation timing links patch-state construction with stochastic attention.

Changes in observation patterns can alter both the representative time of a patch and the temporal spread of its observations. RosFormer retains these properties explicitly through temporal anchors and compactness scores after patch encoding.

When a patch's anchor changes, its temporal offsets from other variables change accordingly. These offsets enter graph construction, affecting the initial state supplied to the LNSDE, while the same anchor specifies when that state begins to evolve. Cross-variable aggregation and continuous-time initialization therefore use a consistent description of the patch's temporal position.

Changes in temporal dispersion are handled across these stages as well. When observations are more dispersed around an anchor, compactness reduces the magnitude of the relative-time edge correction and adjusts the patch's contribution to attention. At equal trajectory similarity, a more concentrated patch receives greater weight. Compactness thus connects how a patch participates in constructing initial states with how its evolved trajectory contributes to the prediction.

2). A direct combination baseline assesses the additional value of this formulation.

The added Patch + SDEFormer-LN baseline combines VIMTS's [C2] patch-processing module with SDEFormer-LN. This comparison retains both established components and evaluates the benefit of RosFormer's specific formulation.

As shown in Tables C.4 and C.5 (see our response to Weakness 2 & Question), RosFormer achieves lower MSE and MAE across all four datasets and evaluated missingness levels, as well as on hospital-disjoint eICU. For example, at 70% missingness on Human Activity, MSE decreases from 4.65×10−34.65 \times 10^{-3} to 3.37×10−33.37 \times 10^{-3}. These results support the practical value of the proposed design under observation-process shifts, complementing the methodological distinctions described above.

The revised manuscript will clarify the connections between these designs, distinguish them from the inherited components, and include the additional comparison.

Reviewer_TWYsOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RosFormer, a stochastic continuous-time Transformer for robust and efficient irregular multivariate time series forecasting under changes in observation density and missingness patterns. It combines patch-based representations with relative-time-aware graph modeling to capture asynchronous cross-variable dependencies, and uses LNSDEs to model stable continuous-time trajectories. Compactness-aware attention aggregates these trajectories, while shared Q/K/V construction reduces numerical integration costs. The paper also extends existing LNSDE stability analysis to attention outputs, deriving a bound on deviations caused by initial perturbations. Experiments on standard forecasting and event prediction benchmarks, controlled test-time missingness, and hospital-disjoint evaluation demonstrate strong predictive performance and robustness, while efficiency experiments show reduced computational costs.

Critical strengths

S1. The proposed framework is well motivated and reasonably designed. The paper combines the computational efficiency of patch-based modeling with the perturbation stability of LNSDEs and continuous-time attention to address observation-process shifts in irregular multivariate time series. Although the individual techniques are largely established, their integration is meaningful and well aligned with the problem.

S2. The theoretical analysis provides additional support for the proposed framework. The paper extends existing LNSDE stability results from individual trajectories to attention outputs, providing a theoretical bound on the propagation of initial perturbations through continuous-time attention.

S3. The experiments demonstrate promising robustness and efficiency. The evaluation covers controlled test-time missingness and real-world cross-hospital distribution shifts. RosFormer achieves consistently strong forecasting performance under increasing missingness levels, while the efficiency analysis shows substantial computational savings from patch-level trajectories and shared Q/K/V construction.

Critical weaknesses

W1. The ablation study does not sufficiently isolate the contributions of individual components to robustness. While the controlled missingness experiments demonstrate RosFormer's robustness relative to existing baselines, the component ablations are conducted only on the hospital-disjoint eICU-CRD dataset. Evaluating these variants under different missingness levels would help clarify how each component contributes to the observed robustness.

W2. The hospital-disjoint evaluation may involve confounding distribution shifts (minor concern). Differences across hospitals may also involve changes in patient populations and physiological distributions. Thus, the improved cross-hospital forecasting performance cannot be attributed solely to robustness against changes in observation patterns.

Questions for authors

Q1. It would be helpful to extend the ablation study to the controlled missingness settings to better understand the robustness contributions of individual components.

Q2. In Appendix C.5, Step 3.1, the inequality E[(X+Y)2]≤E[X2]+E[Y2]\mathbb{E}[(X+Y)^2] \leq \mathbb{E}[X^2]+\mathbb{E}[Y^2] appears to require an additional constant factor or an assumption on the cross term. While this may not affect the main conclusion, I suggest checking this step and carefully reviewing the rigor of the theoretical derivations throughout the paper.

5 comments

Weakness 1 & Question 1: Component Ablations under Controlled Missingness

Reviewer comment:

W1. The ablation study does not sufficiently isolate the contributions of individual components to robustness. While the controlled missingness experiments demonstrate RosFormer's robustness relative to existing baselines, the component ablations are conducted only on the hospital-disjoint eICU-CRD dataset. Evaluating these variants under different missingness levels would help clarify how each component contributes to the observed robustness.

Q1. It would be helpful to extend the ablation study to the controlled missingness settings to better understand the robustness contributions of individual components.

Response:

Thank you for the suggestion. We extended the component ablations to controlled test-time missingness on Human Activity, PhysioNet, USHCN, and MIMIC-III. Following the original protocol, we randomly removed 30%, 50%, or 70% of the observed test inputs and evaluated all variants on the original prediction targets. Full denotes the original test inputs. Each entry reports MSE / MAE, with the lowest value for each metric in bold.

Table B.1. Component ablations under controlled test-time missingness.

Dataset (MSE / MAE scales)VariantFull30%50%70%
Human Activity (×10⁻³ / ×10⁻²)w/o Stochasticity2.81 / 3.263.22 / 3.643.78 / 4.084.92 / 4.90
w/o Linear-Noise Constraint2.74 / 3.203.07 / 3.483.53 / 3.934.68 / 4.76
w/o Relative-Time Edges2.69 / 3.153.01 / 3.463.51 / 3.904.67 / 4.72
w/o Compactness Weighting2.65 / 3.112.91 / 3.403.28 / 3.704.27 / 4.39
RosFormer2.54 / 3.052.71 / 3.242.89 / 3.383.37 / 3.68
PhysioNet (×10⁻³ / ×10⁻²)w/o Stochasticity4.82 / 3.895.55 / 4.406.12 / 4.647.92 / 5.52
w/o Linear-Noise Constraint4.75 / 3.755.22 / 4.145.75 / 4.337.10 / 4.83
w/o Relative-Time Edges4.71 / 3.715.15 / 3.985.66 / 4.197.03 / 4.74
w/o Compactness Weighting4.68 / 3.805.08 / 4.085.62 / 4.306.84 / 4.87
RosFormer4.55 / 3.465.00 / 3.865.61 / 4.086.73 / 4.60
USHCN (×10⁻¹ / ×10⁻¹)w/o Stochasticity4.92 / 2.985.14 / 3.195.21 / 3.215.30 / 3.29
w/o Linear-Noise Constraint4.90 / 2.984.96 / 2.975.01 / 3.085.06 / 3.15
w/o Relative-Time Edges4.89 / 2.964.94 / 3.004.98 / 3.005.02 / 3.10
w/o Compactness Weighting4.88 / 2.954.92 / 2.974.96 / 3.045.03 / 3.08
RosFormer4.84 / 2.924.86 / 2.924.92 / 2.954.96 / 2.98
MIMIC-III (×10⁻² / ×10⁻²)w/o Stochasticity1.73 / 7.531.86 / 7.731.99 / 7.842.07 / 8.56
w/o Linear-Noise Constraint1.58 / 6.851.67 / 7.021.75 / 7.111.80 / 7.49
w/o Relative-Time Edges1.53 / 6.841.62 / 6.851.68 / 6.921.73 / 7.25
w/o Compactness Weighting1.47 / 6.721.57 / 6.811.64 / 6.901.70 / 7.17
RosFormer1.30 / 6.321.43 / 6.681.49 / 6.781.53 / 7.12

RosFormer achieves the lowest MSE and MAE at every missingness level across all four datasets. Removing stochasticity also produces larger increases in both metrics from Full to 70% missingness on every dataset. For example, on Human Activity, MSE increases by 32.7% for RosFormer, compared with 75.1% without stochasticity, supporting its role in limiting degradation as observations become sparser.

The other components improve accuracy across missingness levels, while their effects on degradation vary by dataset and metric. For example, on Human Activity, the MSE gap caused by removing compactness weighting grows from 0.11 under Full inputs to 0.90 at 70% missingness, whereas on MIMIC-III it remains 0.17 in both settings. These comparisons help distinguish improvements in overall accuracy from reductions in missingness-induced degradation. We will include the results and discussion in the revised manuscript.

Weakness 2: Potential Confounding Shifts in the Cross-Hospital Evaluation

Reviewer comment:

W2. The hospital-disjoint evaluation may involve confounding distribution shifts (minor concern). Differences across hospitals may also involve changes in patient populations and physiological distributions. Thus, the improved cross-hospital forecasting performance cannot be attributed solely to robustness against changes in observation patterns.

Response:

To assess whether patient and physiological differences could affect the interpretation of our cross-hospital results, we supplemented our original analysis of sampling patterns with comparisons of age, sex, and physiological measurements. As discussed in our original manuscript, the training and test sets differ in how often variables are observed and how these observations are distributed over time (Figure 5, Appendix A.5.2). The additional comparisons show that, despite these sampling differences, the two sets have similar mean ages, sex proportions, and per-stay physiological averages. These findings provide further support for interpreting RosFormer's cross-hospital performance as evidence of robustness to naturally varying observation patterns.

Specifically, we compared age and sex between the hospital-disjoint training and test sets used in Section 5.1. For each of the 12 physiological variables, we first calculated its average within each ICU stay, then computed the mean and standard deviation of these averages across stays in each set. The tables below report these statistics, together with absolute standardized mean differences (SMDs) and relative changes in physiological means.

Table B.2. Comparison of patient populations.

CharacteristicTraining setTest setAbsolute SMD
Age, mean ± SD (years)64.185 ± 16.32664.553 ± 15.9620.023
Male, n (%)54.28%52.08%0.044

Age is reported as the mean ± standard deviation (SD) in years. The second row gives the percentage of stays involving male patients. The absolute standardized mean difference (SMD) expresses the difference in mean age or male proportion in pooled standard deviation units. Smaller values indicate smaller differences between the splits.

Table B.3. Comparison of physiological distributions.

VariableTraining mean ± SDTest mean ± SDRelative change in meanAbsolute SMD
Systolic NIBP120.391 ± 18.016119.926 ± 17.227−0.39%0.026
Diastolic NIBP65.359 ± 11.10864.591 ± 10.395−1.17%0.071
Mean NIBP81.289 ± 12.83880.063 ± 11.913−1.51%0.099
Glucose141.746 ± 48.972141.012 ± 49.194−0.52%0.015
Blood urea nitrogen26.990 ± 20.19927.109 ± 20.017+0.44%0.006
Creatinine1.536 ± 1.5051.571 ± 1.546+2.27%0.023
Sodium138.625 ± 4.899138.888 ± 4.883+0.19%0.054
Potassium4.052 ± 0.5054.049 ± 0.522−0.08%0.006
Chloride104.790 ± 6.361105.094 ± 6.420+0.29%0.048
Hemoglobin10.725 ± 2.06610.690 ± 2.107−0.33%0.017
White blood cell count12.188 ± 6.11611.859 ± 6.078−2.70%0.054
Platelet count192.983 ± 88.910195.647 ± 90.762+1.38%0.030

Mean and SD summarize the per-stay averages for each variable. Relative change is the percentage change from the training mean to the test mean. A smaller absolute SMD indicates a smaller difference in means relative to the variability across stays.

As shown in Tables B.2 and B.3, the small absolute SMDs and relative changes in physiological means indicate that the training and test sets are similar in the measured patient characteristics and per-stay physiological averages. These results reduce concerns about confounding from these characteristics when interpreting the cross-hospital performance gains. Together with the sampling differences documented in Figure 5, these findings provide further support for RosFormer's robustness to naturally varying observation patterns.

Question 2: Clarification of the Inequality in Appendix C.5

Reviewer comment:

Q2. In Appendix C.5, Step 3.1, the inequality E[(X+Y)2]≤E[X2]+E[Y2]E[(X+Y)^2]\le E[X^2]+E[Y^2] appears to require an additional constant factor or an assumption on the cross term. While this may not affect the main conclusion, I suggest checking this step and carefully reviewing the rigor of the theoretical derivations throughout the paper.

Response:

We would like to clarify that the inequality used in Appendix C.5, Step 3.1 of the submitted manuscript is

\le \sqrt{\mathbb{E}[X^2]}+\sqrt{\mathbb{E}[Y^2]}.$$ We provide the complete proof below. Let $X$ and $Y$ be real-valued random variables with finite second moments. If $\mathbb{E}[Y^2]=0$, both sides equal $\sqrt{\mathbb{E}[X^2]}$. Otherwise, $\mathbb{E}[Y^2]>0$, and $$\begin{aligned} 0 \le \mathbb{E}\left[ \left(X-\frac{\mathbb{E}[XY]}{\mathbb{E}[Y^2]}Y\right)^2 \right] = \mathbb{E}[X^2] -2\frac{(\mathbb{E}[XY])^2}{\mathbb{E}[Y^2]} +\frac{(\mathbb{E}[XY])^2}{\mathbb{E}[Y^2]} \\ = \mathbb{E}[X^2] -\frac{(\mathbb{E}[XY])^2}{\mathbb{E}[Y^2]}. \end{aligned}$$ This implies $$(\mathbb{E}[XY])^2 \le \mathbb{E}[X^2]\mathbb{E}[Y^2], \qquad \text{and} \qquad \mathbb{E}[XY] \le \sqrt{\mathbb{E}[X^2]\mathbb{E}[Y^2]}.$$ Therefore, $$\begin{aligned} \mathbb{E}[(X+Y)^2] = \mathbb{E}[X^2]+2\mathbb{E}[XY]+\mathbb{E}[Y^2] \\ \le \mathbb{E}[X^2] +2\sqrt{\mathbb{E}[X^2]\mathbb{E}[Y^2]} +\mathbb{E}[Y^2] \\ = \left( \sqrt{\mathbb{E}[X^2]}+\sqrt{\mathbb{E}[Y^2]} \right)^2. \end{aligned}$$ Taking square roots gives $$\sqrt{\mathbb{E}[(X+Y)^2]} \le \sqrt{\mathbb{E}[X^2]}+\sqrt{\mathbb{E}[Y^2]}.$$
Reviewer_rLV6OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
1
Read review
Paper summary

RosFormer forecasts irregular multivariate time series under changes in observation patterns. It summarizes each variable's observations in fixed windows into patch embeddings, an GNN and a linear-noise stochastic differential equation (LNSDE) trajectory that supplies each patch's queries, keys and values through linear projections. The evaluation showing superiority over baselines is performed on datasets with left-out historical observations and hospital-specific dataset split.

Critical strengths
  • Baselines are outperformed in both evaluation settings (plus more in the Appendix).
  • The ablation study confirms contributions of the proposed components.
Critical weaknesses
  • The problem definition is not aligned throughout the paper: Section 3.1 defines x^\hat{x} as output, Section 3.4 does not even mention the output, Appendix A.2 and A.3 have Y^\hat{Y}. Furthermore, while the task of event prediction extends the paper's contributions, it comes quite out of nowhere in Appendix A.2.
  • More details about the graph and its GNN could be given.
  • For the eICU-CRD test set, it is not fully clear if the performance can be attributed to RosFormer's capability of dealing with distribution shifts.
  • There are several issues with the writing/structure, specifically in the Appendix: Appendix C header is broken, references to assumptions in - Lemma C.5 are unclear, Corollary C.8 mixes lambda notations, "Assumptions C.2 and C.2", "Relative-Time Edges" vs. "Time-Aware Edges", …
  • No code is provided for review.

Minor comments:

  • Figures should be embedded as PDFs/SVGs.
  • "Normalized power" in Figure 3 is unclear.
  • Few cases of missing space after ".".
  • Give few details about the forecasting decoder in the main paper (and not just reference to Appendix).
Questions for authors
  • Q1: Can you provide more details on the graph construction and the GNN? What are the nodes in the graph representing and what kind of GNN is used? Is it comparable to GNNs used in baselines such as T-PatchGNN?
  • Q2: For Table 2, can you attribute RosFormer's improvement over the baseline to its superior performance regarding the hospital split / can you exclude that it is solely about superior accuracy in general?
5 comments

Weakness 1: The problem definition is not aligned throughout the paper:

Reviewer comment:

The problem definition is not aligned throughout the paper: Section 3.1 defines x^(t)\hat{\mathbf{x}}(t) as output, Section 3.4 does not even mention the output, Appendix A.2 and A.3 have Y^\hat{\mathbf{Y}}. Furthermore, while the task of event prediction extends the paper's contributions, it comes quite out of nowhere in Appendix A.2.

Response:

Thank you for your careful review. For IMTS forecasting, x^(t)\hat{\mathbf{x}}(t) denotes the predicted values at a single query time, while Y^\hat{\mathbf{Y}} collects these predictions across query times and, in Appendix A.3, across a batch. In Section 3.4, ZL(t)Z^L(t) is the encoder output, which is mapped to x^(t)\hat{\mathbf{x}}(t) by the MLP forecasting decoder. We will state this mapping explicitly in Section 3.4 and unify the notation and query-time formulation across the main text and Appendices A.2--A.3.

Event prediction provides a supplementary evaluation of the encoder on irregular event sequences, using task-specific heads to predict the next event's time and type. We will briefly introduce this task and its motivation in Section 3.1, with the detailed formulation and decoders provided in Appendices A.2--A.3.

Weakness 2 & Question 1: Graph Construction and GNN Architecture

Reviewer comment:

More details about the graph and its GNN could be given.

Q1:Can you provide more details on the graph construction and the GNN? What are the nodes in the graph representing and what kind of GNN is used? Is it comparable to GNNs used in baselines such as T-PatchGNN?

Response:

For each temporal window, our graph has one node per variable, represented by a patch embedding summarizing its observations within that window. We use an adaptive GCN with a residual connection for cross-variable aggregation, similar to VIMTS [A1] and T-PatchGNN [A2]. We additionally introduce a relative-time-aware edge correction to account for temporal misalignment between these patches.

Specifically, we combine learnable static variable embeddings with current patch features through a gating mechanism to obtain two node embedding matrices, Ep,1E_{p,1} and Ep,2E_{p,2}, representing target and source variables, respectively. Their product provides the content based edge scores. We then add the temporal correction BpτB_p^\tau and construct the adjacency matrix as Ap=Softmax⁡(ReLU⁡(Ep,1Ep,2⊤+Bpτ)).A_p=\operatorname{Softmax}\left( \operatorname{ReLU}\left(E_{p,1}E_{p,2}^{\top}+B_p^\tau\right) \right). For an edge from variable mm to variable nn, the correction uses the signed offset tˉpm−tˉpn\bar t_p^m-\bar t_p^n, where each anchor tˉpn\bar t_p^n is the mean observation time of that patch. The correction is zero when the anchors coincide and is scaled by source-patch compactness, so more dispersed observations produce a smaller temporal correction. Softmax normalizes each row over source variables; thus, Ap[n,m]A_p[n,m] determines the contribution of variable mm to the update of variable nn.

The GCN uses these weights to aggregate transformed patch features, applies ReLU, and adds each node's original feature through the residual connection. We will clarify the node definitions, graph construction, and GCN update in Section 3.2.

[A1]Hu, Zhangyi, et al. "IMTS is worth time ×\times channel patches: Visual masked autoencoders for irregular multivariate time series prediction." ICML. 2025.

[A2]Zhang, Weijia, et al. "Irregular Multivariate Time Series Forecasting: A Transformable Patching Graph Neural Networks Approach." ICML. 2024.

Weakness 3 & Question 2: Attribution of Cross-Hospital Performance Gains

Reviewer comment:

For the eICU-CRD test set, it is not fully clear if the performance can be attributed to RosFormer's capability of dealing with distribution shifts.

Q2: For Table 2, can you attribute RosFormer's improvement over the baseline to its superior performance regarding the hospital split / can you exclude that it is solely about superior accuracy in general?

Response:

To assess whether RosFormer's advantage in Table 2 reflects robustness to cross-hospital shifts in addition to general predictive accuracy, we added a same-hospital comparison. Using each model's same-hospital performance as a reference, we quantify the relative increase in prediction error under the hospital-disjoint setting. Experimental results show that RosFormer achieves the lowest MSE and MAE in both settings and exhibits the smallest relative increase in MSE. These results suggest that its advantage also reflects reduced sensitivity to cross-hospital shifts, particularly in MSE.

For the same-hospital setting, we kept the original test set fixed and trained each model using cases from the hospitals represented in that test set. We compared these results with the hospital-disjoint results in Table 2. The table below uses the same reporting scale as Table 2. "Same" and "Disjoint" denote the two settings, and relative error increases are calculated from the same-hospital to the hospital-disjoint setting.

Table A.1. Forecasting performance under same-hospital and hospital-disjoint settings on eICU-CRD.

ModelMSE (×10−1\times 10^{-1}) Same / DisjointMAE (×10−1\times 10^{-1}) Same / Disjoint∣ΔMSE∣\lvert\Delta \mathrm{MSE}\rvert (×10−1\times 10^{-1})∣ΔMAE∣\lvert\Delta \mathrm{MAE}\rvert (×10−1\times 10^{-1})
T-PatchGNN3.85 / 3.914.30 / 4.380.060.08
Hi-Patch3.92 / 4.014.34 / 4.410.090.07
VIMTS3.86 / 3.954.33 / 4.400.090.07
ContiFormer3.95 / 4.224.36 / 4.620.270.26
LNSDE4.02 / 4.064.40 / 4.450.040.05
ASTGI3.76 / 3.834.29 / 4.300.070.01
TFMixer3.80 / 3.874.34 / 4.360.070.02
RosFormer3.66 / 3.694.19 / 4.210.030.02

RosFormer's MSE increases by 0.03×10−10.03 \times 10^{-1}, compared with baseline increases ranging from 0.04×10−10.04 \times 10^{-1} to 0.27×10−10.27 \times 10^{-1}. Its MAE increase is also small at 0.02×10−10.02 \times 10^{-1}, although not the lowest. These results provide additional evidence of robustness to the cross-hospital shift, particularly in MSE. We will include this comparison in the revised manuscript.

Weakness 4: Writing and Structure

Reviewer comment:

There are several issues with the writing/structure, specifically in the Appendix: Appendix C header is broken, references to assumptions in - Lemma C.5 are unclear, Corollary C.8 mixes lambda notations, "Assumptions C.2 and C.2", "Relative-Time Edges" vs. "Time-Aware Edges", …

Response:

Thank you for your careful review. We will make the following corrections in the revised manuscript:

  1. Complete the Appendix C heading as "Stochastic Stability Analysis and Proofs".

  2. Correct the references in Lemma C.5 from "Assumptions 3.1, 3.2, and 3.3" to "Assumptions C.2, C.3, and C.4".

  3. Replace λ\lambda in the text of Corollary C.8 with LsoftL_{\mathrm{soft}}, the softmax Lipschitz constant, to match Eq. (26).

  4. Replace "Assumptions C.2 and C.2 (Lipschitz continuity and Linear Growth)" with "Assumption C.2 (Lipschitz continuity)", since this assumption directly gives the bound in Eq. (34).

  5. Use "Relative-Time Edges" consistently throughout the manuscript, including the ablation table.

We will also check the manuscript for other inconsistencies in notation, cross-references, and formatting.

Weakness 5: Code Availability

Reviewer comment:

No code is provided for review.

Response:

We have made our implementation available for review through the following anonymous repository: https://anonymous.4open.science/r/submission-code-2BFC/

Minor Comments 1: Figure Format

Reviewer comment:

Figures should be embedded as PDFs/SVGs.

Response:

Thank you for the suggestion. We will embed the figures as vector graphics (PDF/SVG) in the revised manuscript to improve visual clarity.

Minor Comments 2: Definition of Normalized Power

Reviewer comment:

"Normalized power" in Figure 3 is unclear.

Response:

Figures 3(a) and 3(c) both use Lomb–Scargle periodograms to analyze irregularly sampled time series. We first compute the standard-normalized power at each frequency, which measures the fraction of observed variation explained by a sinusoidal fit relative to a constant-mean baseline. Its values lie between 0 and 1. The subsequent processing differs according to what each panel aims to show.

  • Figure 3(a). We further divide the power at each frequency by the sum of powers over the evaluated frequency grid, so that each spectrum sums to one. We then average these normalized spectra across time-series segments. The vertical axis represents the relative share of spectral power at each frequency, allowing us to examine how observation removal changes the relative distribution of estimated power across frequencies.

  • Figure 3(c). We retain the standard-normalized power to compare the strength of frequency components between the hospital-disjoint training and test sets. We apply Gaussian smoothing to each spectrum to reduce small local fluctuations, then average the smoothed spectra across eligible ICU stays. The vertical axis therefore shows the averaged, smoothed power on the standard-normalized scale, making the overall spectral differences easier to observe.

We will clarify these definitions and processing steps in the figure caption and relabel the vertical axes of Figures 3(a) and 3(c) as "Relative spectral power" and "Smoothed normalized Lomb–Scargle power", respectively.

Minor Comments 3: Typographical Corrections

Reviewer comment:

Few cases of missing space after ".".

Response:

We will add the missing spaces after periods and proofread the manuscript for other typographical errors.

Minor Comments 4: Forecasting Decoder Details

Reviewer comment:

Give few details about the forecasting decoder in the main paper (and not just reference to Appendix).

Response:

We will add a brief description of the forecasting decoder to Section 3.4. At each query time tt, the final encoder representation ZL(t)Z^L(t) is passed through an MLP to predict the values of all variables. These predictions are collected across query times to form the forecasting output, and the model is trained by minimizing the mean squared error against the observed targets.

Comments on the paper

2 comments

Dear Reviewers rLV6, TWYs, and WQ6Q,

The authors have responded to your questions. Please review their responses and consider whether your concerns have been addressed. You may also read the other reviewers' comments and the revised manuscript, and share any further feedback.

Best regards,

Your AC

AuthorsOpenReview ↗

General Response

Dear Area Chair and Reviewers rLV6, TWYs, and WQ6Q,

We thank all three reviewers for their constructive feedback. We appreciate that TWYs recognized RosFormer's robustness relative to baselines under controlled missingness, WQ6Q acknowledged the effectiveness of the overall framework, and rLV6 noted that the event prediction experiments extend the scope of the work. We summarize our four main revisions below.

  1. Clarifications on task, method, theory, and implementation. In Section 3.1, we introduced the event prediction task and explained why we included it as a supplementary experiment. We clarified the graph construction and GNN in Section 3.2, described the forecasting decoder in Section 3.4, and completed the inequality derivation in Appendix C.5. Our implementation is available at https://anonymous.4open.science/r/submission-code-2BFC/.

  2. Cross-hospital evaluation. With the eICU test set fixed, RosFormer showed the smallest MSE increase from same-hospital to hospital-disjoint training (Section 5.2). We also found only small differences in mean age, sex proportions, and per-stay physiological averages between the training and test cohorts (Appendix A.5.2).

  3. Component ablations under controlled missingness. We extended the ablations to four datasets. RosFormer achieved lower MSE and MAE than all four ablated variants at each test-input removal rate of 30%, 50%, and 70% (Section 5.2 and Appendix A.5.1).

  4. RosFormer's integration mechanism. To assess how RosFormer's five components work together, we tested temporal anchors and compactness separately and jointly. Adding both to the SDEFormer-LN baseline with the VIMTS patch encoder lowered both MSE and MAE on all four datasets under every evaluated missingness condition (Section 5.2 and Appendix A.5.1).

Thank you again for your time and thoughtful feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.