- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces REA-DG for irregularly sampled multivariate time-series classification and forecasting. At each observation step, a sample entity connects to currently observed channels and mediates their interactions. Learnable reference entities provide shared context through channel-specific edges. Fast and slow channel memories retain historical information, which is retrieved to initialize subsequent entity states.
The paper evaluates the method on four clinical classification settings and two forecasting datasets, FNSPID and CESNET. REA-DG achieves the highest reported mean performance among the listed baselines. Ablations on P19 examine the reference entities, graph message passing, and memory components.
Critical strengths
-
The architecture addresses relevant aspects of irregular observations. Observation-dependent connectivity, channel-specific memories, and historical retrieval provide a coherent approach to combining asynchronous measurements. Figure 2 makes the overall information flow reasonably clear.
-
The clinical results are promising, particularly on P19. In Table 1, REA-DG improves P19 AUPRC from the strongest listed baseline's 62.5 to 71.3, an increase of 8.8 percentage points. The improvement on MIMIC-III is also substantial, from 48.4 to 52.2. These results warrant further investigation.
-
The ablations establish that several components contribute to performance. In Table 3, removing graph message passing reduces P19 AUPRC from 71.37 to 60.26, while removing reference entities reduces it to 70.03. The reference-count experiment also shows that increasing the number of references does not consistently improve performance.
-
Several evaluation choices are explicitly documented. The appendix describes patient-disjoint clinical splits, training-only normalization statistics for clinical data, validation-based checkpoint selection, and repeated runs. It also discloses checkpoint averaging for forecasting.
Critical weaknesses
W1. The reference-entity mechanism is not specified precisely enough to reproduce
Equation (4) updates the reference state to , whereas Equation (5) constructs reference messages from . In contrast, the real-entity message explicitly uses the updated state . It is unclear whether this asymmetry is intentional or a notation error.
The computation of the attention weights is also missing. Nonnegativity and normalization do not establish whether these weights are learned static parameters or depend on the current sample, time step, channel state, or reference messages. These alternatives imply different mechanisms for providing shared context.
Furthermore, the paper specifies how the real entity is initialized at subsequent steps but does not clearly define whether channel-node and reference-entity states are reset from learned embeddings or carried forward across time. Shared trainable parameters should be distinguished explicitly from sample-specific intermediate states.
A complete algorithm describing initialization, within-layer updates, and state transfer across observation steps would resolve these ambiguities. Important configuration values are also missing, including the backbone dimension , graph depth , clinical learning rates, and the regularization parameters and .
W2. The provenance and comparability of the clinical baseline results require clarification
Several baseline means and standard deviations in Table 1 exactly match the corresponding entries in the original KEDGN paper. The row labeled "KEDGN" corresponds specifically to KEDGN-Name. Other variants in that paper perform better on some metrics. For example, KEDGN-ChatGPT reports P12 AUROC/AUPRC of 87.8/54.5, compared with 87.1/54.1 in the submitted table. Relative to that published variant, REA-DG's P12 improvements are therefore 0.1/0.9 percentage points [1].
This does not reverse the direction of the reported improvements, but it changes their magnitude on some tasks.
More importantly, the manuscript does not identify which results were reproduced and which were taken from previous publications. Appendix A.1 specifies particular fixed splits, but the correspondence between these splits and the reported baseline results is not established. Matching published numbers does not itself demonstrate an unfair comparison; however, the current description is insufficient to verify comparability.
The authors should identify result sources and model variants, and confirm consistency in patient splits, preprocessing, input variables, and metric implementations. A matched comparison with the strongest relevant clinical baseline would be particularly useful.
W3. The CESNET results do not isolate the contribution of the proposed backbone
Equation (16) presents a general forecasting decoder. Appendix A.4, however, states that the CESNET results use a dedicated time-basis query head with fixed multiscale frequencies, and that the original-head control is not included in the main results. Forecasting results also use parameter averaging across the three checkpoints with the lowest validation MSE.
These are legitimate modeling choices, but the resulting gains combine the effects of the backbone, decoder, and checkpoint-selection procedure. Section 4.2 largely attributes the improvements to the integration of current observations, historical information, and reference context.
On CESNET, MSE decreases from 0.8944 for TimeCHEAT to 0.8677 for REA-DG, approximately a 3% relative improvement. The original-head result is therefore important for interpreting the source of this gain.
A comparison between the original and time-basis heads on the same REA-DG backbone would directly address this concern. A simple time-feature-only control would also help establish how much predictive information the historical encoder contributes on this small dataset.
W4. The ablations support component usefulness more clearly than the proposed mechanistic explanations
The real-entity branch in Equations (4)–(6) aggregates messages from observed channels into a shared state and broadcasts contextual information back to channels. This is a reasonable inductive bias, but its advantages over comparable masked aggregation or attention-based feedback mechanisms are not established.
TimeCHEAT already uses channel–time bipartite graphs to represent observations. The distinctive contribution of REA-DG therefore needs to be established through its persistent entity state, historical retrieval, and reference-entity mechanism [2].
The w/o Graph experiment removes the complete message-passing module. Its performance drop demonstrates that this module is useful, but does not isolate the benefit of the proposed topology. The exact replacement computation and retained parameters are also unspecified.
Other interpretations similarly exceed the available controls:
- The claim that reference entities compensate for extreme sparsity is supported only by aggregate P19 results, without analysis across observation-density levels.
- Removing fast or slow memory may alter model capacity; it is unclear whether total state dimensionality is preserved.
- The conclusion attributes benefits to interval-dependent decay, but Table 3 does not include a control that retains fast memory while disabling decay.
- Figure 3 visualizes attention weights, but does not establish the effectiveness of the reference-message diversity regularizer.
A decay-disabled control and an observation-density analysis on the existing test set would provide focused evidence for central claims without requiring a substantially different experimental study.
W5. The CESNET evaluation provides limited evidence for broad forecasting generalization
Appendix A.3 reports only 11 entities, 23 training windows, 11 validation windows, and 12 test windows for CESNET. This can support a valid comparison, but the scope of the conclusion should reflect the small evaluation set.
The very small standard deviation across random seeds indicates stable optimization on this particular split. It does not quantify uncertainty across entities, time periods, or test-window composition. Numerous temporally correlated scalar targets also do not constitute an equivalent number of independent evaluation samples.
Table 4 evaluates overlapping cumulative portions of the same 168-hour forecasts. These results characterize performance within the existing forecast window, but do not provide independent replications.
Per-entity or per-window error differences, together with uncertainty estimates that account for within-entity dependence, would strengthen this evidence.
W6. The interval definition needs a clearer interpretation and forecasting boundary
Equation (8) uses
when both neighboring observations exist, where is the next observation time. This quantity describes local sampling spacing rather than simply the elapsed time since the previous observation.
Using such information is compatible with encoding a complete historical window, provided that both neighboring times lie within the available input history. However, if the intervals or are computed from the full sequence before separating history and forecast targets, the encoder could incorporate observation-pattern information from beyond the forecast origin.
Appendix A.3 states that future target values are excluded from model inputs, but does not establish the scope used to compute these temporal statistics. The authors should clarify this preprocessing order and explain why a two-sided interval is appropriate for decaying historical memory.
I do not regard the equation alone as evidence of leakage. The concern is the unspecified boundary between available history and future observation information.
Questions for authors
- Does Equation (5) use or ? How are computed, and do they depend on the sample or time step?
- How are channel-node and reference-entity states initialized and propagated across observation steps?
- Which entries in Table 1 are reproduced results, and which are quoted? Are their splits and preprocessing identical to those used for REA-DG?
- What is REA-DG's CESNET performance with the original prediction head? How much improvement comes from the time-basis head and checkpoint averaging?
- What computations replace the removed components in the ablations? Is memory dimensionality preserved, and what happens when decay alone is disabled?
- Are and computed strictly within the input history? How are observations at the forecasting boundary handled?