- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces an open dataset, along with a few benchmarks, for models, metrics, and problem formulation. The authors provide a detailed explanation of their tier-based pipeline, as well as some preliminary experimentation with (mostly vanilla) models. Hyperparameters are tuned with optuna (which method? BO?), and the results show that XGboost and non-learning baselines have the best performance in short-term/first-order delay propagation prediction, while GNNs are capable in more nuanced scenarios.
Critical strengths
This is an excellent dataset, and I agree that comparability and gold-standard datasets are absolutely fundamental to true research progress. This problem is still a niche, but I don’t think that serves as an argument to minimise this effort.
The authors try to make this a very thorough paper, covering weather, delays, and a 4-layered pipeline. I really liked the model-specific datasets (tabular, sequential, GNN, Graph-event), which help structure the field into distinct task types and methods. The results generally make sense and are coherent with the state of the art. The hyperparameter tuning is well argued, and the results are reasonably well discussed.
The paper also provides an extremely detailed (and long!) appendix with all details.
Critical weaknesses
- Although I’d still count it as a contribution, the dataset mentioned is already open, so the “value added” of the paper alone is questionable as a “dataset and benchmarks” paper.
- d^{last}{s,i} definition is confusing. First, d^{last}{s,i} is different from d_{s, i, n}, correct (n is also a last event prediction…)? t^s is the observation that maps to t^{sched}{s,i,1}, right (if they are equal, delay is zero; I mean you say that “it is the scheduled time of the next event”, which next event?)? In which circumstance can t^s-t^{sched}{s,i,1} be different from d_{s,i,−1}?
- The baseline models are quite vanilla. Did the authors check which of the existing models mentioned in the literature already provide code? If they are currently the state of the art, why not include them as well?
Questions for authors
Look at the weakness section.
4 comments
Dear authors,
Thank you so much for answering my comments.
Regarding W2, it is now very clear. Please make it clear in the paper (perhaps with a diagram).
W1. Yes, of course, providing data treatment for upcoming research is a contribution. And I didn't say either that this paper wouldn't contribute at all to the "datasets and benchmarks", but my point still stands that, for ICLR, it is not as strong an argument as if you were providing a new dataset W2. This also relates to W1. The "benchmark" argument for ICLR would be much stronger with stronger baselines.
I am still leaning toward keeping the score and am quite willing to hear further from the authors to convince me otherwise. Which other baselines could be added? Even though there are no public implementations, is there enough information in any such paper to build your own version?
Thank you for the follow-up. We will make the requested W2 clarifications in the paper. We understand the distinction you draw between this work and collecting a new source dataset. Regarding the benchmark, we believe our baselines are representative of the main approaches in train-delay research and are more substantial than their short model-family names suggest. Published methods provide enough detail to guide RIDE-specific implementations, as we did for the Graph-event, LSTM, and GNN baselines. Their original architectures cannot be added unchanged because the settings for which they were designed differ from RIDE’s. Below we explain how each baseline was selected and implemented.
Translation: Included as a simple reference method that carries the known delay forward, following similar baselines in [1, 2].
Graph-event: We draw on event-based delay propagation models [3, 4], but their simulation rules depend on the topology, event definitions, and operational information of their datasets. Those rules cannot be applied directly to RIDE, so we had to design and implement our own Graph-event model using RIDE’s topology and train-event records.
XGBoost and MLP: We retained the implementations from [5]. Both are standard methods whose existing prediction setups could be used for RIDE, with routine tuning rather than a new architecture.
LSTM: Prior LSTM studies show how train operations can be modeled as sequences [6–8], but their architectures reflect their particular inputs and prediction tasks. The closest is Sobrie et al. [6], who use an encoder–decoder across the Belgian network to predict five future signal passings from three observed passings. In RIDE, we predict the next 15 scheduled events of every active train from a sampled network snapshot. We extend their encoder–decoder design by giving each decoder step the known features of its future event and using attention pooling over the observed history, rather than repeating one encoded summary across steps. The input also requires change: on the individual lines studied in [7,8], trains ahead of a target form a natural sequence. Across RIDE’s full network, relevant trains can join from different routes along the target’s itinerary, so we summarize surrounding traffic as local features encoded by a separate static-context MLP. Sobrie et al. also use train and spatial features, but feed them into the past-signal sequence rather than a separate context branch. Our resulting LSTM builds on prior sequence models while adapting their inputs and decoder to RIDE’s snapshot setting and multi-event target.
Transformer: We retained the architecture introduced in [1] and used the implementation released with [5]. Its train-token design already fits RIDE’s task of predicting future event delays for multiple trains.
GNN: Prior GNNs show how graph structure can capture interactions in railway operations [9–11], but their architectures reflect the tasks and inputs they were designed for. SAGE-Het and NRI-GraphSAGE predict train delays after fixed time intervals, while Huang et al. represent train events as nodes to model delay propagation. In RIDE, we must predict delays at the next scheduled events of every active train, so we represent those future events as train-to-station edges and make predictions on them. The available inputs also shape the graph. For example, inferred routes let us estimate where trains are on the railway graph at prediction time and derive features for the links they traverse. Using these and other available features requires choices about graph structure, feature placement, and message passing, so we could not use the published architectures unchanged. We also tested GAT and GraphSAGE layers, which are used in the cited papers, on our graph. GINE performed best in these tests, so we retained it [12]. The resulting GNN is a substantial baseline grounded in prior train-delay models and designed for RIDE’s inputs and prediction task.
The labels LSTM, Transformer, and GNN identify broad model families, but do not convey the specific architectures evaluated here. The Transformer follows a published train-delay design, while we designed the encoder–decoder LSTM and heterogeneous GNN for RIDE’s inputs and prediction task, drawing on recent train-delay research. We will describe these designs more explicitly in Section 4.4 so their names do not suggest generic implementations.
References
[1] Arthaud et al., Transformers à Grande Vitesse: Massively Parallel Real-Time Predictions of Train Delay Propagation, 2024.
[2] Wang and Work, Data Driven Approaches for Passenger Train Delay Estimation, 2015.
[3] Goverde, A Delay Propagation Algorithm for Large-Scale Railway Traffic Networks, 2010.
[4] Wei et al., Modeling Cascade Dynamics of Railway Networks under Inclement Weather, 2015.
[5] Elliker et al., Simulation-Driven Railway Delay Prediction: An Imitation Learning Approach, 2026.
[6] Sobrie et al., Capturing Complexity over Space and Time via Deep Learning: An Application to Real-Time Delay Prediction in Railways, 2023.
[7] Gao et al., A Novel Deep Learning Model for Short-Term Train Delay Prediction, 2023.
[8] Huang et al., Modeling Train Operation as Sequences: A Study of Delay Prediction with Operation and Weather Data, 2020.
[9] Li et al., Railway Network Delay Evolution: A Heterogeneous Graph Neural Network Approach, 2024.
[10] Huang et al., Explainable Train Delay Propagation: A Graph Attention Network Approach, 2024.
[11] Li et al., Integrating Graph and Neural Relational Inference for Network-Wide Train Delay Prediction: An Equilibrium between Accuracy and Interpretability, 2026.
[12] Hu et al., Strategies for Pre-Training Graph Neural Networks, 2019.
Thank you for your review. We appreciate your emphasis on the importance of comparable datasets for research progress, and your positive assessment of RIDE’s model-specific datasets and detailed appendix.
W1
While we agree that the underlying railway and weather records are open, preparing them for train delay prediction is substantial work and a central contribution of RIDE. This includes cleaning and aligning train events, reconstructing the network topology, and integrating weather data. RIDE performs this work at nationwide scale and releases both a reusable relational dataset and model-ready datasets. Researchers can build on these releases instead of repeating the data preparation themselves. We believe that saving researchers this substantial effort constitutes significant added value.
W2
Yes, ds,i,n is the delay at the n-th future event, whereas ds,ilast is the last known delay, explained below. ts is the time at which the prediction is made for the snapshot s. For each train i, we distinguish events that have already been observed from events that have not yet been observed. The previous observed event has index −1, and the first future planned event has index 1. Thus, ds,i,−1 is the observed delay at the previous event, whereas ts,i,1sched is the scheduled time of the next event in the train's itinerary. With the same indexing, ts,i,−1sched would be the scheduled time of the previous observed event. To clarify how these quantities relate, consider how ds,ilast is computed. If the train kept its delay from the previous observed event, the next event would occur at ts,i,1sched+ds,i,−1. Subtracting this implied time from ts tells us whether it lies before the snapshot. If the implied time falls before the snapshot, it conflicts with the fact that the event has not yet been observed. We therefore increase the carried-forward delay by that gap, placing the implied event time at ts. This gives a more informative estimate of the current delay of the train: an event that is still unobserved after its scheduled time must be at least that late. This is expressed by ds,ilast=ds,i,−1+max(0,,ts−ts,i,1sched−ds,i,−1), which can be written more intuitively as:
\left\\{\begin{array}{ll} d_{s,i,-1} & \mbox{if }t_{s,i,1}^{\mathrm{sched}}+d_{s,i,-1}\geq t^s,\\\\ d_{s,i,-1}+\bigl[t^s-(t_{s,i,1}^{\mathrm{sched}}+d_{s,i,-1})\bigr] & \mbox{else.} \end{array}\right.$$ We will include these clarifications in the paper, and revise notation where needed. **W3** We checked whether public code was available for the prior methods discussed in the paper. The public implementations we found that fit our benchmark are already used in our MLP and Transformer models [1]. We could not find public implementations for the other cited methods. Additionally, published results use different datasets and prediction tasks, so they do not establish a clear state of the art. Our implementations represent the main model families used in the literature and provide a common starting point for evaluating future methods. **Reference** [1] Elliker et al. (2026), [Simulation-Driven Railway Delay Prediction: An Imitation Learning Approach](https://ojs.aaai.org/index.php/AAAI/article/download/39239/43200).