acceptodds
Under review as a conference paper at ICLR 2027

RideBench: A Large-Scale Exogenous-Aware Benchmark for Ride-Hailing Time Series Forecasting

Abstract

We release Ride-Hailing, a large-scale ride-hailing time series dataset synthesized from marketplace data across 200 spatial areas. Ride-Hailing spans four consecutive years at half-hourly granularity and covers three representative exogenous scenarios: weather disturbance, holiday effect, and large-scale event impact. Built upon Ride-Hailing, we introduce RideBench, a comprehensive benchmark for exogenous-aware ride-hailing forecasting, covering both regular week-ahead forecasting and long-horizon 8-week-ahead forecasting with up to 2,688 prediction steps. RideBench evaluates over 30 representative forecasting methods, including endogenous-only models, exogenous-aware models, and time series foundation models. Our results show that future-known exogenous variables provide clear benefits in regular week-ahead forecasting, especially under weather, holiday, and large-scale event (e.g., major sporting events and concerts) scenarios. However, current exogenous-aware models still struggle to fully capture disturbance-induced pattern changes under complex external contexts. For long-horizon forecasting, existing models cannot simultaneously achieve low pointwise errors, accurate macro-level trends, and reliable near-term forecasts. These findings reveal a clear mismatch between existing forecasting models and real-world ride-hailing requirements, highlighting the need for models that can better exploit future-known exogenous information, scale across heterogeneous areas, and support long-horizon planning.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Time Series Forecasting, Time Series Analysis, Machine Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_qb7pOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Ride-Hailing, a dataset consisting of four years of ride-hailing marketplace observations across 200 areas at half-hourly resolution. It contains nine anonymized targets and weather, holiday, and event covariates. RideBench evaluates 32 methods on one-week (short-term) and eight-week (long-term) forecasting. The paper also combines overall and scenario-specific errors in the benchmark and introduces daily directional accuracy, daily correlation to capture long-term demand evolution. Additionally, they also use first-week accuracy within the longer horizon evaluations. Further analyses examine covariate access, training-area count, historical context, and training objectives.

Critical strengths
  1. The released domain-specific dataset contains four years of data across 200 areas, nine targets, and multiple exogenous categories. This offer a substantial resource for studying heterogeneous marketplace forecasting. The evaluation suite combines novel ideas of evaluating disturbance periods and combining eight-week forecasts with near-term and temporal-aggregation metrics. This could expose behavior obscured by aggregate error alone. The proposed evaluation design is a valuable part of the benchmark contribution, alongside the dataset itself.

  2. The 32 methods span seasonal baselines, channel-independent and multivariate models, exogenous-aware methods, and foundation models which is a broad empirical coverage. The training/testing splits and training-only normalization are appropriately used. Figure 5's comparisons with and without covariates and Figure 6's fixed-target-area scaling study is especially useful as it provides evidence about external information usage and pooled training, beyond simply reporting a leaderboard.

Critical weaknesses
  1. The central dataset is however not sufficiently specified and validated. Section 3.1 describes synthesis, target transformations, weather aggregation, and event-impact encoding. However there is not much explaination about these procedures, and there is no quantitative checking about which original relationships are preserved or removed. The Table 4 is useful as it gives target identifiers and numeric types, but there is no coarse semantic descriptions or characterization of relationships among the nine targets. Figure 3 illustrates plausible patterns, yet leaves the weather-level construction, gain calculations, and heterogeneity representations insufficiently defined. For comparison, LargeST (NeurIPS 2023) documents source and selection procedures and explicitly describes its regional and temporal analyses [1]. Comparable, documentation would make RideBench's content more interpretable and scientific. Additionally, it is not clear what the future weather and event-impact labels signify. Clarifying these points would help strengthen the case about the real forecasting problem this paper claims to represent.

  2. The long-horizon motivation and conclusions need further discussion. ProbTS (NeurIPS 2024) already evaluates point and distributional forecasts across short and long horizons [2], and TIME (ICML 2026) includes MASE/CRPS evaluation and motivates tasks through operational requirements [3]. The relevant distinction may be along the lines of the temporal behavior RideBench evaluates, rather than a general absence of long-horizon or probabilistic benchmarking. This needs further clarification. Another concern is that daily averaging retains weekday–weekend seasonality, so the introduced daily Acc./Corr. metrics do not isolate broad trend. Weekly or seasonally adjusted evaluation would help test that interpretation. I also am not convinced by the argument of "unavoidable trade-off" arising from different winners from each metric. TiDE leads both daily metrics and both full-horizon and first-week RMSE in Table 3.

References

[1] Liu et al. LargeST: A Benchmark Dataset for Large-Scale Traffic Forecasting. NeurIPS 2023, Datasets and Benchmarks. Published paper.

[2] Zhang et al. ProbTS: Benchmarking Point and Distributional Forecasting across Diverse Prediction Horizons. NeurIPS 2024, Datasets and Benchmarks. Proceedings.

[3] Qiao et al. It's TIME: Towards the Next Generation of Time Series Forecasting Benchmarks. ICML 2026. Proceedings.

Questions for authors
  1. The paper would benefit from stating what precisely was synthesized in the dataset and what is being transformed. Also what evidence supports preservation of forecasting-relevant structure? Can you provide descriptions of the nine targets and characterize their dependencies? Since multivariate models are already evaluated, per-target results and cross-target relationship analysis would clarify which dependencies are observed and whether they benefit forecasting.

  2. How are event-impact (scores and categories) determined? This can help explicitly distinguish an operational forecast from forecasting conditional on oracle covariates.

  3. In Figure 3, how are weather levels defined, which targets and normalization produce the plotted values, and how are the non-holiday baseline and percentage annotations computed? What does each heterogeneity point represent, and what transformation produces its x/y coordinates? Please also specify the benchmark's metric aggregation across targets, areas, and overlapping windows, and the categorical adapter used for each model.

  4. What planning decision motivates eight weeks of half-hourly output and does the claimed trend difficulty persist under weekly aggregation or after removing weekly seasonality? A focused analysis of existing predictions, with uncertainty accounting for overlapping origins, would help establish this.

Additionally, a separate fine-tuning study of the foundation models, reporting adaptation data and compute alongside zero-shot performance, would be informative.

11 comments
AuthorsOpenReview ↗

Rebuttal Part 1

Thank you for your very detailed and thoughtful review. Your assessment of both the strengths and weaknesses has been valuable in helping us examine the work more carefully, and we welcome the opportunity to discuss these points in depth.

S1. The released domain-specific dataset contains four years of data across 200 areas, nine targets, and multiple exogenous categories. This offer a substantial resource for studying heterogeneous marketplace forecasting. The evaluation suite combines novel ideas of evaluating disturbance periods and combining eight-week forecasts with near-term and temporal-aggregation metrics. This could expose behavior obscured by aggregate error alone. The proposed evaluation design is a valuable part of the benchmark contribution, alongside the dataset itself.

S2. The 32 methods span seasonal baselines, channel-independent and multivariate models, exogenous-aware methods, and foundation models which is a broad empirical coverage. The training/testing splits and training-only normalization are appropriately used. Figure 5's comparisons with and without covariates and Figure 6's fixed-target-area scaling study is especially useful as it provides evidence about external information usage and pooled training, beyond simply reporting a leaderboard.

Thank you for recognizing these contributions. We provide a large-scale dataset derived from real industrial operations together with a comprehensive benchmark. The forecasting challenges motivating this work arise directly from our production needs and also offer valuable research problems for the academic community.

(1) Dataset. We understand the importance of high-quality datasets. We therefore chose to release a dataset for the ride-hailing setting, where market-driven demand brings together multi-scale periodicity, diverse covariate effects, and heterogeneity across areas. These are challenges that forecasting methods must address in real applications. Releasing this dataset also requires protecting sensitive information. We therefore need to anonymize and transform the data to balance research value with sensitive-information protection. The original submission described the source and processing too briefly. In the revision, we will expand this description and provide target-relationship analyses and quantitative comparisons between the original and processed data, as detailed in the responses below.

(2) Benchmark. We also understand the importance of a well-specified benchmark and have incorporated operational considerations into its design. For example, business evaluation requires errors to reflect the magnitude of deviations on the original scale. We therefore use training-only normalization and inverse-transform predictions before calculating test errors, so MAE and RMSE are expressed in the units of the released targets. We also include target-wise WMAPE to assess errors relative to each target's scale. Beyond the metrics, joint modeling across heterogeneous areas and the choice of exogenous covariates reflect the spatial diversity and external influences encountered in production. Similarly, evaluating first-week accuracy together with daily trajectory consistency over eight weeks reflects the need to support both near-term scheduling and advance planning.

Thank you again for recognizing the value of this work. In the responses below, we explain the dataset construction and evaluation choices in greater detail, within the constraints of sensitive-information protection, and provide additional evidence to address your concerns. We hope these clarifications and results help resolve the questions raised in your review.

AuthorsOpenReview ↗

Rebuttal Part 2

W1. The central dataset is however not sufficiently specified and validated. ... Clarifying these points would help strengthen the case about the real forecasting problem this paper claims to represent.

W2. The long-horizon motivation and conclusions need further discussion. ... This needs further clarification. Another concern is that daily averaging retains weekday–weekend seasonality, so the introduced daily Acc./Corr. metrics do not isolate broad trend. Weekly or seasonally adjusted evaluation would help test that interpretation. I also am not convinced by the argument of "unavoidable trade-off" arising from different winners from each metric. TiDE leads both daily metrics and both full-horizon and first-week RMSE in Table 3.

Thank you for your detailed feedback. We understand that your concerns mainly involve two aspects: the specification and validation of the dataset, and the motivation and interpretation of the long-horizon benchmark. We briefly address these below and provide the detailed responses in Q1–Q5.

(1) Dataset specification and validation. The original description did not sufficiently explain the processing of real operational records, the meaning of the transformed targets, or the evidence for retaining forecasting-relevant properties. We therefore clarify why the dataset is described as synthesized, characterize the relationships among the nine released targets, and provide comparisons between the original and processed data. We also explain the Figure 3 calculations and distinguish the benchmark's weather and activity annotations from the inputs used in production. These points are addressed in Q1–Q3.

(2) Benchmark motivation and interpretation. The eight-week, half-hourly task comes from the need to support both advance planning and near-term scheduling. Accordingly, its evaluation combines full-horizon errors, daily demand changes, and first-week accuracy. The daily metrics capture weekday–weekend and holiday-stage changes that matter for these decisions. Our distinction from ProbTS and TIME lies in combining these requirements with heterogeneous multi-area and exogenous-aware ride-hailing evaluation; we will clarify this positioning in the related work. Regarding the “unavoidable trade-off,” our intended point is that choosing among the evaluated models still requires balancing overall accuracy, first-week accuracy, and day-level trend quality. TiDE's strong RMSE results are valuable, but the operational requirement involves all three dimensions and the complementary metrics within each dimension. For example, the original Table 3 reports:

ModelOverall: WMAPE / MAE / RMSEDaily Acc. / Corr.First week: WMAPE / MAE / RMSE
ModernTCN0.1262 / 4.816 / 14.340.6089 / 0.32730.1193 / 4.587 / 14.28
TiDE0.1328 / 5.069 / 14.320.6137 / 0.37150.1194 / 4.582 / 13.99
Chronos-2 (zero-shot)0.1374 / 5.270 / 15.070.5905 / 0.17020.1140 / 4.382 / 14.27

For operational evaluation, we place particular emphasis on WMAPE, which is listed first in our error tables. It normalizes each target's accumulated absolute error by its accumulated observed magnitude and then averages across targets, allowing us to assess relative error across targets of different scales. MAE measures absolute error on the released scale, while RMSE gives greater weight to large deviations. Thus, TiDE's lower RMSE shows an advantage under a large-error-sensitive criterion, while its overall WMAPE of 0.1328 remains above ModernTCN's 0.1262, and its first-week WMAPE of 0.1194 remains above zero-shot Chronos-2's 0.1140.

Our objective is a model that performs strongly across all three dimensions and their individual metrics. In the original comparison, TiDE's trend and RMSE strengths coexist with gaps in overall and near-term WMAPE. This is the practical model-selection trade-off we intended to emphasize. Our additional eight-week experiment in Q5 shows that fine-tuning Chronos-2 improves all three dimensions simultaneously and achieves a stronger balance, although the best results across individual metrics are still shared by different models. We will incorporate this finding and clarify the interpretation in the revised paper.

The operational motivation is detailed in Q4, and Q5 reports our additional foundation-model fine-tuning experiment.

We will incorporate these explanations and results into the revised manuscript.

AuthorsOpenReview ↗

Rebuttal Part 3

Q1. The paper would benefit from stating what precisely was synthesized in the dataset and what is being transformed. Also what evidence supports preservation of forecasting-relevant structure? Can you provide descriptions of the nine targets and characterize their dependencies? Since multivariate models are already evaluated, per-target results and cross-target relationship analysis would clarify which dependencies are observed and whether they benefit forecasting.

Thank you for your question. The nine targets originate from endogenous market indicators of a real ride-hailing platform and undergo anonymization, desensitization, aggregation, and other processing. After processing, the released targets should not be interpreted as the original business indicators with their original semantics. This is why we describe Ride-Hailing as a synthesized dataset. We are sorry that, to protect sensitive information, we do not disclose more detailed indicator definitions or the complete processing pipeline.

(i) Preservation of forecasting-relevant properties. The transformed data retain characteristics relevant to time-series forecasting research. Figure 3 illustrates multi-scale periodicity, responses to different exogenous factors, and heterogeneity. To further quantify their preservation, we apply the same aggregation procedures to the original and processed data and compare the resulting profiles using correlation (Corr.) and agreement in adjacent-change directions (Acc.).

Periodic profileCorr.Acc.
Daily0.99821.0000
Weekly0.99780.9970
Yearly0.92850.9423

The daily and weekly profile correlations exceed 0.997, and the yearly correlation is 0.9285. Thus, the multi-scale periodic profiles remain highly consistent after processing. The corresponding exogenous-response comparisons are:

Exogenous-response profileCorr.Acc.
Weather disturbance0.97621.0000
Holiday effect0.99540.9730
Event type 10.99790.9787
Event type 20.99200.9787

For weather, we compare response gains across disturbance levels. For holidays, we compare individual public holidays' response-gain curves and average the consistency scores. Correlations across the four response profiles range from 0.9762 to 0.9979. In addition, the proportion of areas with decreasing holiday demand is similar before and after processing (6.50% afterward), retaining regional differences in response direction.

We also rank the 32 models by their average rank across evaluation metrics on each dataset and obtain a Spearman correlation of 0.8132 between the two rankings. Together, these comparisons support the preservation of the examined forecasting profiles and broadly consistent relative model performance.

(ii) Relationships among the nine targets. We further compute the Pearson correlation matrix of the released targets using all 10,512,000 training records, pooling the 200 areas:

Targetendo_1endo_2endo_3endo_4endo_5endo_6endo_7endo_8endo_9
endo_11.0000.5340.7460.8080.8410.9650.5850.8490.783
endo_20.5341.0000.4310.5070.7610.4360.4320.7890.390
endo_30.7460.4311.0000.7900.5960.7180.8650.6270.954
endo_40.8080.5070.7901.0000.6800.7750.7090.6990.793
endo_50.8410.7610.5960.6801.0000.7860.4930.9350.605
endo_60.9650.4360.7180.7750.7861.0000.5600.7540.776
endo_70.5850.4320.8650.7090.4930.5601.0000.5180.817
endo_80.8490.7890.6270.6990.9350.7540.5181.0000.621
endo_90.7830.3900.9540.7930.6050.7760.8170.6211.000

The off-diagonal coefficients range from 0.390 to 0.965. In particular, endo_1–endo_6, endo_3–endo_9, and endo_5–endo_8 have correlations of 0.965, 0.954, and 0.935. Thus, the observed correlations support close relationships among the released targets. We will add the correlation heatmap to the revised main text.

Reviewer_RAS8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RideBench introduces a ride-hailing forecasting dataset and evaluation protocol, rather than a new forecasting architecture. The anonymized and synthesized platform-derived data contain nine target variables across 200 regions, spanning four years at 30-minute resolution, with 14,025,600 region–time records. The benchmark includes one-week and eight-week forecasting, weather/holiday/activity subsets, and metrics for both numerical error and daily trajectory trends. It compares 32 methods, including seasonal baselines, deep forecasting models, exogenous-aware methods, and zero-shot foundation models. Results suggest that exogenous inputs help several models in one-week forecasting, while longer-horizon rankings depend on the metric and forecast segment. Same-model covariate ablations support the practical relevance of external information. The main unresolved questions concern what information is actually available at forecast time and what properties the data transformation preserves.

Critical strengths
  1. A practically motivated benchmark that exposes more than average forecasting error. Combining regional heterogeneity, exogenous disturbances, and eight-week forecasting makes the task useful for studying operational forecasting limitations. Reporting weather, holiday, and activity subsets helps expose failures that aggregate metrics can conceal. The different winners for full-horizon error, trend metrics, and first-week error are informative even without a new model.
  2. Broad comparisons with useful experimental controls. The 32-method evaluation includes simple seasonal baselines, exogenous-aware systems, and foundation models, alongside same-model covariate ablations and inference-cost analysis. For example, the reported TiDE ablation improves weather, holiday, and activity WMAPE by approximately 8.8%, 17.6%, and 4.9%. This gives a more credible basis for discussing covariate utility than cross-architecture rankings alone.
Critical weaknesses
  1. Forecast-time availability of the central exogenous inputs is unresolved. Sections 3.1 and 4.2 treat the weather impact factor as a future-known covariate without establishing whether it comes from a contemporaneously available forecast, realized future weather, or a transformation involving target outcomes. These alternatives define materially different tasks, especially over eight weeks. Scheduled activity occurrence and realized activity intensity also need to be distinguished. Realized weather would support an oracle-covariate benchmark, but not directly a deployable forecasting claim; target-derived factors could be more problematic. Without feature provenance and availability timestamps, it is not possible to determine what the reported gains demonstrate. This is a request for clarification, not a finding that leakage occurred.
  2. The benchmark's data fidelity and evaluation reproducibility are not yet sufficiently auditable. The synthesis and anonymization process is described as preserving temporal, exogenous, and regional structure, but the supplied assessment identifies no reproducible transformation specification or convincing preservation analysis (Section 3.1; Table 4). The nine anonymous targets also lack enough semantic or scale information to interpret performance. This matters because a benchmark may otherwise measure properties introduced by its generator. The evaluation adds uncertainty: the half-year test period has unspecified independent event coverage; encoding and future-covariate adaptations are not fully documented per model; and some Table 2/Figure 5 results differ, such as TimeXer's weather WMAPE of 0.1388 versus 0.1354. These issues weaken the central reusable-benchmark contribution, independently of whether individual models perform well.
Questions for authors
  1. Clarification of available information: For each weather and activity feature, what is its source, construction rule, publication time, and maximum usable lead time? Does any factor depend on future target values? Which results use forecast covariates versus realized future covariates?
  2. Assessment-changing evidence on data fidelity: What transformations create the released targets, and what aggregate evidence shows preservation of seasonality, tails, event responses, regional correlations, and model rankings? This can be reported by the data holder without disclosing sensitive raw records. Please also clarify the release, licensing, and maintenance arrangements.
  3. Clarification of evaluation support: How many independent holidays, activities, and extreme-weather episodes appear in the test set, and how much do the subsets overlap? Are the main ranking differences supported by date- or event-blocked uncertainty estimates?
  4. Clarification of reproducibility: Which model configurations account for the Table 2/Figure 5 discrepancies? Please specify each model's categorical encoding and future-input adaptation, and the order of aggregation across regions and variables. If MAE and WMAPE are computed globally on the same evaluation set, what motivates including both in an average rank when they induce the same ordering?
6 comments
AuthorsOpenReview ↗

Rebuttal Part 1

Thank you for your careful review and constructive suggestions.

W1. Forecast-time availability of the central exogenous inputs is unresolved. ... This is a request for clarification, not a finding that leakage occurred.

Thank you for your question. We are sorry that space limitations prevented us from explaining these issues in sufficient detail in the original manuscript. In fact, the current experiments use observed weather and activity-impact categories determined from retrospective demand responses, including within the eight-week prediction horizon. Below, we explain how these experimental inputs are constructed and used, followed by the information available in deployment.

(i) Input construction and the meaning of the reported gains. The three types of exogenous information have different sources and construction procedures:

InputSource and constructionInformation represented in the experiments
Weather disturbanceAggregating meteorological observationsRealized weather within the prediction window
Holiday effectCalendar dates for holidaysCalendar-based holiday indicators
Large-scale event impactAssigning one of two broad impact categories from the observed demand curve after each activity; setting the binary indicator's active interval from its announced dates or timesRetrospectively identified daytime-dominant or daytime-and-evening response categories

The weather factor is constructed from meteorological observations. Activity-category assignment, in contrast, uses retrospective demand responses. The activity features are two binary indicators; they do not provide a continuous realized demand-impact magnitude. Activities outside the two categories receive no activity label, while the complete target series remain in the dataset.

Historical weather observations are used during model development to learn weather–demand relationships. We therefore use the same observed-weather setting in benchmark evaluation to compare models' ability to capture these relationships separately from errors in a weather-forecast source. Accordingly, the reported gains measure the benefit of exogenous information conditional on observed weather and the released activity-impact annotations.

(ii) Information available in deployment. Deployment uses the latest available information at each prediction origin: calendar dates, announced activity plans, and professional weather forecasts. For activities, concerts and other large events are often announced months in advance, so their schedules can provide information for advance planning; the benchmark categories above summarize retrospectively observed response patterns. For weather, observations in the experimental setting are replaced by the forecasts available at that time.

Because weather forecasts and activity plans can change, the platform updates predictions on a rolling basis to incorporate the latest information. Near-term predictions support fine-grained scheduling, while the daily demand trajectory over eight weeks supports advance planning. This also motivates evaluating first-week accuracy together with longer-horizon daily trajectory consistency.

In summary, the benchmark assesses exogenous modeling under the specified observed-input conditions, and deployment uses the latest information available when each forecast is made. We will expand Sections 3.1 and 4.2 to document the feature sources, construction procedures, and forecast-time availability, and explicitly state the conditions under which the experimental gains are obtained.

AuthorsOpenReview ↗

Rebuttal Part 2

W2. The benchmark's data fidelity and evaluation reproducibility are not yet sufficiently auditable. ...

Thank you for your careful feedback. We have supplemented the data-fidelity comparisons and test-set coverage statistics, and have redrawn Figure 5 using the latest dataset version. Below, we address the dataset specification, evaluation documentation, and numerical inconsistency in turn.

(i) Dataset specification and fidelity. Ride-Hailing is derived from endogenous market indicators of a real ride-hailing platform. We anonymize area and variable identifiers, transform target values, aggregate weather observations, replace detailed activity descriptions with coarse impact categories, and apply other steps that cannot be disclosed for sensitive-information protection. Space limitations restricted the original description. We will expand Section 3.1 to explain these processing principles; the complete pipeline and indicator definitions will not be disclosed.

The nine targets are transformed anonymous endogenous market indicators. To characterize their relationships, we computed their training-set Pearson correlation matrix, whose off-diagonal entries range from 0.390 to 0.965. Forecasting errors are calculated after inverse normalization, so MAE and RMSE are expressed on the scale of the released targets.

To examine whether the patterns motivating the benchmark survive processing, we also applied the same Figure 3 aggregation procedures to the original and processed data. The daily, weekly, and yearly profile correlations are 0.9982, 0.9978, and 0.9285; the weather, holiday, and two activity-response correlations are 0.9762, 0.9954, 0.9979, and 0.9920. Moreover, the Spearman correlation between the 32 models' rankings, based on their average rank across evaluation metrics on each dataset, is 0.8132. Thus, the evidence covers both the forecasting patterns and the relative model performance that the benchmark is intended to study. We provide the detailed before-and-after comparisons in our response to Q2.

(ii) Evaluation documentation. We have computed the scenario coverage in the test set. Weather, holiday, and activity annotations cover 270,681, 201,600, and 31,728 unique area–time points, respectively, corresponding to 15.324%, 11.413%, and 1.796% of the test set. These are coverage counts; the event-segment definitions and scenario intersections are reported separately in Q3.

For input adaptation, every exogenous-aware model was evaluated with ordinal, one-hot, and learnable categorical encodings, together with a no-exogenous setting. Each setting used five learning rates, and we report the configuration with the lowest average rank across evaluation metrics. We will add a per-model appendix table specifying the available inputs, selected adapter and selection criterion, future-input interface adaptation, and training regime. We also explain the metric aggregation and treatment of overlapping prediction windows in Q4.

(iii) Table 2 and Figure 5. Thank you for identifying this inconsistency. Figure 5 showed results from the initial dataset version. We subsequently made minor adjustments to the processed dataset and updated Table 2 using the latest version, but overlooked updating Figure 5. This caused the numerical differences you observed. We have now redrawn Figure 5 using the latest results in the revised manuscript, making it consistent with Table 2.

Both the earlier and updated results support the same two findings: the supplied exogenous information improves forecasting performance, and the gains vary across models and scenarios because models differ in how effectively they use that information. Therefore, correcting Figure 5 leaves these findings unchanged.

In summary, we have corrected the figure and supplemented the validation and coverage evidence. We will incorporate these results, the expanded dataset description, and the model configurations into the revised paper.

AuthorsOpenReview ↗

Rebuttal Part 3

Q1. For each weather and activity feature, what is its source, construction rule, publication time, and maximum usable lead time? Does any factor depend on future target values? Which results use forecast covariates versus realized future covariates?

Thank you for your question. We clarify the sources, construction, and availability of the weather and activity features below, and then specify which inputs are used in the reported experiments.

(i) Weather features. During model development, training, and benchmark evaluation, we use observed weather values, including those within the prediction window. At deployment, we replace these values with the latest available professional weather forecasts to use the most accurate information available at each prediction origin.

Specifically, the benchmark weather factor is obtained by aggregating meteorological observations; it is not derived from demand-target values. Using observations allows us to learn and assess weather–demand relationships separately from weather-forecast errors. Deployment then uses the forecasts available at that time, with the usable lead time determined by the forecast product.

(ii) Activity features. The two binary activity indicators combine announced activity schedules with retrospective impact-category assignment. Specifically, their active intervals follow the publicly announced activity dates or times. We then determine each activity's category from its observed demand curve, distinguishing daytime-dominant and daytime-and-evening responses. Category assignment therefore uses demand information from the activity period.

Accordingly, the availability of the schedule and the availability of the impact category need to be distinguished. Concerts and other major activities can be announced months in advance, so their schedules provide advance information once published; the usable lead time depends on the announcement date and subsequent revisions. The experimental impact categories, however, are assigned after observing the demand response. These categories are retrospective annotations supplied to the forecasting model in the benchmark.

In summary, both the one-week and eight-week experiments use realized weather and the released retrospective activity-impact annotations. Thus, their gains describe forecasting conditional on these inputs. Deployment uses the weather forecasts and announced activity plans available at each prediction origin. We will document these construction rules and availability conditions in Sections 3.1 and 4.2, and explicitly identify the inputs used in the experiments.

Reviewer_7ECzOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Ride-Hailing, a large-scale synthesized time-series dataset derived from ride-hailing marketplace data, and RideBench, a benchmark for exogenous-aware ride-hailing forecasting. The dataset contains four years of half-hourly observations from 200 spatial areas, with nine endogenous variables and external factors representing weather, holidays, and large-scale events. RideBench evaluates more than 30 forecasting methods under week-ahead and eight-week-ahead forecasting settings. The experiments highlight the benefits of exogenous information for regular forecasting and reveal limitations of existing methods in handling external disturbances and maintaining reliable long-horizon predictions.

Critical strengths

S1. The benchmark targets exogenous-aware forecasting with both continuous and categorical, future-known variables and includes realistic, scenario-based evaluations.

S2. The four-year dataset and the combination of regular and eight-week-ahead forecasting tasks provide a potentially useful resource for the research community.

S3. The evaluation is broad, covering more than 30 forecasting methods, including conventional baselines, recent deep learning models, exogenous-aware approaches, and time-series foundation models.

Critical weaknesses

W1. The major concern is what exactly is treated as “future-known” in the 8-week-ahead setting. The benchmark treats weather disturbances, holidays, and large-scale events as future-known inputs, but weather conditions and unexpected disruptions should not be “future-known” in practice. It is suggested to clarify the exogenous feature set and assumptions.

W2. The benchmark relies on a single synthesized marketplace dataset, which is not sufficient to demonstrate its generalization ability to other cities and ride-hailing platforms.

W3. Please provide more details on the synthesis pipeline and the transformations applied to protect privacy. Section 3.1 provides limited information about how the original observations are transformed, how the exogenous variables are constructed, and which statistical or dependency structures are preserved.

W4. It is great that the work covers a large number of baselines. However, different models receive different forms of input adaptation. It is suggested to separate comparison groups according to the information required as input and the training regime.

W5. More baselines that consider exogenous information could be included and compared, such as the work MVSTD (Knowledge-Based Systems, 2025). It directly addresses ride-hailing demand prediction under weather and holiday disturbances. Although it aims at a different task, its model design could be adapted to do the forecasting task.

Questions for authors

Please refer to the weakness

6 comments
AuthorsOpenReview ↗

Rebuttal Part 1

Thank you for recognizing the value of RideBench and for your constructive suggestions.

W1. The major concern is what exactly is treated as “future-known” in the 8-week-ahead setting. The benchmark treats weather disturbances, holidays, and large-scale events as future-known inputs, but weather conditions and unexpected disruptions should not be “future-known” in practice. It is suggested to clarify the exogenous feature set and assumptions.

Thank you for your question. We use observed weather in the experiments, including the eight-week evaluation. We distinguish model development and training from deployment as follows.

(i) Observed values for model development. Model development and training use historical weather observations. This allows the model to learn how weather changes ride-hailing demand from accurate weather inputs. We use the same setting for benchmark evaluation to examine how well different models capture these responses. Using observations separates errors in modeling weather–demand relationships from errors in weather forecasts, making the comparison of exogenous modeling capabilities easier to interpret. The reported results are therefore conditional on observed weather, including over the eight-week horizon. For activities, the benchmark assigns the two impact categories from demand curves observed after each activity, while the binary indicators follow the announced activity dates or times. The experiments therefore also use retrospective activity-impact annotations.

(ii) The latest and most accurate information available in deployment. Holiday dates are known from calendars. Concerts, exhibitions, and major sporting events are usually planned months in advance, so their announced schedules can also be available. For weather, the deployed model uses the latest available professional forecasts, which provide the best available estimate of future conditions at the prediction origin. Their accuracy generally decreases with lead time. Our operational requirements therefore place different emphasis on different parts of the eight-week output: first-week accuracy supports fine-grained scheduling using more reliable near-term information, while the daily demand trajectory over the full eight weeks supports advance planning, where holiday effects and scheduled activities play an important role. This motivates jointly evaluating first-week accuracy and longer-horizon daily trajectory consistency. Forecasts are updated on a rolling basis as new weather forecasts and revised activity plans become available.

We will clarify this production background and the distinction between benchmark inputs and information available in deployment in the revised paper.

W2. The benchmark relies on a single synthesized marketplace dataset, which is not sufficient to demonstrate its generalization ability to other cities and ride-hailing platforms.

Thank you for your comment. We are sorry that we did not sufficiently explain an important point in the submission: the 200 areas come from many different cities, with specific locations withheld to preserve anonymity. These areas cover diverse climatic conditions and local living and travel patterns. Multi-city diversity was explicitly considered when constructing the dataset.

(i) Multi-city coverage and shared modeling. Our objective is to train a shared forecasting model that learns common temporal and exogenous patterns across cities while accounting for area-specific dynamics. Thus, Ride-Hailing already covers multiple cities and heterogeneous local conditions within the same platform. Figure 3 illustrates this heterogeneity, while Figure 6 examines the benefit of adding training data from other areas when forecasting a fixed set of target areas.

(ii) Relevance to other platforms. RideBench primarily studies forecasting across heterogeneous areas and cities; direct transfer between ride-hailing platforms is a separate evaluation setting. Nevertheless, the benchmark is organized around forecasting characteristics that arise across ride-hailing systems: multi-scale periodicity, weather disturbances, holiday effects, large-scale events, and variation across areas and time. These characteristics motivate both the dataset and the evaluation protocol. We therefore expect RideBench to be useful for developing forecasting methods for other ride-hailing systems as well, and its scenario-based and long-horizon evaluations can be applied to their data.

We will expand the dataset description and appendix to explain the multi-city coverage more clearly while preserving location anonymity, and clarify its distinction from direct cross-platform transfer.

AuthorsOpenReview ↗

Rebuttal Part 2

W3. Please provide more details on the synthesis pipeline and the transformations applied to protect privacy. Section 3.1 provides limited information about how the original observations are transformed, how the exogenous variables are constructed, and which statistical or dependency structures are preserved.

Thank you for your suggestion. In the original submission, privacy requirements and space limitations restricted the detail we could provide about the transformation pipeline. We are happy to explain the processing principles and our considerations further here.

Ride-Hailing is derived from endogenous market indicators of a real ride-hailing platform. The data undergo anonymization, desensitization, and aggregation before being made available for research. Processing includes anonymizing area and variable identifiers, transforming endogenous target values, aggregating weather disturbances, replacing raw activity descriptions with coarse-grained event-impact categories, and other steps that cannot be disclosed for sensitive-information protection. Regrettably, we therefore cannot provide the complete processing pipeline.

We can, however, provide evidence that the processed data retain characteristics relevant to forecasting research. Figure 3 shows multi-scale periodicity, diverse exogenous effects, and heterogeneity in the released data. To quantify their preservation, we additionally compare the original and processed data using the same aggregation procedures. We examine the resulting profiles through correlation (Corr.) and agreement in the directions of adjacent changes (Acc.), and compare relative model performance on the two datasets.

(i) Multi-scale periodicity. The daily, weekly, and yearly profiles show the following agreement:

Periodic profileCorr.Acc.
Daily0.99821.0000
Weekly0.99780.9970
Yearly0.92850.9423

Daily and weekly correlations exceed 0.997, while the yearly correlation is 0.9285. The corresponding directional agreement is also high. These comparisons show that the processed data retain the multi-scale periodic patterns present in the original records.

(ii) Exogenous responses. We compare the weather, holiday, and activity-response profiles before and after processing:

Exogenous-response profileCorr.Acc.
Weather disturbance0.97621.0000
Holiday effect0.99540.9730
Event type 10.99790.9787
Event type 20.99200.9787

For weather, we compare response gains across disturbance levels. For holidays, we compare the response-gain curves for individual public holidays and average the consistency scores. Across weather, holidays, and the two activity types, correlations range from 0.9762 to 0.9979. Thus, the external-response profiles remain highly consistent after processing.

(iii) Heterogeneity in regional responses. The proportion of areas with decreasing demand during holidays is similar before and after processing (6.50% in the processed data). The processed data therefore retain the phenomenon that the same holiday can increase demand in some areas and decrease it in others.

(iv) Relative model performance. We also ranked the 32 models by their average rank across evaluation metrics on each dataset and computed the Spearman correlation between the two rankings, obtaining 0.8132. This checks the consequence of processing for the benchmark's central use: comparing forecasting methods. The positive rank association shows that the overall model ordering remains broadly consistent across the original and processed data.

Together, these comparisons provide quantitative evidence that the processed dataset retains the periodic and exogenous-response patterns motivating RideBench, regional differences in holiday-response direction, and broadly consistent model rankings. We will expand Section 3.1 to explain the processing principles and include these validation results.

AuthorsOpenReview ↗

Rebuttal Part 3

W4. It is great that the work covers a large number of baselines. However, different models receive different forms of input adaptation. It is suggested to separate comparison groups according to the information required as input and the training regime.

Thank you for your suggestion. In fact, we have already grouped the methods according to their input information and training regime.

(i) Existing comparison groups. Channel-independent models predict each target from its own history, while multivariate models additionally use the histories of other endogenous targets. Exogenous-aware models further use external covariates. These groups distinguish the information available when making a prediction. The foundation models in the original comparison are evaluated zero-shot and are therefore reported separately from models trained on RideBench. Within this group, Chronos-2 supports multivariate inputs and future covariates, whereas the other evaluated foundation models use historical endogenous inputs. The grouping is summarized below.

MethodsForecasting information usedTraining regime
Seasonal statistical baselinesHistorical endogenous seriesNo task-specific parameter training
Channel-independent modelsEach target's own endogenous historyTrained on RideBench
Multivariate endogenous modelsMultiple endogenous historiesTrained on RideBench
Exogenous-aware modelsEndogenous history and exogenous inputs, including future covariatesTrained on RideBench
Chronos-2Multivariate endogenous history and future exogenous covariatesPretrained; evaluated zero-shot
Other evaluated foundation modelsHistorical endogenous seriesPretrained; evaluated zero-shot

(ii) Categorical input adaptation. If your concern refers to the categorical encodings, every exogenous-aware model was evaluated with the same three adapters: ordinal encoding, one-hot encoding, and learnable categorical embeddings. Together with the no-exogenous setting, this gives four input settings, each evaluated with five learning rates, for 20 configurations per model. We report the configuration with the lowest arithmetic mean rank across the evaluation metrics. The adapters are therefore alternative configurations within the exogenous-aware group, selected through the same search procedure for every model. The reported week-ahead results use the following adapters.

ModelSelected adapter for week-ahead forecasting
TiDEOne-hot
CrossLinearOne-hot
XLinearLearnable categorical embedding
CATSOne-hot
DAGOne-hot
TimeXerOne-hot

In summary, we will add a per-model configuration table to the appendix listing the historical and future inputs, continuous and categorical covariates, selected encoding and selection criterion, future-input interface adaptation, and training regime.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.