- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
ExoRAF is a plug-in head for an endogenous-only forecaster. Discrete exogenous variables (holidays, events) and continuous ones (weather) are encoded into two queries, each refined by attention over a learnable key-value memory. The two are fused and used to attend over a second memory of M residual prototypes, and the weighted prototype, scaled by a step-wise and a channel-wise gate, is added to the backbone forecast. The main benchmark is a new Ride-Hailing dataset (200 areas, 30-minute steps, four years) from an anonymous RideBench repository, plus four EventTSF datasets. Adding ExoRAF to endogenous-only backbones lowers overall WMAPE on Ride-Hailing by 4.5% on average (Table 2).
Critical strengths
- The goal is practical: add exogenous information to an existing forecaster without changing its architecture.
- The module is tested on many backbones (channel-independent, multivariate, exogenous-aware and three foundation models) and in several settings: regular and long-horizon forecasting, unseen areas, cross-backbone transfer, and missing future covariates.
- Code is released, and the paper reports where the module helps little, e.g. large-scale events and exogenous-aware backbones (Table 8).
Critical weaknesses
-
Nothing is retrieved, and the key baseline is missing. Section 3 never describes a step that writes historical residuals into the memory. The query memories are "randomly initialized and learnable", the residual memory holds learned keys and values with between 10 and 50 (Fig. 8a), and all of it is trained end to end with the forecasting loss. This is a softmax mixture over a small set of learned prototypes, i.e. a small parametric output head, which A.2 nevertheless calls "our non-parametric design". The main tables then compare an endogenous-only backbone with the same backbone plus a head that sees every exogenous variable, including time of day, day of week and area id (A.2). Any head with those inputs should help. The paper needs the same backbone given the same exogenous variables in a plain way, as extra input channels or as a linear or MLP correction computed from the exogenous features. The "RRM ⇌ MLP" row in Table 3 keeps both query encoders and both query memories, so it does not answer this.
-
The paper's own tests show the model barely uses the exogenous signals it is built on. Adding 20% noise to weather changes WMAPE in the weather scenario by at most 0.0001 (Table 4). Without any future covariates, PatchTST still gets most of the holiday gain (0.1507 to 0.1368, versus 0.1346 with them; Table 9), and in long-horizon forecasting it does slightly better without them (0.1282 vs 0.1287; Table 10). This looks like a learned offset by time of day and area, not a model of weather or holidays.
-
It does not beat existing exogenous models, and it sometimes makes them worse. TiDE beats three of the five ExoRAF models overall and all five on holidays (Table 2). XLinear (Houston) and TimeXer (New York temperature) beat every ExoRAF row (Table 1). Adding ExoRAF to TiDE hurts it (0.1043 to 0.1050 overall, 0.1282 to 0.1330 on holidays; Table 8), and the transferred module hurts four of eight backbones under events (Fig. 5). "Consistently improves" in the abstract is false.
-
The evaluation cannot be checked or put in context. The main results come from one new dataset in an anonymous repository whose source, city, provider and license are not stated, and the paper does not define how test windows are assigned to the weather, holiday and event scenarios. The EPF electricity-price benchmarks used by TimeXer, the standard testbed for exogenous forecasting, are absent. All backbones run with dropout 0.5 and hidden size 256, with only the learning rate searched (A.2). The tables give no standard deviations (only Fig. 8b shows three seeds), while the overall gains on Ride-Hailing are 0.004 to 0.006 WMAPE. Appendix A.1 is the standard decomposition of squared error around a conditional mean and says nothing specific to this method; it assumes squared loss, while the model is trained with MAE (Fig. 2).
Questions for authors
See weaknesses.
Thanks for your advice.