- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes latency-aware time-series forecasting, where each model is evaluated on the forecast window that remains usable after its own inference latency. It also introduces TRIAD, which distills a student using a teacher shifted by the student latency, together with frequency and correlation losses. Experiments on ECG, gait, and ETTh2 show that latency-aware evaluation can change the preferred model size and that TRIAD can improve usable-window error.
Critical strengths
-
The paper identifies a practically important mismatch between conventional offline forecasting evaluation and real-time deployment, and provides a simple usable-window formulation.
-
The experiments clearly demonstrate that accounting for latency can substantially change which model capacity should be deployed.
Critical weaknesses
-
Model configurations are selected using test errors rather than validation errors, making the headline model-selection results optimistic. Moreover, all reported results use only one random seed.
-
Key parts of the proposed method are not sufficiently isolated experimentally. The train-once-and-rescore factorization is not actually tested, the usable-window teacher target is not directly compared with the nominal-window target, and the frequency/correlation terms contribute at most 0.8%.
Questions for authors
-
Could the authors repeat model selection using validation error and report the resulting performance on an untouched test set over multiple random seeds?
-
Could the authors directly compare distillation using the nominal-window and usable-window teacher targets, and experimentally validate the proposed train-once-and-rescore procedure across devices?