acceptodds
Under review as a conference paper at ICLR 2027

Latency-Aware Time-Series Forecasting with Knowledge Distillation

Abstract

Time-series forecasters are scored offline as if inference took no time. On a stream sampled every Δt, the forecast of a call that takes T is first used d = ⌈T/Δt⌉ ticks later, when its first d steps describe instants already observed. We formalize latency-aware forecasting: a configuration issues its own stale prefix plus the H points the consumer needs and is scored on the remaining usable window. This deployment error factorizes into a device-independent error curve and a device-dependent prefix, so candidates are trained once and re-scored per device. Its optimum can be interior, since larger models lower per-step error but push the usable window further out. On a 1 kHz ECG stream and one x86 core, the usable-window error selects an MLP student a quarter the size of the one that nominal-window accuracy picks. For a selected configuration, reweighting the loss toward the usable window adds no signal to per-step read-outs and little to shared encoders, so TRIAD changes the target instead. It invokes the teacher d steps after the student, so that the teacher's forecast covers exactly the usable window, which helps when the teacher's bias grows with lead time, and adds frequency and correlation terms. With a ModernTCN teacher, TRIAD lowers the error of the selected student by about 3% on ECG and 6% on ETTh2 replayed at 1 ms, in every latency setting. On a 1 kHz gait stream it gains 3 to 12% in two latency settings and loses 0.6% in the third. Removing its frequency or correlation term raises the error by at most 0.8%. Code is available at https://anonymous.4open.science/r/TRIAD-8B7D.

Primary area:
learning on time series and dynamical systems
Keywords:
time-series forecasting, inference latency, knowledge distillation, latency-aware evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_72uSOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes latency-aware time-series forecasting, where each model is evaluated on the forecast window that remains usable after its own inference latency. It also introduces TRIAD, which distills a student using a teacher shifted by the student latency, together with frequency and correlation losses. Experiments on ECG, gait, and ETTh2 show that latency-aware evaluation can change the preferred model size and that TRIAD can improve usable-window error.

Critical strengths
  1. The paper identifies a practically important mismatch between conventional offline forecasting evaluation and real-time deployment, and provides a simple usable-window formulation.

  2. The experiments clearly demonstrate that accounting for latency can substantially change which model capacity should be deployed.

Critical weaknesses
  1. Model configurations are selected using test errors rather than validation errors, making the headline model-selection results optimistic. Moreover, all reported results use only one random seed.

  2. Key parts of the proposed method are not sufficiently isolated experimentally. The train-once-and-rescore factorization is not actually tested, the usable-window teacher target is not directly compared with the nominal-window target, and the frequency/correlation terms contribute at most 0.8%.

Questions for authors
  1. Could the authors repeat model selection using validation error and report the resulting performance on an untouched test set over multiple random seeds?

  2. Could the authors directly compare distillation using the nominal-window and usable-window teacher targets, and experimentally validate the proposed train-once-and-rescore procedure across devices?

Reviewer_Jc1COpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper defines a latency-aware forecasting objective in which a model must predict its latency-induced stale prefix plus the future window needed after inference completes. The authors propose device-dependent model selection from per-step error curves and TRIAD, which distills a teacher evaluated at a later input time and adds frequency cross-variable correlation losses. Experiments use a ModernTCN teacher and DMLP students on ECG, gait, and ETTh2 replayed at millisecond intervals. The reported gains are modest on ECG and ETTh2 and mixed on gait.

Critical strengths

a) A well-motivated decision objective b) The target-shift analysis acknowledges an important limitation

Critical weaknesses

a) The empirical selection procedure does not validate the proposed deployment-selection method. In fact, configurations are selected using test errors, making the reported selections optimistic. Furthermore, the formal latency is batch-one latency (batch size = 1) at the issued horizon, satisfying a fixed-point condition, while Sect. 5.1 times batch 32 at H=96 before training model for H+d. Every device is retrained separately rather than using the proposed train-once Pmax curve, and the edge setting is an injected tenfold prefix rather than a measured device. Consequently, neither the cross-device factorization procedure nor the headline deployment-optimal size choices are adequately validated by the present protocol (Sects 3, 5 and 6).

b) The proposed distillation contributions are not isolated, and the stated loss does not fully describe the reported ablations. Both TRIAD and TimeDistill use the later teacher input, so their comparison does not establish the value of the target shift. Removing the frequency term changes error by at most 0.8%, and the correlation term by at most 0.1% with only one seed. More importantly, Tab. 2 and Sect. 6 say that removing both leaves a multi-scale distillation term, although Eq. 6 only lists supervised, frequency, and correlation losses. Most of the gain is attributed to that retained multi-scale term. These results therefore do not establish a substantial benefit attributable to the newly proposed ingredients; the actual objective must be reconciled with the mathematical description.

Questions for authors

a) What are the selected configurations and held-out errors when configuration selection is performed strictly on validation data? Please also distinguish this from merely reporting the test-best configuration

b) Do the central size-selection results persist when batch-one latency is remeasured at the actual issued horizon H+d and the fixed-point condition is checked? What resourse/scheduling setup realizes the assumed concurrent calls?

c) What exact loss, including all multi-scale terms and coefficients, produced Tabs. 1-2? In a matched, limited ablation, how do nominal versus shifted teacher targets compare under the same loss, and do the frequency/correlation increments persist across seeds?

Reviewer_qET8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper formulates a new setting, latency-aware time-series forecasting, in which inference latency is explicitly accounted for: when a model with lookback window LL predicts PP steps, the first dd steps are already outdated by the time the prediction is available, so only HH steps are usable (P=H+dP = H + d). For this setting, the authors propose TRIAD, a distillation method built on TimeDistill, together with a latency-aware objective for selecting among distilled student models. Students are trained offline, and selection is performed on a validation set. TRIAD achieves better or comparable results relative to the distillation baselines it is compared against.

Critical strengths
  1. Relevant problem formulation. Accounting for inference latency in fast-sampling forecasting applications is a practical and under-explored problem. The P=H+dP = H + d formulation makes the trade-off between model speed and usable horizon explicit, which is a useful framing for real-time deployment.

  2. TRIAD. The authors propose TRIAD, a novel distillation method that targets the usable prediction window. It achieves good results, lowering the selected student's error relative to the baseline in most settings.

Critical weaknesses
  1. Lack of comparison with standard forecasting methods. Selection is evaluated only among TRIAD-distilled students and against TimeDistill, with no comparison to other standard forecasting approaches. These include fast linear models such as DLinear and NLinear [1], which should have a short unusable window, and foundation models such as Chronos-2 [2]. The paper also does not show that existing state-of-the-art models are actually latency-limited to the point of being unusable in real-time applications. Without this evidence, the case for latency-aware selection is not established beyond the distillation setting.

  2. Limited empirical validation. Only three datasets are used, whereas standard forecasting benchmarks evaluate on many more (e.g., nine in [1]). No results across multiple random seeds are reported, which is standard practice in time-series forecasting and could be significant here because the results in Table 2 are close. In addition, the analysis of the gain from selection is limited. An analysis in the style of Figure 3 for all datasets would strengthen the results. Together, these issues limit confidence in the generality of the results.

[1] Zeng et al., Are Transformers Effective for Time Series Forecasting?, AAAI 2023.

[2] Ansari et al., Chronos-2: From Univariate to Universal Forecasting, 2025.

Questions for authors

Clarifications

  1. Equation 1. You divide by CC (the number of channels), which relates to the cost/Δt\Delta t rather than the prediction length. If every configuration is normalized by the same CC, is this term necessary, or can it be factored out?
  2. Student architectures. Figure 2 shows several student architectures, but Section 5 states that only DMLP students are used. Did you run experiments that select across different architectures?
  3. Long-horizon forecasting. Since P=H+dP = H + d, a slower model must predict a longer horizon. How does your approach compare to long-horizon forecasting methods?
  4. Online selection. Did you try updating the selection online during deployment, rather than fixing it on the validation set?

Suggestions

  1. Can you add an analysis showing that current state-of-the-art forecasters are latency-limited in real-time settings?
  2. Can you add to Table 1 a baseline that compares the best single student against latency-aware selection (as in Figure 3)? Can you also extend the Figure 3 analysis to all datasets?

Presentation

  1. Figure 1.
    • Can you mark dd and HH clearly, including on the x-axis of the left panel?
    • Should the indices there be Xs+1X_{s+1} to Xs+d+HX_{s+d+H}?
    • XX appears only in Figure 1 in the main paper.
  2. Figure 2. Can you annotate which part of the figure corresponds to which stage of the pipeline?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.