- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper discusses the challenge in multivariate time-series forecasting, where the authors are distinguishing observed historical irregularities (such as transient surges or abrupt shifts) that carry predictive value for future horizons from those that act as transient noise. They propose NEIN-TSF (Not Every Irregularity is Noise in Time-Series Forecasting:
-
the first component is FDRP (Forecast-Damage Ranking Pretraining), the idea is to train a forecasting backbone to capture predictive consequences rather than raw input-space deviations;
-
the second component is Supported Predictive Evidence (SPE) With the pretrained backbone frozen to produce base forecast , SPE constructs a bank of historical candidates paired with support scores .
-
The third component is Utility-Controlled Evidence Extrapolation (HAEU + CFUS), Horizon-Adaptive Evidence Utility (HAEU) fits position-wise scalar multipliers via ridge-regularized least squares on calibration split .
Critical strengths
I found the Consequence-Aware Formulation is novel, in fact rather than relying on heuristic input-space anomaly thresholds or unconditional extrapolation, NEIN-TSF introduces forecast-consequence learning through teacher-derived damage ranking (FDRP) and couples it with support-bounded residual refinement (SPE). This provides explicit structural bounds on maximum correction magnitude () and prevents catastrophic forecast drift. Also they have provided empirical validation thorough, they used 15 diverse benchmarks, multi-seed variance bounds.
Critical weaknesses
I am bit concerned about the multi-stage offline training, basically they have proposed pipeline involves multiple sequential training, calibration, and selection steps: (i) pretraining a teacher forecaster , (ii) counterfactual generation and FDRP backbone pretraining via pairwise damage ranking, (iii) freezing and optimizing SPE evidence fusion networks, (iv) fitting HAEU ridge coefficients and (v) CFUS checkpoint selection on .
However what I noticed inference remains lightweight and single-pass in equation 2, the paper lacks quantitative reporting on total offline training time and FLOPs relative to end-to-end baseline models. Maybe reporting wall-clock training times and memory usage for each stage across representative datasets would clarify offline overhead.
Questions for authors
Could you provide a breakdown of wall-clock training times and memory footprints for each of the offline stages (Teacher training, FDRP pretraining, SPE optimization, HAEU calibration, CFUS selection) compared to training standard end-to-end models like iTransformer or PatchTST on large datasets (such as ECL and Traffic)?
For HAEU, one scalar is fit per horizon position , shared across all variables . Have you considered variable-specific or channel-grouped utility coefficients , especially for highly heterogeneous multivariate datasets like Weather or Traffic where variables exhibit vastly different noise levels and persistence lifetimes?
5 comments
Author Response to 8Soa (2/3): Weakness and Q1
We also provide a reference-protocol resource comparison on Electricity and Traffic following the efficiency setup of OLinear . To maximize transparency, we reproduce all 12 baseline rows from OLinear's published Table 41 (corresponding to Table 37 in an earlier version) and add our newly measured NEIN-TSF inference footprint. The baseline rows are published measurements from OLinear, not new re-runs on our GPU, so cross-study latency/memory numbers should not be interpreted as controlled same-hardware speedups.
Table R7a. E3-B Computational efficiency on Electricity (ECL).
| Model | Params (M) | FLOPs (G) | T.T. (ms/iter) | T.M. (GB) | I.T. (ms/iter) | I.M. (GB) |
|---|---|---|---|---|---|---|
| OLinear | 4.790 | 1.650 | 7.750 | 0.450 | 2.110 | 0.170 |
| OLinear-C | 4.580 | 1.650 | 6.940 | 0.420 | 2.090 | 0.170 |
| Leddam | 8.560 | 5.320 | 65.190 | 1.440 | 14.780 | 0.490 |
| CARD | 1.390 | 5.070 | 78.250 | 5.080 | 28.210 | 0.630 |
| Fredformer | 12.120 | 5.550 | 26.590 | 1.240 | 7.080 | 0.260 |
| iTransformer | 4.830 | 1.870 | 10.660 | 0.700 | 3.460 | 0.230 |
| TimeMixer++ | 1.190 | 64.020 | 106.060 | 1.020 | 34.100 | 0.360 |
| TimeMixer | 0.120 | 1.740 | 57.090 | 4.240 | 22.790 | 0.820 |
| DLinear | 0.020 | 0.010 | 1.370 | 0.040 | 0.200 | 0.020 |
| TimesNet | 300.580 | 293.980 | 509.170 | 5.820 | 157.130 | 1.410 |
| PatchTST | 3.760 | 12.480 | 65.830 | 3.020 | 20.520 | 0.930 |
| FreTS | 0.420 | 0.130 | 3.780 | 0.300 | 0.480 | 0.230 |
| NEIN-TSF | 0.028 | 0.305 | 17.435 | 0.178 | 3.199 | 0.234 |
Table R7b. E3-B Computational efficiency on Traffic.
| Model | Params (M) | FLOPs (G) | T.T. (s/iter) | T.M. (GB) | I.T. (ms/iter) | I.M. (GB) |
|---|---|---|---|---|---|---|
| OLinear | 6.170 | 4.910 | 0.020 | 1.010 | 5.710 | 0.430 |
| OLinear-C | 4.690 | 4.910 | 0.020 | 0.950 | 5.670 | 0.430 |
| Leddam | 8.560 | 15.240 | 0.190 | 3.740 | 45.680 | 1.250 |
| CARD | 0.980 | 10.470 | 0.180 | 9.500 | 61.210 | 1.690 |
| Fredformer | 11.090 | 14.780 | 0.100 | 5.760 | 31.600 | 1.840 |
| iTransformer | 4.830 | 6.450 | 0.040 | 2.950 | 13.450 | 1.270 |
| TimeMixer++ | 4.730 | 588.470 | OOM | OOM | 1610.660 | 9.350 |
| TimeMixer | 0.120 | 4.660 | 0.170 | 11.330 | 67.210 | 2.200 |
| DLinear | 0.020 | 0.020 | 0.001 | 0.070 | 0.200 | 0.040 |
| TimesNet | 301.690 | 288.720 | 0.500 | 5.860 | 15.640 | 1.420 |
| PatchTST | 3.760 | 33.520 | 0.170 | 8.010 | 55.100 | 2.440 |
| FreTS | 0.420 | 0.360 | 0.010 | 0.780 | 3.480 | 0.590 |
| NEIN-TSF | 0.031 | 0.924 | 0.020 | 0.564 | 8.158 | 0.572 |
T.T. and T.M. denote training time and GPU memory; I.T. and I.M. denote inference time and GPU memory. Baseline results are taken from OLinear's published efficiency table; NEIN-TSF results are newly measured. For NEIN-TSF, T.T./T.M. refer to the profiled backbone fine-tuning iteration, while inference measurements cover the full prediction pipeline.
The results provide additional evidence of NEIN-TSF's compact parameterization and practical inference resource requirements under the reference protocol. Detailed offline costs of its five principal stages are reported separately in Table R6. Source: OLinear, published supplementary Table 41.
Author Response to 8Soa (3/3): Q2
Q2. Why share a horizon-wise coefficient across channels rather than fitting channel-specific coefficients?
We agree that channel heterogeneity makes this an important design choice. HAEU intentionally uses one coefficient α_h per forecast position shared across variables, controlling the number of calibration parameters under chronologically limited calibration data. We compared a global scalar, the proposed horizon-wise scalar, channel-grouped coefficients with 8 and 16 groups, and channel-specific α_(h,c). All variants use the same ridge coefficient 10^-4, and coefficient clipping to [0, 1.2].
Table R8 — E6: Weather, calibration granularity (three-seed means). Lower MSE/MAE is better; clip ratio is descriptive, not a metric to minimize independently.
| HAEU calibration | Cal. MSE | Forward MSE | Test MSE | Test MAE | Clip ratio |
|---|---|---|---|---|---|
| Global α | 0.255 | 0.246 | 0.235 | 0.258 | 0.333 |
| Horizon-wise α_h | 0.253 | 0.243 | 0.233 | 0.256 | 0.064 |
| Grouped-8 | 0.249 | 0.244 | 0.235 | 0.257 | 0.091 |
| Grouped-16 | 0.247 | 0.245 | 0.234 | 0.257 | 0.132 |
| Channel-specific α_h,c | 0.244 | 0.248 | 0.235 | 0.258 | 0.181 |
Channel-specific calibration fits the calibration set most closely but does not generalize as well to later periods: its calibration MSE is 0.244, compared with 0.253 for horizon-wise HAEU, while its forward-validation and test MSEs are 0.248 and 0.235, respectively. The proposed horizon-wise HAEU obtains the lowest forward-validation MSE (0.243) and test MSE/MAE (0.233 / 0.256). This suggests that sharing α_h offers a favorable flexibility–generalization trade-off under the tested Weather setting.
We thank the reviewer for these actionable requests, which prompted clearer quantification of the computation–performance trade-off and direct validation of the coefficient-sharing design.
Author Response to 8Soa (1/3): Weakness and Q1
We thank the reviewer for recognizing the consequence-aware formulation, the explicit bounds on residual correction, and the breadth of evaluation. We address the requested computational transparency and HAEU coefficient granularity with stage-level profiling (E3) and controlled coefficient-granularity comparisons (E6).
Weakness and Q1. What does the multi-stage offline procedure cost?
We agree that the practical value of improved forecasting accuracy must be considered alongside computational cost. The distinct stages of NEIN-TSF serve complementary purposes: teacher-based consequence supervision, FDRP representation learning, support-conditioned SPE correction, horizon-wise utility calibration, and chronological checkpoint selection. We therefore profiled the wall-clock time and peak GPU allocation of the five core operations, as requested.
Table R6a — E3-A: stage-wise offline profiling of NEIN-TSF. GPU peak is the maximum observed within each named stage (and the maximum across stages in the Total row), not the sum.
Additional Comparison with End-to-End Forecasting Baselines
To further address the reviewer's concern regarding computational overhead, we additionally reproduce iTransformer and PatchTST on Electricity (H=96) using an NVIDIA GeForce RTX 5080. We report their measured offline wall-clock time, peak GPU memory, model parameters, and inference latency.
Table R6b. Computational cost of reproduced end-to-end forecasting baselines on Electricity (H=96).
All measurements were obtained on an NVIDIA GeForce RTX 5080 using two recorded seed runs (2022 and 2023). The current baseline records contain two completed seed measurements (2022 and 2023); the reported offline times reflect the recorded experimental units. These results complement the stage-wise computational analysis of NEIN-TSF in Table R6.
The largest measured costs among these operations are FDRP pretraining and SPE evidence construction/optimization, whereas coefficient fitting and the final checkpoint-selection calculation themselves are lightweight. This decomposition clarifies where the reported framework's computational work is concentrated.