- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Sufficient Patch Selector (SPS), a plug-in framework for long-term time series forecasting that reformulates patching as two stages: candidate generation and candidate selection. SPS constructs multi-scale patch candidates using dominant periods identified through spectral analysis, and learns to retain a compact subset under a fixed patch budget. It also introduce a posterior-preservation objective that minimizes the KL divergence between the selected-patch predictor and a full-history teacher.
Critical strengths
1.The paper provides a useful two-stage perspective that explicitly separates the construction of candidate patches from the selection of informative patches.
2.Meaningful empirical evaluation and useful controlled comparisons.
Critical weaknesses
1.The experiments do not directly compare the spectral generator against fixed-scale, uniformly sampled multi-scale, or random multi-scale generators under identical candidate and selection budgets.
2.A reduction in IPS demonstrates closer agreement with the chosen teacher, rather than directly establishing preservation of the true predictive information. The manuscript does not provide sufficient evidence that the IPS-based conclusions are insensitive to the teacher architecture or predictive uncertainty calibration.
3.The selector uses a global context vector derived from the entire historical input, and the selected representation is produced through mask-aware attention. This complicates the interpretation of the output as information retained solely within the selected patches.
Questions for authors
See critical weakness
6 comments
- Accuracy. The teacher is not an upper bound: SPSNet has lower MSE in 25/32 settings and lower MAE in 28/32. The teacher is more accurate on Traffic. Its average error is close to that of the iTransformer column of Table 1 (0.329 / 0.336), so it is a standard full-history model.
- Calibration. The teacher is over-dispersed. Its nominal 90% intervals cover 92.4–98.3% of the test targets, 94.8% on average. SPSNet's intervals cover 91.9% on average and are closer to 90% on all eight datasets, with slight under-coverage on ETTm1 and ETTm2 (88.8% and 88.9%). SPSNet also has the lower NLL on all eight datasets (0.646 against 0.722), while CRPS is tied (0.230 against 0.231).
- Caveat. The NLL gap reflects the variances more than the means: the teacher has the lower CRPS on five datasets, and on Traffic it has the lower MSE and still the higher NLL. We do not claim that SPSNet is the better probabilistic forecaster. The table shows that the teacher is not a close stand-in for .
[Experiment 1b] Variance-Robust and Teacher-Free IPS. We recompute IPS on the 60 plug-in settings (three backbones × five datasets × four horizons) under four variance treatments: the submission's heads, the same heads calibrated on validation, all heads refitted with one NLL recipe, and one shared variance, where only forecast disagreement counts.
| IPS computed with | +SPS below its backbone | +SPS below +KL | +KL below its backbone | Mean rank of SPSNet among 10 models |
|---|---|---|---|---|
| submission heads | 60/60 | 54/60 | 60/60 | 1.00 |
| calibrated heads | 60/60 | 49/60 | 60/60 | 1.00 |
| refitted heads | 60/60 | 51/60 | 59/60 | 1.00 |
| shared variance | 60/60 | 52/60 | 60/60 | 1.00 |
- Sensitivity to the variance calibration: magnitudes. The variance term is 92–99% of every IPS value in the submission. The size of an IPS value, and the percentage reductions in Tables 6 and 7, therefore mostly reflect how each model's variance head was fitted. The revision reports orderings instead.
- Sensitivity to the variance calibration: orderings. They are stable. +SPS is below its backbone in 60/60 settings under all four treatments and below +KL in 49–54/60. SPSNet has rank 1.00 among the 10 models (SPSNet, the three backbones, and their +KL and +SPS variants). With a shared variance, IPS reduces to the distance between the student's and the teacher's forecasts, and the conclusions are the same.
- Teacher-free test. Held-out NLL needs no teacher, and by Proposition 1 it orders methods as exact-reference IPS would. In [Experiment 1a], SPSNet's NLL is below that of the full-history teacher on all eight datasets. In exact-reference terms the teacher is further from than its student, which is one more reason not to read teacher-relative IPS as lost information.
[Experiment 1c] Sensitivity to the Teacher. We train SPSNet at H=96 with teachers stopped after 1, 3 and 10 epochs, with the full teacher, with no teacher, and with a teacher of SPSNet's own architecture without patch selection. IPS uses the submission's heads. Mean over six datasets, relative changes are averaged over datasets:
| Training teacher | Teacher MSE | SPSNet MSE | SPSNet MSE vs. full teacher | IPS vs. its own teacher | IPS vs. the full teacher |
|---|---|---|---|---|---|
| 1 epoch | 0.325 | 0.255 | +4.7% | 0.836 | 0.776 |
| 3 epochs | 0.289 | 0.252 | +3.2% | 0.462 | 0.410 |
| 10 epochs | 0.268 | 0.248 | +0.9% | 0.298 | 0.252 |
| full (submission) | 0.259 | 0.247 | – | 0.209 | 0.209 |
| none (no KL) | – | 0.252 | +2.9% | – | 0.408 |
| architecture-matched | 0.249 | 0.247 | +0.1% | 0.223 | 0.219 |
- Teacher quality matters for accuracy. Removing the full teacher raises MSE by 2.9%, and a 1-epoch teacher is worse than none.
- IPS depends on the teacher. The same student architecture has an IPS between 0.209 and 0.836 depending on its teacher. An IPS value describes a student–teacher pair and not a patching strategy alone.
- A weak teacher is not copied. With the 1-epoch teacher (MSE 0.325), SPSNet reaches 0.255, and its IPS against that teacher is the highest in the table (0.836). The supervised loss keeps the student away from a poor teacher, so a low IPS does not arise from imitating one.
- Sensitivity to the teacher architecture. Trained with a teacher of SPSNet's own architecture, SPSNet's MSE changes by 0.1%. Its IPS against the iTransformer teacher, which it never saw in training, is 0.219, against 0.209 when trained with that teacher and 0.408 with no teacher. The low IPS of SPSNet is therefore not tied to one teacher architecture. This experiment varies the teacher for SPSNet. It does not re-rank the baselines against a second teacher.
Summary. IPS shows agreement with the chosen teacher and not preservation of the true predictive information. The teacher is as accurate as a standard iTransformer but over-dispersed, and it is not an upper bound on SPSNet. The magnitudes of IPS are driven by the variance heads, while the orderings are unchanged under four variance treatments and under a teacher of a different architecture. The revision defines IPS as teacher-relative, adds Proposition 1 and [Experiment 1], and reports held-out NLL as the teacher-free comparison.
W3. The selector uses a global context vector, and the output is produced through mask-aware attention, which complicates reading it as information retained solely within the selected patches.
We agree that the output cannot be read as information retained solely within the selected patches. The revision spells out the mechanism in Section 3.4 and Appendix C:
- Tokens are local, and g is used only for scoring. Each candidate token is computed from its own interval and position (Equation 8). The global context g enters the scorer (Equations 10–11) and is not passed downstream.
- Pooling reads only the selected tokens. At inference the mask is binary, so unselected candidates receive exactly zero attention (Equation 28).
- The index set depends on the full history. Which K candidates are kept is a function of the full history, and it can carry up to bits per series.
Three measurements show how much each route matters.
[Experiment 2b] Selector Controls. Only the selection rule changes, on identical spectral candidates. "All pooled" attends over all 80 candidates into the same K output slots:
| Selector | MSE | MAE | MSE vs. learned | Learned better in |
|---|---|---|---|---|
| Learned (submission) | 0.317 | 0.320 | – | – |
| Learned, zeroed global context | 0.318 | 0.320 | +0.3% | 18/29 |
| All pooled | 0.322 | 0.323 | +1.6% | 23/29 |
| Recency (K latest-ending candidates) | 0.323 | 0.323 | +1.9% | 23/29 |
| Uniform (every N/K-th candidate) | 0.324 | 0.324 | +2.2% | 25/29 |
| Random K per series | 0.325 | 0.325 | +2.5% | 27/29 |
- Global context (first measurement). Zeroing g changes MSE by 0.3% and leaves MAE unchanged, and the learned selector is better in 18/29 settings. The scores rely mainly on the content of each candidate, and the global context adds little.
- Index set (second measurement). Recency, uniform and random selection keep K candidates without looking at the content of the history. They are 1.9–2.5% worse than learned selection. This is the whole value of choosing the candidates as a function of the history, so it is an upper bound on what the index set contributes as a side channel.
- Attention over the candidates. "All pooled" lets the K slots attend to all 80 candidates and is 1.6% worse than attending to the selected 10 (23/29). The accuracy of SPS does not rely on access to the unselected candidates.
We thank the reviewer for the valuable and constructive feedback.
W1. The spectral generator is not compared with fixed-scale, uniformly sampled multi-scale, or random multi-scale generators under identical candidate and selection budgets.
[Experiment 2a] Candidate Generator Controls. Only the period list changes. Every variant has N=80 candidates and K=10 selected patches, with the same learned selector, encoder, pooling, KL and training recipe. SPSNet, 29 settings (seven datasets × four horizons, and Traffic at H=96). MSE and MAE are means of the eight dataset averages, and relative changes are averaged over settings:
Summary. Under identical candidate and selection budgets, the spectral generator is better than fixed-scale, uniformly sampled multi-scale and random multi-scale generators, by 1.6–3.2% in MSE and in 22–27 of 29 settings. The revision adds this table to the ablation study (Appendix J).
W2. Lower IPS shows closer agreement with the chosen teacher, not preservation of the true predictive information.
We agree, and [Experiment 1] tests both sensitivities: the values of IPS depend strongly on the variance calibration and on the teacher, while the orderings between methods are stable. We now report IPS as teacher-relative.
Proposition 1. For any predictive distribution qm(Y∣Z) with Z=S(X),
E[DKL(P(Y∣X)∥qm(Y∣Z))]≥I(Y;X∣Z),E[−logqm(Y∣Z)]=H(Y∣X)+E[DKL(P(Y∣X)∥qm(Y∣Z))].
The gap in the inequality is E[DKL(P(Y∣Z)∥qm(Y∣Z))], the approximation error of the predictor. IPS therefore bounds the information loss only if its reference is the true P(Y∣X), and no such bound holds for a teacher. The difference in held-out NLL between two methods equals the difference in their exact-reference IPS, which gives a teacher-free test.
[Experiment 1a] Teacher Accuracy and Calibration. Mean over four horizons, with each model's own variance head: