- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes StarTS, a model-agnostic dynamic sample reweighting framework that combines current fitting difficulty and historical mastery to adaptively prioritize samples during time series forecasting. Extensive experiments across diverse datasets and forecasting backbones demonstrate its consistent effectiveness in improving prediction accuracy and outperforming existing sample-aware training methods.
Critical strengths
- Clear motivation
- The experimental results have verified StarTS's effectiveness.
- The idea of selecting samples based on their difficulty level is quite interesting.
- Well written
Critical weaknesses
- The paper exhibits noticeable signs of AI-generated content, particularly in the presentation of its contributions and the design of the methodology figure, raising concerns about the clarity and originality of its presentation.
- The core idea of this paper is essentially closely related to curriculum learning [1], which has already been extensively explored and validated in previous studies.
- The paper lacks a clear discussion of high-level challenges and methodological insights. Specifically, curriculum learning can achieve similar sample selection effects, but the authors fail to identify the limitations of existing approaches or explain the unique challenges that motivate their proposed method.
- Compared with Selective Learning[2], the performance improvements are rather marginal and may fall within the variance caused by different random seeds.
- All evaluated datasets are relatively small in terms of the number of channels (fewer than 10). It remains unclear whether StarTS can maintain its effectiveness and computational efficiency on large-scale multivariate datasets, such as Electricity and Traffic, which contain hundreds of channels.
- It is also unclear whether StarTS remains effective on datasets with relatively low noise levels, such as PEMS04. Additional experiments are needed to verify whether the proposed method generalizes beyond noisy forecasting scenarios.
[1]MTGNN, KDD2020
[2]Selective Learning, NeurIPS 2025
Questions for authors
- What is the fundamental difference between StarTS and curriculum learning? Both approaches appear to share the same core principle of prioritizing samples from easy to difficult. The authors should clarify the key distinctions and unique contributions of StarTS.
- Can StarTS maintain its computational efficiency on larger-scale datasets? Since the proposed framework introduces additional computational overhead, its scalability and efficiency on high-dimensional multivariate time series datasets remain unclear.
- Does StarTS remain effective on datasets with relatively stable data distributions, such as PEMS04 and PEMS08? Additional experiments on such datasets would help demonstrate the generalizability of the proposed approach.
4 comments
W2–W3 & Q1: Relationship to curriculum learning and the methodological contribution
Answer:
We thank the reviewer for raising this conceptual question. Unlike conventional curriculum learning methods such as Self-Paced Learning (SPL), which follow a rigid monotonic schedule from easy to hard samples based solely on instantaneous loss thresholds, StarTS dynamically couples Fitting Range Evaluation (FRE) with Mastery Memory (MM) to adapt sample priorities non-monotonically throughout training:
- Non-monotonic recovery under regime shifts: FRE evaluates instantaneous difficulty relative to moving loss statistics (, Eqs. 3–5) rather than fixed thresholds, while MM updates and decays historical mastery (Eq. 7) so that samples experiencing sudden regime shifts regain training priority rather than remaining permanently suppressed.
- Symmetry breaking among equal-loss windows: Unlike single-signal heuristics (LT, HW, SL, DropoutTS, SPL), for windows within the active fitting range sharing the exact same instantaneous loss , Eq. (8) differentiates their training weights via prioritizing newly learnable windows over consistently mastered ones.
Three-seed paired comparison between StarTS and Self-Paced Learning for Test MSE and MAE, averaged across four horizons and seeds 42, 43, and 2024:
| Dataset | Backbone | Raw | StarTS | SPL |
|---|---|---|---|---|
| ExchangeRate | OLinear | 0.290/0.379 | 0.273/0.370 | 0.290/0.379 |
| MultiPatchFormer | 0.284/0.379 | 0.270/0.375 | 0.298/0.388 | |
| TimeMixer++ | 0.274/0.365 | 0.260/0.357 | 0.275/0.366 | |
| COVID19 | OLinear | 1.958/0.635 | 1.641/0.576 | 1.910/0.626 |
| MultiPatchFormer | 1.431/0.599 | 1.163/0.530 | 1.337/0.533 | |
| TimeMixer++ | 0.904/0.424 | 0.855/0.408 | 0.938/0.425 | |
| NABCPU | OLinear | 1.205/0.336 | 1.203/0.340 | 1.205/0.335 |
| MultiPatchFormer | 1.206/0.312 | 1.204/0.311 | 1.209/0.314 | |
| TimeMixer++ | 1.210/0.323 | 1.208/0.326 | 1.208/0.322 | |
| Sunspots | OLinear | 1.084/0.745 | 1.081/0.744 | 1.078/0.742 |
| MultiPatchFormer | 0.655/0.555 | 0.645/0.552 | 0.638/0.549 | |
| TimeMixer++ | 0.706/0.585 | 0.692/0.575 | 0.742/0.597 |
As shown above, StarTS outperforms SPL in 9 of 12 dataset and backbone combinations, reducing OLinear MSE on COVID19 from 1.910 to 1.641 (−14.1%) and TimeMixer++ MSE on Sunspots from 0.742 to 0.692 (−6.7%). Across all evaluated settings, StarTS achieves lower mean MSE than all five competing baselines in the majority of cases (12/12 vs. Raw, LT, and HW; 11/12 vs. DropoutTS; 10/12 vs. SL; 9/12 vs. SPL), confirming its methodological and empirical advantage over monotonic curriculum heuristics.
W4: Small improvements over Selective Learning and random-seed variation
Answer:
We thank the reviewer for raising this point. Across the three-seed evaluations in Table 3 (Section 5.2, p. 7), StarTS achieves lower mean MSE than Selective Learning (SL) in 10 of 12 backbone and dataset combinations, reducing average MSE over Raw by 5.6% for OLinear, 6.3% for MultiPatchFormer, and 3.1% for TimeMixer++ (whereas SL changes average MSE by −0.1%, −0.1%, and −10.9%, respectively). To verify robustness beyond default parameters, we further conducted a four-candidate validation grid search over selection ratios for Selective Learning on TimeMixer++ across COVID19 and Sunspots under seed 42:
Comparison against validation-tuned Selective Learning on TimeMixer++ under seed 42, averaged across four horizons:
| Dataset | Method | Test MSE | Test MAE | MSE Impv. vs. Raw |
|---|---|---|---|---|
| COVID19 | Raw | 0.880 | 0.419 | — |
| StarTS | 0.816 | 0.401 | +7.27% | |
| Selective Learning () | 0.956 | 0.419 | −8.62% | |
| Sunspots | Raw | 0.760 | 0.609 | — |
| StarTS | 0.681 | 0.568 | +10.31% | |
| Selective Learning () | 0.754 | 0.608 | +0.69% |
Even against validation-tuned Selective Learning above, StarTS achieves 14.64% lower MSE on COVID19 (0.816 vs. 0.956) and 9.68% lower MSE on Sunspots (0.681 vs. 0.754), confirming that continuous, history-aware weighting outperforms binary sample rejection consistently across seeds and tuning budgets.
W5–W6 & Q2–Q3: High-dimensional scalability and generalization to stable, periodic environments
Answer:
We thank the reviewer for raising the questions on high-dimensional scalability and generalization to stable, periodic regimes. To directly evaluate both aspects, we ran PatchTST and TimeMixer++ on the 170-channel PeMS08 traffic benchmark (, , batch size 4, seed 42, NVIDIA A6000), selecting one StarTS configuration per backbone via four-horizon validation MAE for accuracy and using default parameters for runtime profiling. These results and complexity descriptions have been added as a new table (Table 7, Section 5.4, p. 9, highlighted in blue) in the revised manuscript.
Performance and resource profiling on high-dimensional PeMS08 (170 channels, four-horizon averages, seed 42):
| Backbone | Method | Test MSE / MAE | MSE / MAE Impv. | Time / Epoch (s) | Peak GPU Mem. (MiB) |
|---|---|---|---|---|---|
| PatchTST | Raw | 0.3218 / 0.3564 | — | 18.86 | 272.04 |
| StarTS | 0.3105 / 0.3511 | +3.52% / +1.48% | 36.04 | 272.24 | |
| TimeMixer++ | Raw | 0.2666 / 0.3347 | — | 180.61 | 3728.45 |
| StarTS | 0.2629 / 0.3309 | +1.39% / +1.13% | 196.11 | 3728.66 |
- Complexity and memory scaling: Because StarTS computes weights directly over scalar window-level losses , its weighting updates incur only arithmetic operations independent of channel count and horizon . It stores bytes of CPU state, comprising a 32-bit mastery value and a Boolean visitation flag per window and totaling 51.8 KiB on PeMS08, with zero inference overhead. As shown above, scaling to 170 channels adds only 0.20–0.21 MiB of peak GPU memory, matching the negligible +0.10 MiB overhead on 8-channel ExchangeRate in Table 6.
- Generalization to stable, periodic environments: On this 170-channel periodic traffic benchmark, StarTS reduces average test MSE by 3.52% for PatchTST and 1.39% for TimeMixer++, complementing the periodic electricity gains in Table 2, such as the 12.04% and 10.60% MSE reductions on ETTh1 and ETTh2 for Autoformer. Even in periodic regimes without abrupt shifts, overlapping sliding windows repeat similar cycles throughout training, and Mastery Memory attenuates redundant updates on consistently well-fitted cycles so the forecaster can refine subtle phase and amplitude variations.
W1: Presentation of the contributions and methodology figure
Answer:
We thank the reviewer for the feedback on presentation clarity. In the revised manuscript, we have rewritten the contribution statements in Section 1 (p. 2) to focus on concrete mechanisms and empirical findings, and redesigned Figure 2 (Section 4, p. 4) into a cleaner computational workflow directly aligned with Eqs. (3)–(9) to clearly distinguish training-time sample reweighting from unchanged inference.