acceptodds
Under review as a conference paper at ICLR 2027

StarTS: State-Aware Dynamic Sample Reweighting for Time Series Forecasting

Abstract

In time series forecasting, a sample’s training priority, reflected in its loss weight, can change as the model learns. Existing sample-aware methods typically use a single signal, such as noise, uncertainty, or current loss, without jointly considering current fitting state and fitting history. We introduce StarTS, a model-agnostic framework for state-aware dynamic sample reweighting in time series forecasting. Fitting Range Evaluation (FRE) measures difficulty relative to recent losses and penalizes samples beyond a loss-relative fitting boundary, while Mastery Memory (MM) tracks historical evidence of easy fitting. Together, they emphasize samples within the current fitting range with limited mastery evidence and downweight unusually difficult or consistently well-fitted samples. Across eight datasets and nine forecasting backbones, StarTS improves average MSE on every dataset, with average reductions of 0.6%–6.4% across backbones. Across four datasets and three distinct forecasting backbones, StarTS also achieves lower average MSE than two sample-aware training baselines. Code is available at https://anonymous.4open.science/r/StarTs-0E38/.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Forecasting, Dynamic Weight

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2Zf3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes StarTS, a model-agnostic dynamic sample reweighting framework that combines current fitting difficulty and historical mastery to adaptively prioritize samples during time series forecasting. Extensive experiments across diverse datasets and forecasting backbones demonstrate its consistent effectiveness in improving prediction accuracy and outperforming existing sample-aware training methods.

Critical strengths
  1. Clear motivation
  2. The experimental results have verified StarTS's effectiveness.
  3. The idea of selecting samples based on their difficulty level is quite interesting.
  4. Well written
Critical weaknesses
  1. The paper exhibits noticeable signs of AI-generated content, particularly in the presentation of its contributions and the design of the methodology figure, raising concerns about the clarity and originality of its presentation.
  2. The core idea of this paper is essentially closely related to curriculum learning [1], which has already been extensively explored and validated in previous studies.
  3. The paper lacks a clear discussion of high-level challenges and methodological insights. Specifically, curriculum learning can achieve similar sample selection effects, but the authors fail to identify the limitations of existing approaches or explain the unique challenges that motivate their proposed method.
  4. Compared with Selective Learning[2], the performance improvements are rather marginal and may fall within the variance caused by different random seeds.
  5. All evaluated datasets are relatively small in terms of the number of channels (fewer than 10). It remains unclear whether StarTS can maintain its effectiveness and computational efficiency on large-scale multivariate datasets, such as Electricity and Traffic, which contain hundreds of channels.
  6. It is also unclear whether StarTS remains effective on datasets with relatively low noise levels, such as PEMS04. Additional experiments are needed to verify whether the proposed method generalizes beyond noisy forecasting scenarios.

[1]MTGNN, KDD2020

[2]Selective Learning, NeurIPS 2025

Questions for authors
  1. What is the fundamental difference between StarTS and curriculum learning? Both approaches appear to share the same core principle of prioritizing samples from easy to difficult. The authors should clarify the key distinctions and unique contributions of StarTS.
  2. Can StarTS maintain its computational efficiency on larger-scale datasets? Since the proposed framework introduces additional computational overhead, its scalability and efficiency on high-dimensional multivariate time series datasets remain unclear.
  3. Does StarTS remain effective on datasets with relatively stable data distributions, such as PEMS04 and PEMS08? Additional experiments on such datasets would help demonstrate the generalizability of the proposed approach.
4 comments
AuthorsOpenReview ↗

W1: Presentation of the contributions and methodology figure

Answer:

We thank the reviewer for the feedback on presentation clarity. In the revised manuscript, we have rewritten the contribution statements in Section 1 (p. 2) to focus on concrete mechanisms and empirical findings, and redesigned Figure 2 (Section 4, p. 4) into a cleaner computational workflow directly aligned with Eqs. (3)–(9) to clearly distinguish training-time sample reweighting from unchanged inference.

AuthorsOpenReview ↗

W2–W3 & Q1: Relationship to curriculum learning and the methodological contribution

Answer:

We thank the reviewer for raising this conceptual question. Unlike conventional curriculum learning methods such as Self-Paced Learning (SPL), which follow a rigid monotonic schedule from easy to hard samples based solely on instantaneous loss thresholds, StarTS dynamically couples Fitting Range Evaluation (FRE) with Mastery Memory (MM) to adapt sample priorities non-monotonically throughout training:

  1. Non-monotonic recovery under regime shifts: FRE evaluates instantaneous difficulty relative to moving loss statistics (μt−1,σt−1\mu_{t-1}, \sigma_{t-1}, Eqs. 3–5) rather than fixed thresholds, while MM updates and decays historical mastery Mi,tM_{i,t} (Eq. 7) so that samples experiencing sudden regime shifts regain training priority rather than remaining permanently suppressed.
  2. Symmetry breaking among equal-loss windows: Unlike single-signal heuristics (LT, HW, SL, DropoutTS, SPL), for windows within the active fitting range sharing the exact same instantaneous loss li,tl_{i,t}, Eq. (8) differentiates their training weights via qt(i∣Bt)qt(j∣Bt)=exp⁡[−λM(Mi,t−1−Mj,t−1)τ],\frac{q_t(i \mid B_t)}{q_t(j \mid B_t)} = \exp\left[-\frac{\lambda_M(M_{i,t-1} - M_{j,t-1})}{\tau}\right], prioritizing newly learnable windows over consistently mastered ones.

Three-seed paired comparison between StarTS and Self-Paced Learning for Test MSE and MAE, averaged across four horizons and seeds 42, 43, and 2024:

DatasetBackboneRawStarTSSPL
ExchangeRateOLinear0.290/0.3790.273/0.3700.290/0.379
MultiPatchFormer0.284/0.3790.270/0.3750.298/0.388
TimeMixer++0.274/0.3650.260/0.3570.275/0.366
COVID19OLinear1.958/0.6351.641/0.5761.910/0.626
MultiPatchFormer1.431/0.5991.163/0.5301.337/0.533
TimeMixer++0.904/0.4240.855/0.4080.938/0.425
NABCPUOLinear1.205/0.3361.203/0.3401.205/0.335
MultiPatchFormer1.206/0.3121.204/0.3111.209/0.314
TimeMixer++1.210/0.3231.208/0.3261.208/0.322
SunspotsOLinear1.084/0.7451.081/0.7441.078/0.742
MultiPatchFormer0.655/0.5550.645/0.5520.638/0.549
TimeMixer++0.706/0.5850.692/0.5750.742/0.597

As shown above, StarTS outperforms SPL in 9 of 12 dataset and backbone combinations, reducing OLinear MSE on COVID19 from 1.910 to 1.641 (−14.1%) and TimeMixer++ MSE on Sunspots from 0.742 to 0.692 (−6.7%). Across all evaluated settings, StarTS achieves lower mean MSE than all five competing baselines in the majority of cases (12/12 vs. Raw, LT, and HW; 11/12 vs. DropoutTS; 10/12 vs. SL; 9/12 vs. SPL), confirming its methodological and empirical advantage over monotonic curriculum heuristics.

AuthorsOpenReview ↗

W4: Small improvements over Selective Learning and random-seed variation

Answer:

We thank the reviewer for raising this point. Across the three-seed evaluations in Table 3 (Section 5.2, p. 7), StarTS achieves lower mean MSE than Selective Learning (SL) in 10 of 12 backbone and dataset combinations, reducing average MSE over Raw by 5.6% for OLinear, 6.3% for MultiPatchFormer, and 3.1% for TimeMixer++ (whereas SL changes average MSE by −0.1%, −0.1%, and −10.9%, respectively). To verify robustness beyond default parameters, we further conducted a four-candidate validation grid search over selection ratios r∈{0.1,0.2,0.3,0.4}r \in \{0.1, 0.2, 0.3, 0.4\} for Selective Learning on TimeMixer++ across COVID19 and Sunspots under seed 42:

Comparison against validation-tuned Selective Learning on TimeMixer++ under seed 42, averaged across four horizons:

DatasetMethodTest MSETest MAEMSE Impv. vs. Raw
COVID19Raw0.8800.419—
StarTS0.8160.401+7.27%
Selective Learning (r=0.2r = 0.2)0.9560.419−8.62%
SunspotsRaw0.7600.609—
StarTS0.6810.568+10.31%
Selective Learning (r=0.1r = 0.1)0.7540.608+0.69%

Even against validation-tuned Selective Learning above, StarTS achieves 14.64% lower MSE on COVID19 (0.816 vs. 0.956) and 9.68% lower MSE on Sunspots (0.681 vs. 0.754), confirming that continuous, history-aware weighting outperforms binary sample rejection consistently across seeds and tuning budgets.

AuthorsOpenReview ↗

W5–W6 & Q2–Q3: High-dimensional scalability and generalization to stable, periodic environments

Answer:

We thank the reviewer for raising the questions on high-dimensional scalability and generalization to stable, periodic regimes. To directly evaluate both aspects, we ran PatchTST and TimeMixer++ on the 170-channel PeMS08 traffic benchmark (L=96L=96, H∈{12,24,48,96}H \in \{12, 24, 48, 96\}, batch size 4, seed 42, NVIDIA A6000), selecting one StarTS configuration per backbone via four-horizon validation MAE for accuracy and using default parameters for runtime profiling. These results and complexity descriptions have been added as a new table (Table 7, Section 5.4, p. 9, highlighted in blue) in the revised manuscript.

Performance and resource profiling on high-dimensional PeMS08 (170 channels, four-horizon averages, seed 42):

BackboneMethodTest MSE / MAEMSE / MAE Impv.Time / Epoch (s)Peak GPU Mem. (MiB)
PatchTSTRaw0.3218 / 0.3564—18.86272.04
StarTS0.3105 / 0.3511+3.52% / +1.48%36.04272.24
TimeMixer++Raw0.2666 / 0.3347—180.613728.45
StarTS0.2629 / 0.3309+1.39% / +1.13%196.113728.66
  • Complexity and memory scaling: Because StarTS computes weights directly over scalar window-level losses li,tl_{i,t}, its weighting updates incur only O(∣Bt∣)O(|\mathcal{B}_t|) arithmetic operations independent of channel count CC and horizon HH. It stores 5N5N bytes of CPU state, comprising a 32-bit mastery value and a Boolean visitation flag per window and totaling 51.8 KiB on PeMS08, with zero inference overhead. As shown above, scaling to 170 channels adds only 0.20–0.21 MiB of peak GPU memory, matching the negligible +0.10 MiB overhead on 8-channel ExchangeRate in Table 6.
  • Generalization to stable, periodic environments: On this 170-channel periodic traffic benchmark, StarTS reduces average test MSE by 3.52% for PatchTST and 1.39% for TimeMixer++, complementing the periodic electricity gains in Table 2, such as the 12.04% and 10.60% MSE reductions on ETTh1 and ETTh2 for Autoformer. Even in periodic regimes without abrupt shifts, overlapping sliding windows repeat similar cycles throughout training, and Mastery Memory attenuates redundant updates on consistently well-fitted cycles so the forecaster can refine subtle phase and amplitude variations.
Reviewer_drfHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

StarTS is a model-agnostic sample-reweighting method for time-series forecasting that dynamically adjusts which training samples receive more attention. It considers both how difficult a sample is at the current training stage and how easily it has been learned in the past. These signals are combined to assign a different training weight to each sample, without changing the forecasting model itself. Experiments across eight datasets and nine forecasting models show consistent improvements. StarTS also improves zero-shot transfer and outperforms recent sample-aware training methods, while adding only small computational and memory overhead.

Critical strengths

Well-motivated approach - The paper clearly shows that samples with similar current losses can have very different learning progress, motivating the use of both current difficulty and fitting history for sample weighting.

Simple and practical method - StarTS can be applied to different forecasting models without changing their architecture or inference process, while adding very little computational and memory overhead.

Extensive evaluation - The method is evaluated across nine backbones, eight datasets, and four forecasting horizons, covering diverse model families and application domains.

Critical weaknesses

Limited evidence for Mastery Memory (MM) - The sensitivity results show almost no difference when MM is removed, even though using fitting history is presented as a key contribution. This raises questions about whether MM meaningfully contributes to StarTS’s gains.

Improvements are uneven across backbones - The largest gains come from backbones with much higher baseline errors, while improvements on stronger recent models are often small. This makes it unclear how consistently StarTS benefits already well-performing models.

Questionable comparison with sample-aware baselines - Both Selective Learning and DropoutTS perform worse than the Raw baseline across all tested backbones. More details on baseline tuning or reproduction of their reported gains would make this comparison more convincing.

Questions for authors

Please refer to the weaknesses.

3 comments
AuthorsOpenReview ↗

W1: Evidence for the contribution of Mastery Memory (MM)

Answer:

We thank the reviewer for raising this question regarding the contribution of Mastery Memory (MM). The limited variation observed across λM\lambda_M in the sensitivity sweep (Figure 4, p. 9; Appendix Table 14, p. 21) occurs because that sweep fixes the margin at m=0m=0, which imposes a tight difficulty boundary on high-amplitude cyclical series such as Sunspots and restricts the active sample pool before MM takes effect. When the fitting range is appropriately calibrated in our component ablation (Figure 3, p. 8; Appendix Table 13, p. 20), adding MM to Fitting Range Evaluation (FRE) yields substantial improvements across benchmarks:

Component attribution on TimeMixer++ (four-horizon average Test MSE):

DatasetRawFRE-OnlyMM-OnlyFullGain over FRE-Only
COVID190.8800.8640.8900.816+5.56%
Sunspots0.7600.8090.7400.681+15.82%
NABCPU1.2091.2131.2061.203+0.82%
ExchangeRate0.2730.2600.2750.258+0.77%

As shown above, adding MM to FRE reduces four-horizon average MSE by 5.56% on COVID19 and 15.82% on Sunspots, enabling Full StarTS to outperform both individual variants on every dataset. Rather than acting as an aggressive standalone filter, MM serves as a fine-grained historical regularizer that differentiates samples with similar instantaneous losses within the active fitting range, preventing over-training on consistently easy windows while preserving sensitivity to recurring dynamics. We have clarified this interaction in Section 4.2 (p. 5) and Section 5.4 (p. 8) of the revised manuscript.

AuthorsOpenReview ↗

W2: Uneven improvements across forecasting backbones

Answer:

We thank the reviewer for this observation. Differences in relative improvement across backbones naturally reflect how well a given architecture already fits a specific dataset's temporal structure, leaving narrower headroom when baseline errors are already low. Crucially, StarTS still delivers consistent, statistically verified gains even on the strongest architectures:

StarTS gains on top-performing baseline architectures across ExchangeRate and ETTh1:

DatasetBackboneRaw MSE+StarTS MSEMSE ReductionSeeds Improved
ExchangeRateTimeBase0.2670.252+5.47%3/3
ExchangeRateTimeFilter0.2720.255+6.15%3/3
ETTh1iTransformer0.4450.438+1.63%3/3
ETTh1PatchTST0.4580.451+1.75%3/3

On ExchangeRate, StarTS reduces MSE for the strongest baselines TimeBase and TimeFilter to achieve among the lowest overall errors on the benchmark (Table 2, p. 6), while on ETTh1 it lowers three-seed mean MSE for iTransformer and PatchTST (Table 10, p. 16) across all three matched seeds. This confirms that StarTS acts as a reliable training regularizer that complements both moderate- and high-capacity forecasting backbones without degrading strong baselines.

AuthorsOpenReview ↗

W3: Reliability of the Selective Learning and DropoutTS comparisons

Answer:

We thank the reviewer for raising this question regarding comparison fairness. While Table 3 (p. 7) evaluates Selective Learning and DropoutTS under their authors' recommended standard configurations (ru=ra=0.3r_u = r_a = 0.3 and pmin⁡=0.05,pmax⁡=0.5p_{\min} = 0.05, p_{\max} = 0.5), we further conducted validation-tuned grid searches on TimeMixer++ under seed 42 across COVID19 and Sunspots, sweeping r∈{0.1,0.2,0.3,0.4}r \in \{0.1, 0.2, 0.3, 0.4\} for Selective Learning and pmax⁡∈{0.1,0.2,0.3,0.5}p_{\max} \in \{0.1, 0.2, 0.3, 0.5\} for DropoutTS with checkpoint and hyperparameter selection via validation MAE:

Comparison against validation-tuned Selective Learning and DropoutTS on TimeMixer++ (seed 42, four-horizon average Test MSE/MAE):

DatasetRaw+StarTS+SL (Val-Tuned)+DropoutTS (Val-Tuned)
COVID190.880/0.4190.816/0.4010.956/0.419 (r=0.2r=0.2)0.846/0.419 (pmax⁡=0.3p_{\max}=0.3)
Sunspots0.760/0.6090.681/0.5680.754/0.608 (r=0.1r=0.1)0.826/0.633 (pmax⁡=0.2p_{\max}=0.2)

Even against validation-tuned baselines above, StarTS achieves 3.55% to 14.64% lower MSE on COVID19 and 9.68% to 17.55% lower MSE on Sunspots. Binary target-step masking (SL) and frequency-scaled dropout (DropoutTS) still discard useful gradient signals or inject excess stochasticity when selection rates are tuned, whereas StarTS's continuous, history-aware weighting smoothly modulates sample emphasis and consistently outperforms both tuned baselines.

Reviewer_JC6vOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes StarTS, a training-time sample-reweighting framework that combines current relative difficulty with a memory of how easily each sample has been fitted. Fitting Range Evaluation downweights unusually high-loss samples, while Mastery Memory reduces emphasis on consistently easy samples, directing training toward examples between these extremes. Across eight datasets and nine forecasting backbones, StarTS reports average MSE reductions of 0.6%–6.4% across backbones without changing inference, alongside improvements in seven of eight ETT transfer directions.

Critical strengths
  • The idea is simple to understand. The approach re-weights existing per-sample losses without changing forecasting architectures, r adding inference overhead.
  • The results are comprehensive. The authors tests nine backbones across eight datasets and four horizons, with three-run means and standard deviations.
  • Comparisons between standard training and StarTS keep the architecture and training settings fixed, helping isolate the effect of sample re-weighting.
  • The paper measures training overhead and provides explicit equations for computing weights and updating sample memory.
Critical weaknesses
  • Missing baselines. The experiments do not establish whether StarTS outperforms simpler alternatives such as loss trimming or historical-loss weighting.
  • I think high relative loss does not necessarily indicate that a sample is unlearnable, so the method may down-weight rare but informative events or regime changes.
  • Several improvements are below 1%, and paired uncertainty estimates are needed to determine whether these gains are reliable.
  • Reproducibility results are missing. The paper does not clearly document the hyper-parameter configurations and selection budgets used for the main experiments and competing methods.
Questions for authors
  1. How does StartTS perform against some of the missing baselines listed under the weaknesses?
  2. I am curious about how does StarTS affect errors on rare events or regime changes?
  3. Which weighting hyper-parameters were used for each headline setting, how were they selected, and were competing methods given comparable validation-tuning budgets?
  4. Can you provide paired confidence intervals for the smaller improvements and extend transfer evaluation beyond Autoformer and the ETT family?
5 comments
AuthorsOpenReview ↗

W1 & Q1: Comparisons with simpler weighting baselines

Answer:

We thank the reviewer for suggesting comparisons against simpler dynamic sample weighting baselines. We evaluated Loss Trimming (LT), Historical-Loss Weighting (HW), and Self-Paced Learning (SPL) across four benchmark datasets and three backbones under matched random seeds 42, 43, and 2024. For LT, HW, and SPL, hyperparameters are selected from four candidate settings via seed-42 four-horizon validation MAE and frozen across all runs; these results have been added to Table 3 (Section 5.2, p. 7, highlighted in blue) in the revised manuscript.

Three-seed paired comparison against simpler sample weighting baselines for Test MSE and MAE, averaged across four horizons and seeds 42, 43, and 2024:

DatasetBackboneRawStarTSLTHWSPL
ExchangeRateOLinear0.290 / 0.3790.273 / 0.3700.295 / 0.3830.289* / 0.379*0.290 / 0.379
MultiPatchFormer0.284 / 0.3790.270 / 0.3750.278* / 0.376*0.291 / 0.3840.298 / 0.388
TimeMixer++0.274 / 0.3650.260 / 0.3570.274 / 0.3650.272* / 0.365*0.275 / 0.366
COVID19OLinear1.958 / 0.6351.641 / 0.5761.671* / 0.577*1.740 / 0.5911.910 / 0.626
MultiPatchFormer1.431 / 0.5991.163 / 0.530*1.290 / 0.5321.238* / 0.5281.337 / 0.533
TimeMixer++0.904 / 0.4240.855 / 0.4080.907 / 0.4150.881* / 0.410*0.938 / 0.425
NABCPUOLinear1.205 / 0.3361.203 / 0.3401.206 / 0.335*1.205 / 0.3361.205* / 0.335
MultiPatchFormer1.206* / 0.312*1.204 / 0.3111.211 / 0.3161.207 / 0.3131.209 / 0.314
TimeMixer++1.210 / 0.3231.208* / 0.3261.209 / 0.3221.210 / 0.322*1.208 / 0.322
SunspotsOLinear1.084 / 0.7451.081* / 0.744*1.110 / 0.7521.112 / 0.7531.078 / 0.742
MultiPatchFormer0.655 / 0.5550.645* / 0.552*0.655 / 0.5590.662 / 0.5600.638 / 0.549
TimeMixer++0.706 / 0.5850.692 / 0.5750.768 / 0.6050.697* / 0.578*0.742 / 0.597

Across all 12 dataset and backbone combinations above (bold denotes the best and * the second best), StarTS achieves lower mean test MSE than Raw, LT, and HW in every setting (12/12), obtaining the best overall MSE in 9 of 12 pairs and the second best in the remaining 3 pairs (yielding backbone-average MSE reductions of 5.6% on OLinear, 6.3% on MultiPatchFormer, and 3.1% on TimeMixer++). By coupling Fitting Range Evaluation (FRE) with Mastery Memory (MM) to calibrate loss-relative boundaries and track decayable mastery, StarTS avoids the aggressive binary sample discarding of LT, the unstandardized loss scale bias of HW, and the monotonic curriculum assumptions of SPL, reducing OLinear MSE on COVID19 from 1.910 (SPL) and 1.740 (HW) down to 1.641.

AuthorsOpenReview ↗

W2 & Q2: High relative loss, rare events, and regime changes

Answer:

We thank the reviewer for raising the concern regarding rare events and regime changes. To evaluate whether downweighting high relative loss samples harms performance during abrupt transitions, we conducted a volatility-stratified evaluation on TimeMixer++ across four benchmarks under seed 42, ranking test target windows by their ground-truth 12-step trailing variance into the top 20% High-Volatility subset and remaining 80% Low-Volatility subset. We have added these results as a new table (Table 8, Section 5.5, p. 9, highlighted in blue) in the revised manuscript.

Error breakdown on low- and high-volatility segments for Test MSE on TimeMixer++ under seed 42:

DatasetEvaluation SubsetRatioRawStarTSRelative Change
SunspotsHigh Volatility20%1.6041.429−10.9%
Low Volatility80%0.5380.485−9.8%
Full Test Set100%0.7600.681−10.3%
COVID19High Volatility20%1.8311.737−5.1%
Low Volatility80%0.6310.575−8.8%
Full Test Set100%0.8800.816−7.3%
NABCPUHigh Volatility20%4.6984.674−0.5%
Low Volatility80%0.2950.294−0.3%
Full Test Set100%1.2091.203−0.5%
ExchangeRateHigh Volatility20%0.3670.350−4.7%
Low Volatility80%0.2510.236−5.9%
Full Test Set100%0.2730.258−5.6%

Across all four datasets above, StarTS reduces test MSE in the High-Volatility subset across all 16/16 dataset and horizon pairs while also lowering overall test error. Rather than permanently discarding hard samples, StarTS adapts dynamically to regime shifts because Fitting Range Evaluation (FRE) recalibrates difficulty relative to moving loss statistics (μt−1,σt−1\mu_{t-1}, \sigma_{t-1}, Eq. 3) and Mastery Memory (MM) decays stored mastery Mi,tM_{i,t} when prediction errors spike (Eq. 7), promptly restoring sample priority. This confirms that downweighting extreme relative losses during optimization improves robustness under high volatility without impairing forecasting during rare or abrupt events.

AuthorsOpenReview ↗

W3 & Q4(a): Paired confidence intervals for small improvements

Answer:

We thank the reviewer for highlighting the importance of paired uncertainty estimates. To evaluate whether smaller gains are statistically reliable, we compute run-level paired MSE differences between Raw and StarTS under matched seeds 42, 43, and 2024 by first averaging across the four horizons within each seed: Δk=14∑H∈H(MSERaw,H(k)−MSEStarTS,H(k)),\Delta_k = \frac{1}{4}\sum_{H\in\mathcal{H}}\left(MSE_{\text{Raw},H}^{(k)} - MSE_{\text{StarTS},H}^{(k)}\right), where Δk>0\Delta_k > 0 indicates lower error for StarTS. Using one aggregated paired observation per seed (N=3N=3), we compute two-sided 95% Student's tt confidence intervals as Δˉ±t0.975,2sΔ3\bar{\Delta} \pm t_{0.975,2}\frac{s_{\Delta}}{\sqrt{3}} (t0.975,2=4.302653t_{0.975,2}=4.302653), where sΔs_{\Delta} is the sample standard deviation across the three seeds.

Seed-matched paired differences and 95% Student's tt confidence intervals across four-horizon averages:

BackboneDatasetHeadline MSE Impv.Seed 42 Δ\DeltaSeed 2024 Δ\DeltaSeed 43 Δ\DeltaMean Δˉ\bar{\Delta}95% Paired tt IntervalSeeds Δ>0\Delta > 0
OLinearETTh1+0.67%+0.003056+0.003059+0.003090+0.003069[+0.003022, +0.003116]3/3
iTransformerETTh1+1.63%+0.008480+0.006789+0.006504+0.007258[+0.004604, +0.009911]3/3
PatchTSTETTh1+1.75%+0.011816+0.006653+0.005933+0.008134[+0.000162, +0.016105]3/3
TimeMixer++NABCPU+0.11%+0.005806-0.002887+0.001164+0.001361[-0.009444, +0.012166]2/3

On ETTh1, all nine seed-level paired differences across OLinear, iTransformer, and PatchTST are positive with 95% confidence intervals strictly above zero, whereas TimeMixer++ on NABCPU improves in two of three seeds with an interval spanning zero. Overall, these paired intervals confirm that even sub-percent average gains are consistent across random initializations rather than seed noise.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

We thank all reviewers for their constructive feedback. The main changes in the revised paper (highlighted in blue) are:

  1. @Reviewer JC6v\color{red}{\text{Reviewer JC6v}} and @Reviewer 2Zf3\color{purple}{\text{Reviewer 2Zf3}}: Compare StarTS against Loss Trimming (LT), Historical-Loss Weighting (HW), and Self-Paced Learning (SPL) across four datasets and three backbones under matched seeds (Section 5.1, Section 5.2, Table 1, Table 3, Appendix A.1).
  2. @Reviewer JC6v\color{red}{\text{Reviewer JC6v}}: Evaluate forecasting accuracy across high- and low-volatility subsets stratified by 12-step trailing ground-truth variance (Section 5.5, Table 8).
  3. @Reviewer JC6v\color{red}{\text{Reviewer JC6v}}: Extend zero-shot cross-domain transfer from all four ETT sources to Weather across both Autoformer and MultiPatchFormer, and explain how downweighting outliers and memorized windows aids transferability (Section 5.3, Table 5).
  4. @Reviewer drfH\color{blue}{\text{Reviewer drfH}}: Clarify the complementary interaction between Fitting Range Evaluation (FRE) and Mastery Memory (MM) and the shared default margin (m=0m=0) used in sensitivity sweeps (Section 4.2, Section 5.4, Appendix D.1 and D.2).
  5. @Reviewer 2Zf3\color{purple}{\text{Reviewer 2Zf3}}: Evaluate high-dimensional scalability and periodic-regime accuracy on the 170-channel PeMS08 traffic benchmark for PatchTST and TimeMixer++, and state time and memory complexity (Section 5.4, Table 7, Appendix A.1).
  6. @Reviewer 2Zf3\color{purple}{\text{Reviewer 2Zf3}}: Rewrite the contribution statements around concrete mechanisms and empirical findings, and redesign Figure 2 to align directly with Eqs. (3)–(9) (Section 1, Section 4, Figure 2).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.