acceptodds
Under review as a conference paper at ICLR 2027

When and Why Adversarial Training Improves Time-Series Forecasting

Abstract

While adversarial training exhibits contrasting clean-accuracy effects across vision and language, whether and why it benefits unperturbed data in time-series forecasting remains an open question. In this work, we uncover an unexpected within-domain clean-effect reversal: standard raw-history adversarial training (AT) reduces unperturbed test MSE by up to for expressive nonlinear forecasters, yet systematically degrades compact linear models. Through exact finite-network risk identities and asymptotic expansions, we trace this divergence to an intrinsic competition between prediction-variance contraction and parameter displacement: while loss-guided updates act as a frequency-selective regularizer that dampens low-frequency prediction variance, fitting perturbed histories pulls parameters away from the clean empirical minimum, incurring a displacement penalty that can outweigh variance gains in low-capacity architectures. To govern this trade-off, we propose Displacement-Guided Adversarial Training (DGA). DGA introduces an elastic parameter anchor to a pre-trained empirical risk minimization (ERM) checkpoint, effectively suppressing weakly constrained parameter drift while preserving beneficial variance regularization. Across 56 forecasting configurations spanning eight architectures and seven benchmarks, validation-selected DGA outperforms paired ERM in 51 settings (), turning linear-model degradations into consistent clean gains (e.g., MSE on DLinear). Furthermore, DGA transfers effectively to language fine-tuning and vision classification, establishing parameter displacement control as a general stabilizer for adversarial optimization.

Primary area:
learning on time series and dynamical systems
Keywords:
Time-Series Forecasting, Adversarial Training, Bias-Variance Trade-off, Trajectory Control

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_xucFOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies why adversarial training improves clean forecasting performance for nonlinear models but hurts compact linear models. It attributes this difference to the trade-off between prediction-variance reduction and parameter displacement. The author proposes DGA, which anchors adversarially trained parameters to a clean ERM checkpoint using an ℓ2\ell_2 penalty.

Critical strengths
  1. This paper reveals a capacity-dependent reversal in the effect of adversarial training on clean performance.
  2. The experiments cover multiple architectures, datasets, forecasting horizons, and random seeds.
Critical weaknesses
  1. The novelty is limited because DGA is essentially adversarial training with an ℓ2\ell_2 penalty toward a clean ERM checkpoint, closely related to existing parameter-anchoring methods.
  2. The comparison may be unfair because DGA searches five anchoring coefficients over five seeds, while ERM and other regularization baselines are not given an equivalent tuning budget.
  3. Selecting DGA from twenty-five training runs may introduce a substantial validation-selection advantage.
  4. The clean-only anchored baseline performs slightly better than DGA in the diagnostic linear settings, making it unclear whether adversarial perturbations are necessary.
  5. The method has high training cost, some improvements are small.
Questions for authors

Besides the weakness section, another concern is that the paper’s figures and text appear to have been excessively processed by LLMs.

Reviewer_WCznOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The authors investigate the phenomenon where adversarial training has differing effect on performance depending on domain, data modality, and model size. For example, in general it increases downstream clean performance for language models but decreases performance for image models. Due to the many variables changed between the two settings, making it hard to attribute a cause, the authors focus on the time-series prediction case. There, they find that the direction of the adversarial training <-> clean performance effect depends solely on the capacity of the model: bigger models benefit while smaller models are hurt. They attribute this to an interaction between perturbation geometry and model capacity. Using their suggested approach DGA, they obtain benefits on forecasting benchmarks.

Critical strengths
  • The shown trend of capacity vs benefit/hindrance from adversarial training is clear in the time-series domain
  • Performance gains from the model happen in the great majority of experiments
Critical weaknesses

Both components of the method, 1) blending normal loss and adversarial loss, and 2) initial checkpoint distance regularisation are well-known in the literature, even together

(a) + (b) Adversarial fine-tuning combined with closeness to the base or earlier model

  • [ACL 2020] Jiang et al., SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization (arXiv:1911.03437)
  • [NeurIPS 2021] Dong et al., How Should Pre-Trained Language Models Be Fine-Tuned Towards Adversarial Robustness? (arXiv:2112.11668)
  • [CVPR 2024] Wang et al., Pre-trained Model Guided Fine-Tuning for Zero-Shot Adversarial Robustness (arXiv:2401.04350)

(a) Weighted average of clean and adversarial loss

  • [ICLR 2015] Goodfellow et al., Explaining and Harnessing Adversarial Examples (arXiv:1412.6572)
  • [ICLR 2017] Kurakin et al., Adversarial Machine Learning at Scale (arXiv:1611.01236)
  • [ICML 2019] Zhang et al., Theoretically Principled Trade-off between Robustness and Accuracy (arXiv:1901.08573)

(b) L2 or other regularization toward the base model

  • [ICML 2018] Li et al., Explicit Inductive Bias for Transfer Learning with Convolutional Networks (arXiv:1802.01483)
  • [CVPR 2022] Wortsman et al., Robust Fine-Tuning of Zero-Shot Models (arXiv:2109.01903)

Benefits from adversarial training in the vision domain are not observed even with their method. An intermediate performance between no-adversarial and full-adversarial are seen, which is not surprising given the blended objective part of DGA. The claim "demonstrating that adversarial generalization is governed by model capacity rather than data modality" seems too strong, because the dynamics observed still vary between text, vision, and time-series.

Questions for authors
  • If the benefits or hindrance from adversarial training come from insufficient capacity, why is this not observed in vision models?
  • Is the main contribution of the work the finding that capacity modulates the behaviour in time-series?
Reviewer_feh4OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies why standard adversarial training can improve clean forecasting error for expressive nonlinear models but degrade compact linear forecasters, attributing this to a trade-off between variance reduction and parameter displacement. It proposes Displacement-Guided Adversarial Training (DGA), which anchors the model near a pretrained ERM solution while retaining adversarial regularization, and evaluates it on 7 time-series benchmarks with 8 forecasting architectures spanning nonlinear and linear models. Across 56 forecasting configurations, validation-selected DGA outperforms paired ERM in 51 settings, and the paper further reports transfer of the method to language fine-tuning and vision classification.

Critical strengths

The paper studies why standard adversarial training can improve clean forecasting error for expressive nonlinear models but degrade compact linear forecasters, attributing this to a trade-off between variance reduction and parameter displacement. It proposes Displacement-Guided Adversarial Training (DGA), which anchors the model near a pretrained ERM solution while retaining adversarial regularization, and evaluates it on 7 time-series benchmarks with 8 forecasting architectures spanning nonlinear and linear models. Across 56 forecasting configurations, validation-selected DGA outperforms paired ERM in 51 settings, and the paper further reports transfer of the method to language fine-tuning and vision classification.

The paper is well written, proven with a lot of experiments, datasets, and different settings. Adversarial training can improve clean forecasting accuracy for expressive nonlinear time-series models.

Critical weaknesses

I don't have much to say; it would be nice to see how this work handles the time series where:

  1. the variation in the time series is too high.
  2. Variation is not due to errors in hardware or environment.
  3. Such a domain is physiological signals, like EEG, ECG, fNIRS, etc.

Do you have any comments on that?

Thank you

Questions for authors

I don't have much to say; it would be nice to see how this work handles the time series where:

  1. the variation in the time series is too high.
  2. Variation is not due to errors in hardware or environment.
  3. Such a domain is physiological signals, like EEG, ECG, fNIRS, etc.

Do you have any comments on that?

Thank you

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.