- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DAD4TS, a diffusion-based data augmentation framework for time-series forecasting with limited training data. It jointly trains a generator and a forecasting model, using prediction errors to guide sample generation and Thompson sampling to select augmentation strategies. The framework also introduces a training-free latent transformation and rectified flow for efficient generation. Experiments on six datasets with six forecasting models show an overall RMSE improvement of 12.05%, outperforming several existing augmentation methods.
Critical strengths
-
The paper addresses a practical problem. Improving forecasting performance with limited training data is meaningful, and task-oriented data generation is a reasonable direction.
-
I appreciate the adaptive augmentation design. Updating the generator alongside the forecaster allows sample generation to respond to the current training state.
-
The experimental coverage is fairly broad, including six datasets, six forecasting models, and several recent augmentation baselines. The analysis of window normalization also provides some useful observations.
Critical weaknesses
-
Novelty is somewhat limited. AutoDA-Timeseries and ReAugment already explore forecasting-guided augmentation. DAD4TS mainly combines related ideas with diffusion generation and Thompson sampling.
-
The contribution of individual modules is not sufficiently clear. More direct ablations would help identify which components actually drive the improvements.
-
The gains are uneven across forecasting models. Most improvements come from models without window normalization, while the benefits for stronger normalized models are relatively small.
-
The augmentation policy requires additional validation rollouts during training. I am concerned about the computational overhead and possible overfitting to the small validation set.
Questions for authors
-
What is the main methodological difference between DAD4TS and AutoDA-Timeseries beyond using a diffusion generator?
-
Could the authors provide ablations for Thompson sampling, prediction-error weighting, and the training-free latent transformation?
-
How does DAD4TS compare with AutoDA under the same computational budget? Are the improvements over AutoDA statistically significant?
-
How do the authors prevent repeated validation-based augmentation selection from overfitting, especially on very small datasets?
4 comments
(Continued from the previous comment.)
Q3: Same budget and significance vs AutoDA-Timeseries.
DAD4TS-U uses validation data only for early stopping, exactly like AutoDA-Timeseries. Under this matched budget, we compared accuracy and each method's own cost (training and generation; German, one T4 GPU, one job at a time, per TSF model and horizon; the total including the TSF model depends on early stopping and is a reference). The results are as follows:
| Method | RMSE change (%) | p vs DAD4TS-U | Own cost (s) | Total, reference (s) |
|---|---|---|---|---|
| ReAugment | −1.41 | 2e-26 | 28.5 | 44.4 |
| TimeVAE | −1.72 | 2e-28 | 52.0 | 66.7 |
| AutoDA-Timeseries | −9.83 | 2.4e-7 | 62.8 | 79.0 |
| DAD4TS (submitted) | −12.05 | 0.42 | 401.3 | 420.4 |
| DAD4TS-U | −12.05 | – | 27.2 | 46.5 |
Overall, DAD4TS-U is significantly more accurate than AutoDA-Timeseries at a lower cost. However, on the normalizing TSF models alone (as split in Table 2 of our paper), the improvement over AutoDA-Timeseries is not significant (−2.77% vs −1.94%, p = 0.60).
Revisions. We will make DAD4TS-U the recommended configuration, add the ablation, cost and significance results and the training protocol, and move the label sampler and its diagnostics to the appendix.
The additional ablations and efficiency comparisons substantially address my concerns, particularly by demonstrating the value of joint generator–forecaster training and eliminating costly validation rollouts; I will raise my rating from 2 to 3.
Thank you for acknowledging our rebuttal efforts.
Dear Reviewer g9YX,
We would like to thank you once again for your efforts and constructive comments. We would be happy to address any further questions you may have.
Best regards,
Authors
We sincerely appreciate the reviewer’s insightful comments and constructive suggestions. All new experiments cover the full grid (6 datasets, 6 TSF models, 4 horizons, 3 seeds); p-values are from Wilcoxon signed-rank tests on log RMSE.
Q1 / W1: Difference from AutoDA-Timeseries (and ReAugment).
They learn how to transform existing training windows. In contrast, DAD4TS learns a generator of new windows, weighted by the TSF model's current errors and regenerated every epoch. To isolate this, we kept DAD4TS's data processing and changed only how new windows are obtained. The results are as follows:
We confirmed that generating windows yields better performance than simply transforming existing windows. Furthermore, coupling the generator with the TSF model adds a significant 4.5 points (p = 2e-12). The difference is also statistically significant when compared to AutoDA-Timeseries (p = 2.4e-7).
Q2 / W2: Ablations of the three modules.
We removed or replaced each module, and built the latent transformation up step by step. Since the label sampler was ineffective, we call the variant that draws processing labels uniformly at random every epoch DAD4TS-U (same −12.05%), and used it as the base of all other ablations. The results are as follows:
W3: Uneven gains across TSF models.
iTransformer/PatchTST/TimesNet normalize each input window, removing the level and scale variation that DAD4TS generates; copying level and scale from the real window costs 12.65 points on RNN/LSTM/Transformer but only 0.95 points on these models. The comparison methods show the same pattern (AutoDA-Timeseries −12.74% → −1.94%; others near zero or worse). DAD4TS-U still gives the largest gain there (−2.77%), though not significantly above AutoDA-Timeseries (p = 0.60). We will present this as a limitation with its explanation.
W4 / Q4: Validation rollouts and overfitting.
We appreciate this well-founded concern, which changed our recommended configuration. If labels are drawn uniformly at random each epoch (DAD4TS-U), no rollouts are needed and validation data serve only for early stopping, as for every comparison method. Accuracy stays the same (DAD4TS-U −12.047% vs DAD4TS (submitted) −12.051%, p = 0.42), and the label sampler reduces validation MSE by only 1.8%. Given the cost (Q3) and fair use of validation data, we adopt DAD4TS-U as the recommended configuration.
(Continued in the next comment.)