- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes PHaSe, an adaptive time series data selection method that groups instances into temporal pattern prototypes and estimates their evolving hardness from recent loss trajectories. PHaSe transfers prototype hardness to instance sampling probabilities and dynamically updates the prototypes during training. Experiments across forecasting and classification datasets, multiple backbones, and time series foundation models show consistent improvements over full data training and existing selection methods, supported by ablation, pruning, and runtime analyses.
Critical strengths
-
Useful pattern level formulation. PHaSe aggregates evolving loss trajectories across related temporal patterns rather than scoring each instance independently. This provides a well motivated and practically simple way to stabilize adaptive selection while accounting for shared structure among time series instances.
-
Broad and consistent empirical support. PHaSe is evaluated across forecasting and classification tasks, three conventional backbones, and three time series foundation models. Its consistent improvements, together with component ablations, data pruning experiments, five seed evaluation, and runtime measurements, provide substantial evidence that the proposed selection mechanism is effective beyond a single dataset or architecture.
Critical weaknesses
-
Limited validation under persistent noise. PHaSe prioritizes patterns with sustained high losses, which may also correspond to mislabeled or irreducibly noisy instances. The paper acknowledges this failure mode but does not evaluate it under controlled label noise or temporal corruption, leaving its robustness in the motivating real world setting unclear.
-
The source of the prototype level gains is not fully isolated. The ablations show that prototype aggregation provides most of the improvement, while loss trend estimation, assignment gating, and dynamic maintenance contribute more modestly. A simple baseline using the same prototypes with uniform or current loss based allocation would help determine whether the full adaptive hardness mechanism is necessary.
Questions for authors
-
Could the authors report PHaSe’s performance under controlled label noise or temporal corruption?
-
Could the authors compare PHaSe with a simpler selector that uses the same prototypes but applies uniform allocation or only the current mean prototype loss?
2 comments
Thanks the author for the comments. My main concerns have been addressed and I will maintain my score.
Response to NSJE: persistent noise and prototype allocation
Thank you for the constructive questions about persistent noise and the source of prototype-level gains.
W1 / Q1. Controlled corruption and persistent high losses.
We agree that persistent high loss can arise from noise or unlearnable examples, so clean-data improvements alone do not establish robustness. We evaluate our method's ability to handle noise on Weather, M4-Monthly, and EthanolConcentration using DLinear, TimesNet, and iTransformer.
We add noise to the training inputs. We first normalize them, add zero-mean, unit-variance Gaussian noise in the normalized space, and transform them back to the original scale before subsequent processing. Corrupted inputs are fixed across epochs. The percentage is the fraction of corrupted training instances. Validation and test data remain clean.
The endpoints are shown below. Appendix E.3, Table 14 contains the complete results at 0%, 1%, 5%, 10%, 15%, and 20% corruption.
Under the tested corruption conditions, our model's performance degradation remains limited as noise increases. One possible explanation is that assignment concentration in Eq. (8) reduces the priority of noisy inputs when their memberships are spread across patterns, making them less likely to be sampled.
W2 / Q2. Uniform and current-loss allocation with the same prototypes.
We evaluate both requested controls. Uniform prototype allocation retains prototype construction and dynamic maintenance, assigns equal sampling mass to prototypes, and distributes it to instances through normalized memberships. It removes loss-based priorities and the assignment gate. Current prototype loss is equivalent to the "w/o loss trend estimation" variant. It replaces the historical trend signal with the latest observation loss and aggregates it through the same soft memberships.
The complete main-text Table 4 is reproduced below. Current prototype loss corresponds to "w/o loss trend estimation", and fixed clustering corresponds to "w/o prototype updates". MSE/MAE are long-term forecasting metrics, sMAPE is short-term forecasting, and accuracy is classification. Entries are mean±standard deviation over five seeds after equal-weight averaging across datasets and backbones within each task.
PHaSe outperforms both requested controls on all three tasks. Compared with uniform allocation, it reduces MSE by 0.0040 and sMAPE by 0.054, and improves accuracy by 0.82 percentage points. It also improves all three task metrics over current-loss allocation, supporting the combination of prototype maintenance, loss-driven priorities, gating, and loss history.
The extended ablation also compares random grouping and instance-level loss smoothing. Random grouping replaces k-means while retaining the other steps. Instance-level loss smoothing takes the arithmetic mean of each instance's latest W losses without prototype aggregation. PHaSe also outperforms these simpler alternatives. Among the five component removals, removing prototype aggregation causes the largest degradation, supporting pattern aggregation for data selection.
The full noise protocol and results are in Appendix E.3, Table 14. The unified ablation is Table 4.
We hope these clarifications and additional results address your remaining concerns. If so, we would be grateful if you would consider a more favorable overall recommendation.