- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes AnchorMix, a two-stage distillation framework that transfers knowledge from heterogeneous time-series forecasters into a single anchor model. They introduce CycleRoutedTQNet, a student architecture comprising five internal forecasting branches and a sample-adaptive router. In stage 1, this architecture is trained as an anchor using ground-truth labels and forecasts from Chronos-2. In stage 2, the final CycleRoutedTQNet is initialized from the anchor and trained using ground truth together with a convex target that combines predictions from the frozen anchor and a task-specific external teacher. The method is evaluated on five datasets and generally improves performance over the selected external teachers and recently published baselines.
Critical strengths
S1. Practical compression of complementary forecasters. AnchorMix provides an effective way to transfer complementary knowledge from heterogeneous forecasters into a single deployable student while removing all teacher models at inference. This directly addresses the latency, memory, and deployment overhead of runtime ensembles.
S2. Strong empirical performance. Across five datasets and four forecasting horizons, the single student improves most of the MSE/MAE cells relative to the selected external teachers and recent published baselines, including ensemble-based methods.
Critical weaknesses
W1. Reliance on dataset-specific teacher selection. AnchorMix requires selecting an external teacher that provides useful signals for each dataset. However, the paper assigns different teachers to different datasets without sufficiently explaining or validating the teacher-selection procedure. This creates a practical bottleneck when applying the method to a new dataset and makes it difficult to determine how strongly the reported performance depends on the selected teacher. Similarly, the Stage-1 anchor relies on cached Chronos-2 predictions, but the effect of using this potentially imperfect teacher signal is not sufficiently examined.
W2. Marginal improvements over the selected teachers. The performance improvements over the selected external teachers are generally very small and are not consistent across all datasets and metrics. This raises the question of whether distilling complementary forecasts provides a meaningful benefit over simply selecting and deploying a strong individual forecaster.
Questions for authors
Q1. What exact procedure was used to select the external teacher, and the other task-specific optimization settings? Please specify the detailed selection process used for each decision.
Q2. How dependent are the final results on Chronos-2 supervision in Stage 1? Could you report results when the anchor is trained without the Chronos-2 term, as well as with another time-series forecasting foundation model in its place?
Q3. How sensitive is the method to the choice of heterogeneous teacher? Could you apply both MoHETS and PMDformer—and, if possible, another reasonable teacher—to the same dataset–horizon tasks under a common train/validation-only selection protocol?
Q4. Could you provide comparisons with recent strong single-model forecasters and a simple validation-selected best-single-model baseline?
Q5. Could you evaluate the method on more diverse multivariate datasets beyond the four closely related ETT benchmarks using a standardized evaluation framework such as the TSlib?