- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents Ada-MoF, an adaptive mixture-of-experts framework in the frequency domain for time series forecasting. The authors propose an adaptive expert allocation mechanism to address the frequency coverage imbalance caused by fixed expert selection, along with Gaussian band-pass filters to mitigate the edge oscillations associated with the Gibbs phenomenon. Experiments on public benchmark datasets demonstrate competitive forecasting performance compared with seven baseline methods.
Critical strengths
-
The overall framework is straightforward and easy to follow, with a clear motivation for adaptive expert selection and frequency-domain feature decoupling.
-
The proposed method achieves promising forecasting improvements with a relatively small number of additional parameters. The resulting model remains lightweight, with approximately 0.2 million parameters in the reported configuration.
Critical weaknesses
-
Ada-MoF appears to function more as a plug-in module than as an independent forecasting model, and its methodological contribution should be more clearly positioned. Although integrating Ada-MoF into existing baselines generally improves their forecasting performance, the comparison between TimeMixer + Ada-MoF and standalone baselines in Table 2 may not be entirely fair, as the former benefits from an additional module and increased model capacity. Furthermore, the implementation details of how Ada-MoF is integrated into different forecasting backbones should be more thoroughly described to ensure reproducibility.
-
It remains unclear to what extent adaptive expert selection improves forecasting performance compared with conventional fixed Top-K selection. A controlled comparison under the same backbone and expert configurations would help isolate the contribution of the adaptive expert budget. In addition, more in-depth analyses, such as the distribution of activated experts across different samples, are needed to validate the effectiveness of the proposed mechanism. It is also unclear whether the learnable frequency partitioning actually produces frequency-specialized experts. Visualizations of the learned Gaussian filters and expert activation patterns across frequencies would provide stronger evidence for this claim.
Questions for authors
Does Equation (13) mathematically guarantee non-overlapping frequency subbands? If so, could the authors clarify how this property is achieved?
@W1
We agree that Ada-MoF is a plug-in, not a standalone forecasting model. Table 2 is a system-level comparison and cannot isolate causality.
For TimeMixer, Ada-MoF is inserted independently at every temporal scale, right after temporal embedding and before Past Decomposable Mixing (PDM). Here B is mini-batch size, L the original look-back length, V the number of input variables (called N in the implementation), s a scale index, Ls the sequence length at scale s with L0=L, D the embedding dimension (
d_model), Fs=⌊Ls/2⌋+1 the number of one-sided real-FFT bins, M the number of candidate experts, Kb,s the sample-specific number of active experts at scale s, and H the forecasting horizon. The product B⋅V means each variable of each mini-batch item is treated as one channel-independent sequence.At each scale s, input X∈RB×L×V is normalized and downsampled to X(s)∈RB×Ls×V. It is permuted to [B,V,Ls] and reshaped to [B⋅V,Ls,1], folding the variable axis into the batch axis without merging variables, then mapped to D latent channels to give E(s)∈RB⋅V×Ls×D. Transposing the last two axes yields E(s)∈RB⋅V×D×Ls, so Ada-MoF operates along the temporal axis while D holds latent embedding channels rather than the original V variables.
A real FFT along Ls gives Z(s)∈CB⋅V×D×Fs. The router summarizes ∣Z(s)∣ to predict M expert scores and the budget Kb,s, then activates the corresponding experts. Each selected Gaussian spectral filter multiplies the complex FFT coefficients, changing spectral amplitude while retaining the input phase. The gated expert spectra are combined, and an inverse real FFT of length Ls returns E(s)∈RB⋅V×D×Ls. Transposing back gives [B⋅V,Ls,D]. All scales are passed to PDM, which mixes decomposed seasonal/trend information across scales, and the prediction head maps them to the future horizon and restores the channel-independent layout, yielding [B,H,V].
As shown in Table 6 of the original appendix, TimeMixer+Ada-MoF adds only 2M parameters, which is under 0.5% of the total, yet brings about a 2% MSE improvement over TimeMixer, and Table 7 further validates its generalization by showing consistent gains when Ada-MoF is added to multiple models.
@W2:
For ETTh1-96, the four scales collectively exhibit selections from K=1 through K=5. This view exposes both sample-level and run-to-run routing variation while preserving the original counts. Adaptive K is directionally better than validation-tuned fixed K=3 and compute-matched fixed K=4 in mean MSE.
@Q1:
Equation (13) does not guarantee non-overlapping sub-bands and is not the implemented operation. The corrected positive Gaussian windows have infinite tails and are therefore softly overlapping.
For the representative ETTh1-96, the learned finest-scale Gaussian windows have a mean pairwise cosine overlap of 0.051, the measured overlaps across the four scales are approximately 0.05, 0.05, 0.06, and 0.13, with the coarsest scale selecting K=2 for 3,333 routed samples and K=3 for 16,162; since real positive windows multiply complex FFT coefficients, the expert-specific spectra retain the original phase for the inverse FFT.