acceptodds
Under review as a conference paper at ICLR 2027

WaveFM: A Wavelet-Native Pretrained Model for Industrial Forecasting

Abstract

Time-series foundation models have largely converged on a recipe of fixed-length time-domain patches and parameter counts in the hundreds of millions. On industrial telemetry, which is non-stationary and dominated by transients and harmonics, patching averages together the high-frequency structure that drives operational decisions. We introduce WaveFM, a 10M-parameter pretrained industrial forecasting model that replaces time-domain patching with tokens over a 5-level discrete wavelet decomposition, exposing six dyadic frequency bands to band-specific processing and output heads. A latent bottleneck shortens the cross-channel sequence from to tokens ( fewer token-pair interactions). On 15 industrial and public datasets, zero-shot WaveFM achieves accuracy comparable to Chronos-2-Full (0.301 vs. 0.302 MSE; geometric-mean relative MSE 0.995, 95% CI [0.983, 1.008]), including on the seven public datasets alone (0.323 vs. 0.322), with one-twelfth the parameters and faster multivariate inference (2.5 ms per window). Same-corpus, parameter-matched controls show that pretraining data alone does not reproduce this result: PatchTST trained on the same corpus reaches 0.353. On high-band-energy windows and around detected transients, WaveFM's error is 6.2% and 7.6% lower than Chronos-2-Full's, with the largest gains in the finest detail bands. WaveFM trails Chronos-2-Full on a GIFT-Eval tier outside its pretraining domain (0.319 vs. 0.302) and at longer horizons.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Time-series forecasting, wavelet tokenization, industrial forecasting, multivariate forecasting, efficient inference.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_NVYYOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents WaveFM, a time series foundation model designed for industrial forecasting. By replacing traditional time-domain patches with wavelet-based tokens and introducing frequency-aware positional embeddings, WaveFM aims to better capture transient and high-frequency patterns, making it suitable for industrial forecasting scenarios. Extensive experiments on 7 public and 8 proprietary industrial datasets demonstrate that WaveFM achieves forecasting accuracy comparable to state-of-the-art TSFMs, with fewer model parameters and faster inference.

Critical strengths
  1. The methodology is well-motivated. Compared with time-domain patching and Fourier transforms, wavelet transforms are particularly suitable for capturing transient and non-stationary signals that commonly occur in industrial systems, as their multi-resolution representation preserves both temporal localization and frequency information. The wavelet-based tokenization design is also reasonable.

  2. The experimental results, especially the comparisons with same-corpus pretrained baselines and the high-frequency error analysis, provide useful evidence of the effectiveness of the wavelet-based design. The analysis of high-frequency windows and transient events is particularly relevant to the motivation of this paper.

Critical weaknesses
  1. The presentation of the methodology needs improvement. For example, the authors should clarify how the attention mechanisms operate on different types of tokens during the intra-channel and cross-channel phases, and how the compressed channel summaries are incorporated into the subsequent intra-channel phase. In particular, it is unclear how the output of the second cross-channel phase is utilized by the final output heads. The architecture figure should also be reorganized to more accurately illustrate the information flow and the details of each phase.

  2. Some architectural choices require further justification or empirical ablation. For example, why is a 5-level DWT adopted, and why is each channel compressed into 25 tokens? Additional ablation studies on the latent bottleneck and special token designs, particularly the CLS-only re-injection mechanism, would help establish the effectiveness of these design choices.

Questions for authors
  1. What is the overall MASE performance of WaveFM on the complete GIFT-Eval benchmark, beyond the five high-frequency subsets reported in the paper?

  2. Why does WaveFM adopt a relatively long context length of 1,536 time steps? Is there any empirical evidence showing how the forecasting performance changes with different context lengths?

1 comment

Dear Authors, I could not locate the corresponding revisions in the updated PDF, as the changes do not appear to be clearly highlighted or marked. This makes it difficult to verify whether my concerns have been adequately addressed.

Reviewer_5vhUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes WaveFM, a 10M-parameter wavelet-native pretrained model for industrial time-series forecasting, motivated by the limitations of conventional time-domain patching in capturing high-frequency oscillations and transient events. The model employs a five-level discrete wavelet transform (DWT) to decompose each input signal into six frequency bands, which are tokenized using band-specific convolutional projections. These representations are processed using frequency-aware rotary positional embeddings and interleaved intra-channel and cross-channel attention. A latent bottleneck compresses each channel's representation to reduce the computational cost of cross-channel interactions while preserving fine-grained information through skip connections. The architecture further incorporates band-specific forecasting heads that combine linear predictions with gated nonlinear corrections, followed by an inverse DWT to reconstruct the time-domain forecast.

The model is pretrained on a corpus of 53 industrial and public time-series datasets and evaluated in a zero-shot setting against several pretrained foundation models and same-corpus baselines. WaveFM achieves industrial-tier MSE comparable to Chronos-2-Full (0.301 vs. 0.302), using approximately 12× fewer parameters and achieving 13.8× faster multivariate inference. The authors also report 6.2% lower error on high-frequency-energy windows and 7.6% lower error around transient events compared with Chronos-2-Full. The evaluation includes per-frequency-band error analysis, architectural ablations, domain-adapted comparisons, and experiments at longer forecasting horizons.

Critical strengths
  1. WaveFM achieves industrial-tier forecasting performance comparable to Chronos-2-Full (0.301 vs. 0.302 MSE) while using approximately 12× fewer parameters and providing 13.8× faster multivariate inference. This represents a meaningful accuracy–efficiency trade-off, particularly for industrial applications involving multiple sensor channels and latency-sensitive forecasting. The results suggest that a compact, domain-specialized architecture can remain competitive with substantially larger pretrained foundation models.
  2. The authors go beyond aggregate forecasting metrics by evaluating performance on high-frequency-energy windows, transient neighborhoods, and individual wavelet bands. WaveFM achieves 6.2% and 7.6% lower errors on high-frequency-energy windows and around transients, respectively, compared with Chronos-2-Full. Additionally, the inclusion of same-corpus, approximately parameter-matched baselines, domain-adapted comparisons, and architectural ablations strengthens the empirical evaluation by partially disentangling architectural improvements from the benefits of industrial pretraining. These experiments provide meaningful evidence supporting the practical effectiveness of the proposed approach.
Critical weaknesses
  1. The proposed approach exhibits substantial overlap with prior work, particularly WaveFM (Park et al., ICLR 2026 TSALM Workshop), which already explores continuous wavelet representations, scale-specific tokenization and attention, band-specific forecasting heads, and inverse wavelet reconstruction. While the submitted paper introduces additional mechanisms, including frequency-aware RoPE, conditioned residual correction, and latent-bottleneck cross-channel attention, it remains unclear whether these constitute a substantial methodological advance beyond existing wavelet-native architectures. The lack of a clear distinction from this closely related work weakens the paper's novelty contribution.
  2. The paper's motivation relies on the assumption that time-domain patching attenuates high-frequency information and that wavelet decomposition provides a more suitable representation for forecasting. However, learned patch embeddings do not inherently act as low-pass filters and can theoretically preserve high-frequency components. Although the authors provide raw-patching ablations and same-corpus comparisons, these experiments do not isolate whether the improvements arise specifically from wavelet decomposition or from other architectural choices, such as band-specific processing and conditioned output heads. Consequently, the central explanation for why the proposed representation improves forecasting remains insufficiently validated. 3.Despite motivating WaveFM through improved preservation of high-frequency oscillations and transient dynamics, the evaluation primarily relies on MSE, MAE, and per-band squared errors. Although the authors additionally examine high-frequency-energy windows and transient neighborhoods, these evaluations still measure pointwise prediction errors rather than directly assessing dynamical fidelity. Important characteristics such as oscillation amplitude, phase, dominant frequency, and transient onset timing are not evaluated separately. Since models with similar MSE can exhibit substantially different temporal and spectral behavior, the current results do not fully establish whether WaveFM better preserves the signal characteristics central to its motivation.
  3. WaveFM employs a fixed five-level dyadic wavelet decomposition with band-specific processing across heterogeneous industrial signals. However, the physical meaning of each frequency band depends on the sampling rate, while variations in operating speed, load, or dominant frequency can shift meaningful signal components between bands. This raises questions about whether the learned frequency-specific representations remain effective under changing operating regimes. Although the authors evaluate datasets with different sampling rates and conduct mother-wavelet ablations, they do not directly investigate robustness to controlled frequency shifts or changes in operating conditions. This limitation is particularly relevant given that non-stationarity is a central characteristic of the industrial applications motivating the method.
Questions for authors

Q1. How does the proposed WaveFM differ fundamentally from Park et al.'s WaveFM (ICLR 2026 TSALM Workshop), which also employs continuous wavelet representations, scale-specific processing, band-specific forecasting heads, and inverse DWT reconstruction? Which components constitute the primary methodological advances?

Q2. Section 1 argues that time-domain patching attenuates high-frequency information, although learned patch projections need not be low-pass filters. Given the architectural confounding in the raw-patching ablation (Section 5.3), what evidence isolates the benefit of wavelet decomposition itself?

Q3. Section 5.4 reports lower MSE on high-frequency-energy windows and around transients, but does not separately measure amplitude, phase, frequency, or event-timing errors. Do the saved predictions demonstrate improvements in these dynamical properties?

Q4. Since WaveFM uses fixed dyadic frequency bands, how robust are its predictions when changing operating conditions shift meaningful oscillations between bands? Do the existing test datasets contain such frequency or operating-regime shifts?

Q5. Table 2 reports a geometric-mean relative MSE of 0.995 with a 95% CI of [0.983, 1.008] against Chronos-2-Full, while Section 5.2 explicitly avoids claiming formal equivalence. What performance difference would the authors consider practically meaningful, and does the available evidence support the claimed comparable accuracy within that margin?

Q6. Appendix D.6 reports a single validation-selected WaveFM checkpoint, whereas the same-corpus controls report three-seed averages. Is there evidence that the observed gains, particularly the smaller architectural ablation improvements, exceed the variability associated with training initialization?

Q7. Section 3.3 reports approximately 15× fewer cross-channel token-pair interactions, while Table 5 reports substantial end-to-end inference gains. How much of the measured speedup is attributable specifically to latent compression rather than model size, implementation differences, or baseline inference strategies?

Q8. Section 3.3 compresses each channel to 24 latent queries plus CLS before cross-channel attention, while detailed band tokens bypass the bottleneck. What evidence supports the choice of 24 queries, and is performance sensitive to this compression when modeling short-lived cross-sensor dependencies?

Reviewer_BRBsOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper introduces WaveFM, a 10M-parameter forecasting model that represents signals as wavelet-frequency bands rather than time-domain patches. It achieves industrial forecasting accuracy comparable to Chronos-23-Full zero-shot, with substantially fewer parameters and faster multivariate inference.

Critical strengths
  • WaveFM represents signals as wavelet-frequency bands rather than time-domain patches.
  • It achieves industrial forecasting accuracy comparable to Chronos-23-Full zero-shot, with substantially fewer parameters and faster multivariate inference.
  • Extensive numerical experiments on evaluation benchmarks and comparisons with baselines.
Critical weaknesses
  • The proposed approach should be positioned more clearly relative to WaveFM (Park et al., 2026), published at the ICLR 2026 TSALM Workshop (https://openreview.net/forum?id=tReSUkX914). Park et al. explored a closely related idea: decomposing time series into multiple wavelet scales and learning representations through masked coefficient reconstruction, attention within and across scales, and reconstruction of the original signal through an inverse wavelet transform.

  • The authors should explicitly discuss this work and clarify the conceptual and technical differences between the two approaches. Such a comparison is necessary to assess the novelty and additional value of the proposed method.

Questions for authors
  • My main concern is the novelty of the proposed idea compared with the work published at the ICLR workshop mentioned above.
  • Is there a specific reason for fixing the time-series length at T=1536T = 1536 ?
  • Formatting: The references are not listed in alphabetical order.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.