acceptodds
Under review as a conference paper at ICLR 2027

FractalLoss: Fractal Structure of the overlooked Non-Dominant Components in Time Series Forecasting

Abstract

Despite rapid progress in time series forecasting (TSF), model training still largely relies on mean squared error (MSE), which often produces over-smoothed predictions thereby misses fine-grained temporal dynamics. We investigate this limitation from a spectral perspective and show that MSE is energy-preferential: it emphasizes high-energy frequency components while overlooking low-energy ones. Consequently, dominant spectral peaks are well fitted, whereas numerous low-energy background components remain underfitted. More importantly, extensive empirical analysis reveals that these individually weak yet collectively strong background components are not random residuals, but instead exhibit a clear fractal structure characterized by strong linearity in the log-frequency log-energy spectrum. Motivated by these observations, we propose FractalLoss, a model-agnostic regularizer that encourages the recovery of fractal structure for more faithful forecasting. We further theoretically show that FractalLoss provides energy-independent guidance, helping mitigate underfitting across background components irrespective of their spectral energy. Experiments on real-world benchmarks demonstrate forecasting gains and substantially improved fractal structure preservation.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
machine learning, fractal structure, fairness, sequence modeling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_vjxeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper identifies a fractal structure in the low-energy frequency components often overlooked by MSE-based forecasting and then proposes FractalLoss to preserve this structure during training. Experiments on seven datasets and five backbones show improvements in forecasting accuracy over baselines.

Critical strengths

This paper is well motivated and aims to solve an important problem.

FractalLoss can be applied to different forecasting models.

Critical weaknesses

FractalLoss uses a fitted straight line as the goal for all background elements. But deviations from this line may contain useful information. The paper does not discuss how this assumption might affect the predictions.

This paper emphasizes the problem of over-smoothed predictions, but it fails to show how FractalLoss can help recover local changes.

Questions for authors

Table 3 shows some comparisons against direct spectral matching. How about directly matching the log-energy of background components without fitting a fractal line?

Can FractalLoss improve forecasting performance on samples where the background spectrum does not fit a straight line particularly well?

Can you provide some quantitative results showing how well FractalLoss captures local fluctuations or sudden changes?

2 comments

Part 1/2

We apologize for the delay in submitting our rebuttal due to health issues. Thank Reviewer vjxe for the constructive feedback and helpful suggestions. Below, we address each concern and question in turn and welcome any further questions.

W1 & Q1: Effect of deviations from the fitted fractal line on predictions.

  • FractalLoss outperforms direct log-energy matching (denoted as FractalLoss∗\text{FractalLoss}^{*}). We compare the two objectives in Tables R1-1 and R1-2, FractalLoss consistently achieves lower MSE and MAE and substantially lower FSE and FFE, indicating that preserving the fractal structure of background components provides additional benefits beyond log transformation. We have updated Table 3 and Section 5.3 (line 432-434) of the manuscript accordingly.

  • Deviations from the fitted line are difficult to predict from historical observations. To better understand the performance gap between FractalLoss and FractalLoss∗\text{FractalLoss}^{*}, we investigate whether log-energy deviations can be predicted given historical observations. Specifically, we compare the history-conditioned Predictor (Fredformer) with two simple baselines that DO NOT use historical observations: (1) Zero, which always predicts zero, and (2) Mean, which predicts the training-set mean deviation at each frequency. As shown in Table R1-3, the predictor performs worst across 5 datasets, suggesting the deviations are difficult to predict given history. Zero consistently performs best, further supporting the rationale for matching the fractal line.

Table R1-1. MSE, MAE, FSE, and FFE (all ↓\downarrow) comparison between FractalLoss∗\text{FractalLoss}^{*} and FractalLoss using DLinear backbone. The results are averaged across horizons 96,192,336,720{96, 192, 336, 720}.

\hline \text{Metric} & \text{MSE} && \text{MAE} && \text{FSE} && \text{FFE} & \cr \text{Dataset} & \text{FractalLoss}^{\ast} & \text{FractalLoss} & \text{FractalLoss}^{\ast} & \text{FractalLoss} & \text{FractalLoss}^{\ast} & \text{FractalLoss} & \text{FractalLoss}^{\ast} & \text{FractalLoss} \cr \hline \text{ETTh2} & 0.428 & \mathbf{0.424} & 0.438 & \mathbf{0.428} & 0.268 & \mathbf{0.059} & 0.236 & \mathbf{0.060} \cr \text{ETTm2} & 0.336 & \mathbf{0.335} & 0.378 & \mathbf{0.371} & 0.245 & \mathbf{0.175} & 0.145 & \mathbf{0.020} \cr \text{Weather} & 0.265 & \mathbf{0.263} & 0.275 & \mathbf{0.266} & 0.248 & \mathbf{0.088} & 0.187 & \mathbf{0.069} \cr \text{Electricity} & 0.210 & \mathbf{0.191} & 0.290 & \mathbf{0.266} & 0.203 & \mathbf{0.048} & 0.133 & \mathbf{0.056} \cr \text{Exchange} & 0.337 & \mathbf{0.335} & 0.403 & \mathbf{0.402} & 0.238 & \mathbf{0.115} & 0.342 & \mathbf{0.316} \cr \hline \end{matrix}$$ **Table R1-2.** MSE, MAE, FSE, and FFE (all $\downarrow$) comparison between $\text{FractalLoss}^{*}$ and FractalLoss using **Fredformer** backbone. The results are averaged across horizons ${96, 192, 336, 720}$. $$\begin{matrix} \hline \text{Metric} & \text{MSE} && \text{MAE} && \text{FSE} && \text{FFE} & \cr \text{Dataset} & \text{FractalLoss}^{\ast} & \text{FractalLoss} & \text{FractalLoss}^{\ast} & \text{FractalLoss} & \text{FractalLoss}^{\ast} & \text{FractalLoss} & \text{FractalLoss}^{\ast} & \text{FractalLoss} \cr \hline \text{ETTh2} & 0.364 & \mathbf{0.362} & 0.399 & \mathbf{0.390} & 0.096 & \mathbf{0.060} & 0.125 & \mathbf{0.049} \cr \text{ETTm2} & 0.283 & \mathbf{0.276} & 0.321 & \mathbf{0.319} & 0.301 & \mathbf{0.160} & 0.213 & \mathbf{0.127} \cr \text{Weather} & 0.247 & \mathbf{0.245} & 0.275 & \mathbf{0.273} & 0.138 & \mathbf{0.051} & 0.145 & \mathbf{0.096} \cr \text{Electricity} & 0.177 & \mathbf{0.176} & 0.279 & \mathbf{0.274} & 0.125 & \mathbf{0.100} & 0.095 & \mathbf{0.079} \cr \text{Exchange} & 0.349 & \mathbf{0.343} & 0.396 & \mathbf{0.395} & 0.129 & \mathbf{0.087} & 0.114 & \mathbf{0.028} \cr \hline \end{matrix}$$ **Table R1-3.** log-energy deviation MAE $\downarrow$ comparison between the history-conditioned predictor and two history-independent baselines on 96 forecasting horizon. $$\begin{matrix} \hline \text{Method} & \text{ETTh2} & \text{ETTm2} & \text{Weather} & \text{Electricity} & \text{Exchange} \\\\ \hline \text{Zero} & \mathbf{1.544} & \mathbf{1.476} & \mathbf{1.529} & \mathbf{1.585} & \mathbf{1.092} \\\\ \text{Mean} & 1.596 & 1.529 & 1.546 & 1.613 & 1.152 \\\\ \text{Predictor} & 1.660 & 1.645 & 1.707 & 1.716 & 1.318 \\\\ \hline \end{matrix}$$

Part 2/2

W1 & Q2: Performance on samples where the background spectrum does not fit a straight line particularly well

Thank you for the insightful feedback. Within each dataset, we ranked test samples by the R2R^2 of the fractal line fit and evaluated the bottom 10% ones, the results are shown in Table R2-1.

  • FractalLoss still improves forecasting performance on these samples with relatively weak fractal structure. Compared with MSE Loss, adding FractalLoss reduces forecasting MSE by 0.55% to 3.74% across the five datasets.
  • The gains are smaller when the fitting R2R^2 is lower. Among the bottom-10% subsets, FractalLoss reduces MSE by only 0.55%–0.92% on Weather and Electricity, which have the lowest R2R^2, but by 3.74% on Exchange, which has the highest R2R^2. This phenomenon is consistent with the conclusion from Table R1-3: deviations from the fitted line are difficult to predict given history, so matching the line may offer less benefit when the deviations account for a larger share (lower fitting R2R^2).

Table R2-1. Forecasting MSE (↓\downarrow) with and without FractalLoss on the 10% test samples with the lowest R2R^2. The forecasting horizon is 96.

\hline \text{Dataset} & \text{Samples} & R^2\text{ Range} & \text{w/o FractalLoss} & \text{w/ FractalLoss} \cr \hline \text{ETTh2} & 279 & [0.785, 0.792] & 0.325 & \mathbf{0.314} \cr \text{ETTm2} & 1143 & [0.803, 0.841] & 0.276 & \mathbf{0.268} \cr \text{Weather} & 1045 & [0.674, 0.708] & 0.201 & \mathbf{0.200} \cr \text{Electricity} & 517 & [0.609, 0.722] & 0.183 & \mathbf{0.181} \cr \text{Exchange} & 143 & [0.814, 0.863] & 1.202 & \mathbf{1.157} \cr \hline \end{matrix}$$ # W2 & Q3: Quantitative results for local fluctuations Thank you for the comment, we have introduced two additional metrics to describe local fluctuations or sudden change, the results are shown in **Table R2-2.** - **First-order difference MAE**: This metric measures the direction and magnitude of changes between consecutive time steps. Existing work [1-2] use it to capture local fluctuations in time series. For a forecast $\hat{y}$ and target $y$ over a horizon of $H$ steps, we compute: $$\frac{1}{H-1}\sum_{t=1}^{H-1} \left|(\hat{y}_{t+1}-\hat{y}_t)-(y_{t+1}-y_t)\right|.$$ > [1] TDT Loss Takes It All: Integrating Temporal Dependencies among Targets into Non-Autoregressive Time Series Forecasting. Xiong et al., 2024. > [2] LETS Forecast: Learning Embedology for Time Series Forecasting. Majeedi et al., 2025. - **Ramp Score**: It captures the direction and steepness of local rises and falls. Prior studies [3-4] use it to evaluate sudden changes. For a forecast $\hat{y}$ and target $y$ over a horizon of $H$ steps, let $s_t(x)$ denote the slope over $[t,t+1]$ in the piecewise-linear approximation, we compute: $$\frac{1}{H-1}\sum_{t=1}^{H-1} \left|s_t(\hat{y})-s_t(y)\right|.$$ > [3] Shape and Time Distortion Loss for Training Deep Time Series Forecasting Models. Le Guen and Thome, 2019. > [4] All sky imaging-based short-term solar irradiance forecasting with Long Short-Term Memory networks. Hendrikx et al., 2024. - **FractalLoss recovers local fluctuations better than MSE-only training.** As shown in **Table R2-2**, adding FractalLoss lowers first-order difference MAE in all 10 dataset–backbone combinations and Ramp Score in 9 of 10, suggesting that it **alleviates over-smoothing prediction.** **Table R2-2.** Ramp Score and first-order difference MAE (both $\downarrow$) with and without FractalLoss. The results are averaged across horizons ${96, 192, 336, 720}$. $$\begin{matrix} \hline \text{Backbone} & \text{Metric} & \text{Training loss} & \text{ETTh2} & \text{ETTm2} & \text{Weather} & \text{Electricity} & \text{Exchange} \cr \hline \text{Fredformer} & \text{Ramp Score}\downarrow & \text{MSE} & 0.212 & 0.191 & 0.091 & \mathbf{0.147} & 0.136 \cr & & \text{+FractalLoss} & \mathbf{0.208} & \mathbf{0.187} & \mathbf{0.088} & 0.148 & \mathbf{0.125} \cr & \text{First-order difference MAE}\downarrow & \text{MSE} & 0.168 & 0.101 & 0.082 & 0.149 & 0.158 \cr & & \text{+FractalLoss} & \mathbf{0.164} & \mathbf{0.097} & \mathbf{0.074} & \mathbf{0.144} & \mathbf{0.143} \cr \hline \text{DLinear} & \text{Ramp Score}\downarrow & \text{MSE} & 0.259 & 0.189 & 0.087 & 0.144 & 0.123 \cr & & \text{+FractalLoss} & \mathbf{0.256} & \mathbf{0.183} & \mathbf{0.085} & \mathbf{0.140} & \mathbf{0.119} \cr & \text{First-order difference MAE}\downarrow & \text{MSE} & 0.162 & 0.095 & 0.078 & 0.143 & 0.144 \cr & & \text{+FractalLoss} & \mathbf{0.159} & \mathbf{0.090} & \mathbf{0.073} & \mathbf{0.140} & \mathbf{0.131} \cr \hline \end{matrix}$$
Reviewer_RxhbOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FractalLoss, a model-agnostic auxiliary objective for time series forecasting. The authors first express MSE in the Fourier domain and observe that, at equal relative prediction error, higher-energy frequency components contribute more strongly to the loss. They then report that background components, defined by excluding local spectral maxima, approximately follow a power-law spectral envelope. FractalLoss encourages predicted background log-energies to match a straight line fitted to the ground-truth log-frequency–log-energy spectrum and combines this regularizer with temporal MSE. Experiments cover seven datasets, five forecasting backbones, and four prediction horizons; the authors report MSE improvements in 33 of 35 dataset–backbone combinations alongside substantial reductions in two proposed spectral-structure metrics (FSE, FFE).

Critical strengths
  1. Simple, architecture-independent formulation\textbf{Simple, architecture-independent formulation}. FractalLoss can be incorporated into existing supervised forecasting pipelines without changing the forecasting architecture or requiring an additional inference-time module. The fitted target is low-dimensional and easy to interpret.

  2. Broad evaluation across forecasting backbones\textbf{Broad evaluation across forecasting backbones}. Table 1 evaluates five substantially different backbones on seven datasets. The reported improvements are reasonably consistent across these settings, providing stronger evidence of practical usefulness than a single-backbone evaluation would.

  3. Evaluation beyond pointwise forecasting error\textbf{Evaluation beyond pointwise forecasting error}. This paper examines spectral properties of predictions rather than relying exclusively on MSE and MAE. Although the proposed metrics have limitations, this diagnostic perspective is valuable for understanding how auxiliary objectives affect predicted signals.

Critical weaknesses
  1. The evidence does not establish the claimed distinction between fractal structure and random residuals.

Sections 1 and 4 assert that background components are "not random noise" because their spectra are log-log linear. Power-law spectra arise from AR(1), fBm, random walks, and colored noise, so spectral regularity does not imply non-randomness or predictability. Appendix A.4 only derives the power law from an assumed scale invariance—it does not validate that assumption on the data. No colored-noise, random-walk, detrending, or tapering controls are provided. Fig. 4(e–f) shows background spectra are stable across windows, which fixed-shape colored noise also produces; stability is not evidence of structure.

  1. The theoretical interpretation exceeds what the propositions establish.

Prop. 1's "energy preference" exists only in the relative-error coordinate ρk\rho_k; in the absolute form of Eq. 6, all frequencies are equally weighted, so no preferential learning dynamics is shown. Prop. 2 establishes energy-independent sensitivity to relative energy error ξk\xi_k—not gradients w.r.t. Fourier coefficients or model parameters—and ϵ=10−8\epsilon=10^{−8} amplifies gradients near zero energy. The claim that FractalLoss "distribut[es] corrective pressure across backgrounds" is therefore not established.

  1. The ablations do not isolate the contribution of the fractal-line target.

The direct-matching baseline changes both the target representation and the loss geometry relative to FractalLoss. A particularly important missing comparison is direct background log-energy matching: $\mathcal{L}_{\mathrm{log}}=\frac{1}{|I_B|}\sum_{k\in I_B} \left[\log(\widehat E_k+\epsilon)-\log(E_k+\epsilon)\right]^2$. This baseline would test whether fitting a power-law envelope contributes beyond the relative-error emphasis induced by logarithms. Additional comparisons with energy-normalized spectral losses and a flexible smoothed spectral envelope would further clarify the source of improvement.

  1. There are material inconsistencies in the experimental reporting.

Table 4 states five-seed averaging, yet Fredformer/ETTh2/τ\tau=192 baseline FSE is 0.432±0.007 in Table 5 but 0.172 in Table 6; FractalLoss/Exchange/τ=96 FSE is 0.241±0.000 vs. 0.052. These gaps exceed any std and cannot be due to rounding. Until reconciled, the FSE/FFE improvements—a main selling point—are not reproducible as reported.

  1. FSE/FFE cannot support "structure recovery," and admit a counterexample.

FSE measures only slope error; FFE only R2R^2 error. Neither measures intercept or phase. Ignoring ϵ\epsilon, a prediction y^=cy(c≠1)\hat{\mathbf{y}}=c\mathbf{y}(c\neq 1) leaves slope and R2R^2 unchanged, so FSE/FFE are near zero while MSE is arbitrarily large. Also, the conditional mean can legitimately be smoother than realizations; the paper never analyzes when matching realization-level spectral energy helps or harms.

  1. Several methodological and evaluation details remain underspecified.

Eq. 1 defines a full DFT index set, but Figs. 2–3 plot a one-sided spectrum; k>τ/2k>\tau/2 bins are mirrored negative frequencies, and treating them as positive changes the fitted slope and R2R_2. The paper does not state retained bins, DC/Nyquist handling, or normalization—yet Fig. 2 and FSE/FFE depend on exactly these choices. Additionally, "peak" (local maxima within radius rr, Eqs. 8–9) is conflated with "dominant" (globally high energy); tuning budgets for baselines, runtime/memory overhead, and significance tests on the main tables are all unreported.

Questions for authors

See weaknesses.

3 comments

part 1/3

We apologize for submitting our rebuttal late due to health issues. We thank Reviewer Rxhb for the constructive comments. Below, we address each weakness and question in turn.

W1: Evidence of fractal structure

  • "Random noise" refers to white noise. We used "random noise" as shorthand for white noise to make the paper easy to read. We have revised Sections 1(line 19-21) and 4 (line 237-239) to use "white noise" explicitly.
  • FractalLoss uses the observed spectral pattern regardless of the underlying process. We use fractal structure to describe the approximate linear relationship between log frequency and log energy in the background spectrum. FractalLoss fits this line and uses it as the target for the energy-independent supervision described in Proposition 2. The same procedure applies whether the observed pattern arises from an AR(1) process, fractional Brownian motion, a random walk, or colored noise. For example, colored noise can have random realizations while its power spectrum specifies how energy is distributed across frequencies on average. This regularity provides a target for supervising the forecast’s background energy even when individual fluctuations are unpredictable.

W2: Theoretical Justification

  • Proposition 1. Equal weighting in Eq. 6 is consistent with the energy preference we describe. If a forecast underestimates two Fourier coefficients by the same fraction, Y^k=(1−δ)Yk\hat Y_k=(1-\delta)Y_k, each contributes ∣Y^k−Yk∣2=δ2∣Yk∣2|\hat Y_k-Y_k|^2=\delta^2|Y_k|^2 to the squared error. Correcting the coefficient with ten times the energy therefore reduces MSE ten times as much. This is a property of the objective. The rate at which a network learns either coefficient also depends on the model and optimizer.

  • Proposition 2. For a fitted target energy Ek⋆>0E_k^\star>0 and predicted energy E^k=cEk⋆\hat E_k=cE_k^\star, the log-energy error is log⁡E^k−log⁡Ek⋆=log⁡c\log \hat E_k-\log E_k^\star=\log c. Thus, correcting the same multiplicative error yields the same reduction in the log-energy penalty at every background frequency. The larger derivative near zero is consistent with this result: a fixed absolute change represents a larger relative change at lower energy. We will replace “distributes corrective pressure” with “assigns equal penalties to equal multiplicative energy errors,” which states the result directly without implying equal parameter gradients.

part 2/3

W3: Comparison with direct log-energy matching and existing spectral losses

  • FractalLoss vs. direct log-energy matching: We present the comparison in Table R2-1 and Table R2-2. The results support that preserving the fractal structure of background components provides additional benefits beyond log transformation. We have updated Table 3 (page 8) and Section 5.3 (line 432-434) of the manuscript accordingly.

  • FractalLoss vs. existing spectal losses: we compared FractalLoss with FFL [1], BSP [3], and FreIE [4] in Table R2-3.

Table R2-1. MSE, MAE, FSE, and FFE (all ↓\downarrow) comparison between FractalLoss∗\text{FractalLoss}^{*} and FractalLoss using DLinear backbone.

\hline \text{Metric} & \text{MSE} && \text{MAE} && \text{FSE} && \text{FFE} & \cr \text{Dataset} & \text{FractalLoss}^{\ast} & \text{FractalLoss} & \text{FractalLoss}^{\ast} & \text{FractalLoss} & \text{FractalLoss}^{\ast} & \text{FractalLoss} & \text{FractalLoss}^{\ast} & \text{FractalLoss} \cr \hline \text{ETTh2} & 0.428 & \mathbf{0.424} & 0.438 & \mathbf{0.428} & 0.268 & \mathbf{0.059} & 0.236 & \mathbf{0.060} \cr \text{ETTm2} & 0.336 & \mathbf{0.335} & 0.378 & \mathbf{0.371} & 0.245 & \mathbf{0.175} & 0.145 & \mathbf{0.020} \cr \text{Weather} & 0.265 & \mathbf{0.263} & 0.275 & \mathbf{0.266} & 0.248 & \mathbf{0.088} & 0.187 & \mathbf{0.069} \cr \text{Electricity} & 0.210 & \mathbf{0.191} & 0.290 & \mathbf{0.266} & 0.203 & \mathbf{0.048} & 0.133 & \mathbf{0.056} \cr \text{Exchange} & 0.337 & \mathbf{0.335} & 0.403 & \mathbf{0.402} & 0.238 & \mathbf{0.115} & 0.342 & \mathbf{0.316} \cr \hline \end{matrix}$$ **Table R2-2.** MSE, MAE, FSE, and FFE (all $\downarrow$) comparison between $\text{FractalLoss}^{*}$ and FractalLoss using **Fredformer** backbone. $$\begin{matrix} \hline \text{Metric} & \text{MSE} && \text{MAE} && \text{FSE} && \text{FFE} & \cr \text{Dataset} & \text{FractalLoss}^{\ast} & \text{FractalLoss} & \text{FractalLoss}^{\ast} & \text{FractalLoss} & \text{FractalLoss}^{\ast} & \text{FractalLoss} & \text{FractalLoss}^{\ast} & \text{FractalLoss} \cr \hline \text{ETTh2} & 0.364 & \mathbf{0.362} & 0.399 & \mathbf{0.390} & 0.096 & \mathbf{0.060} & 0.125 & \mathbf{0.049} \cr \text{ETTm2} & 0.283 & \mathbf{0.276} & 0.321 & \mathbf{0.319} & 0.301 & \mathbf{0.160} & 0.213 & \mathbf{0.127} \cr \text{Weather} & 0.247 & \mathbf{0.245} & 0.275 & \mathbf{0.273} & 0.138 & \mathbf{0.051} & 0.145 & \mathbf{0.096} \cr \text{Electricity} & 0.177 & \mathbf{0.176} & 0.279 & \mathbf{0.274} & 0.125 & \mathbf{0.100} & 0.095 & \mathbf{0.079} \cr \text{Exchange} & 0.349 & \mathbf{0.343} & 0.396 & \mathbf{0.395} & 0.129 & \mathbf{0.087} & 0.114 & \mathbf{0.028} \cr \hline \end{matrix}$$ **Table R2-3.** MSE, MAE, FSE, and FFE (all $\downarrow$) for 4 training objectives across five datasets. $$\begin{matrix} \hline \text{Metric} & \text{MSE}\downarrow & \text{MAE}\downarrow & \text{FSE}\downarrow & \text{FFE}\downarrow \cr \begin{matrix} \hline \text{Dataset} \cr \hline \text{ETTh2} \cr \text{ETTm2} \cr \text{Weather} \cr \text{Electricity} \cr \text{Exchange} \end{matrix} & \begin{matrix} \hline \text{FFL} & \text{BSP} & \text{FreIE} & \text{FractalLoss} \cr \hline 0.380 & 0.371 & 0.373 & \mathbf{0.362} \cr 0.285 & 0.280 & 0.282 & \mathbf{0.276} \cr 0.251 & 0.248 & 0.249 & \mathbf{0.245} \cr 0.179 & 0.177 & 0.180 & \mathbf{0.176} \cr 0.356 & 0.349 & 0.351 & \mathbf{0.343} \end{matrix} & \begin{matrix} \hline \text{FFL} & \text{BSP} & \text{FreIE} & \text{FractalLoss} \cr \hline 0.401 & 0.398 & 0.400 & \mathbf{0.390} \cr 0.331 & 0.329 & 0.326 & \mathbf{0.319} \cr 0.274 & 0.282 & 0.289 & \mathbf{0.273} \cr 0.278 & 0.276 & 0.277 & \mathbf{0.274} \cr 0.406 & 0.403 & 0.403 & \mathbf{0.395} \end{matrix} & \begin{matrix} \hline \text{FFL} & \text{BSP} & \text{FreIE} & \text{FractalLoss} \cr \hline 0.199 & 0.173 & 0.195 & \mathbf{0.060} \cr 0.452 & 0.467 & 0.365 & \mathbf{0.160} \cr 0.237 & 0.239 & 0.231 & \mathbf{0.051} \cr 0.269 & 0.298 & 0.258 & \mathbf{0.100} \cr 0.163 & 0.147 & 0.144 & \mathbf{0.087} \end{matrix} & \begin{matrix} \hline \text{FFL} & \text{BSP} & \text{FreIE} & \text{FractalLoss} \cr \hline 0.287 & 0.268 & 0.381 & \mathbf{0.049} \cr 0.249 & 0.229 & 0.204 & \mathbf{0.127} \cr 0.246 & 0.396 & 0.132 & \mathbf{0.096} \cr 0.198 & 0.223 & 0.273 & \mathbf{0.079} \cr 0.168 & 0.159 & 0.137 & \mathbf{0.028} \end{matrix} \cr \hline \end{matrix}$$ > [1] Focal Frequency Loss for Image Reconstruction and Synthesis. Jiang et al., 2021. > [2] Tuning Frequency Bias in Neural Network Training with Nonuniform Data. Yu et al., 2023. > [3] Binned Spectral Power Loss for Improved Prediction of Chaotic Systems. Chakraborty et al., 2025. > [4] FreIE: Low-Frequency Spectral Bias in Neural Networks for Time-Series Tasks. Sun et al., 2025.

part 3/3

W4: Experimental reporting

We apologize for the confusion. Table 6 reports forecasting results for all horizons. We ran these experiments alongside those in Table 1 using the default peak-selection radius r=5r=5. We ran the five-seed experiments in Table 5 after the hyperparameter sensitivity study and inadvertently retained r=19r=19, the largest radius tested. Changing the radius changes which frequencies enter the background fit and therefore affects both FSE and FFE. We have updated Table 5 of the manuscript with five-seed results using r=5r=5, consistent with Table 6.

W5: FractalLoss already consider the fitted intercept and MSELoss maintain phase

  • FractalLoss accounts for both the slope and the intercept during training. Eq.13 of the manuscript defines FractalLoss as:
= \frac{1}{|I_B|}\sum_{k\in I_B}\left[\hat{v}_k-L(u_k)\right]^2 = \frac{1}{|I_B|}\sum_{k\in I_B}\left[\hat{v}_k+\alpha u_k-\beta\right]^2.$$ At each background frequency, the slope $-\alpha$ determines how the target log-energy varies with frequency, and the intercept $\beta$ sets its overall level. FractalLoss thus encourages complete structure fidelity. - **FractalLoss supervises energy but our overall Training Objective includes MSE:** As shown in **Eq. 15** of the manuscript, our objective combines FractalLoss and temporal MSE supervision, which contains the phase information. $$\mathcal{L}_{total}=\lambda \mathcal{L}_{fractal}+ (1- \lambda)\mathcal{L}_{MSE}$$ **FractalLoss also improves recovery of the fitted intercept.** To evaluate how accurately forecasts reproduce the level of the background spectrum, we introduce Fractal Intercept Error (FIE), the mean absolute difference between the predicted and ground-truth fitted intercepts: $$\mathrm{FIE}=\frac{1}{N}\sum_{n=1}^{N}\left|\hat{\beta}^{(n)}-\beta^{(n)}\right|.$$ As shown in the **Table R3-1**, adding FractalLoss to MSE reduces FIE in all 35 dataset–backbone combinations. Together with the FSE and FFE results in **Table 1** of the manuscript, this supports that FractalLoss improves fidelity to the fitted fractal line. **Table R3-1.** Fractal Intercept Error (FIE $\downarrow$) with and without FractalLoss across 7 datasets and 5 backbones. $$\begin{matrix} \hline \text{Backbone} & \text{iTransformer} & \text{PatchTST} & \text{DLinear} & \text{Fredformer} & \text{PhaseFormer} \cr \begin{matrix} \hline \text{Dataset} \cr \hline \text{ETTh1} \cr \text{ETTh2} \cr \text{ETTm1} \cr \text{ETTm2} \cr \text{Weather} \cr \text{Electricity} \cr \text{Exchange} \end{matrix} & \begin{matrix} \hline \text{MSE} & \text{+FractalLoss} \cr \hline 1.128 & \mathbf{0.065} \cr 1.053 & \mathbf{0.159} \cr 1.689 & \mathbf{0.108} \cr 1.847 & \mathbf{0.044} \cr 1.490 & \mathbf{0.043} \cr 0.406 & \mathbf{0.119} \cr 0.305 & \mathbf{0.055} \end{matrix} & \begin{matrix} \hline \text{MSE} & \text{+FractalLoss} \cr \hline 0.831 & \mathbf{0.144} \cr 1.016 & \mathbf{0.156} \cr 1.764 & \mathbf{0.132} \cr 1.762 & \mathbf{0.053} \cr 1.321 & \mathbf{0.131} \cr 0.308 & \mathbf{0.109} \cr 0.338 & \mathbf{0.055} \end{matrix} & \begin{matrix} \hline \text{MSE} & \text{+FractalLoss} \cr \hline 0.918 & \mathbf{0.056} \cr 0.899 & \mathbf{0.080} \cr 1.592 & \mathbf{0.070} \cr 1.909 & \mathbf{0.068} \cr 1.624 & \mathbf{0.068} \cr 0.343 & \mathbf{0.093} \cr 0.389 & \mathbf{0.145} \end{matrix} & \begin{matrix} \hline \text{MSE} & \text{+FractalLoss} \cr \hline 0.859 & \mathbf{0.104} \cr 0.827 & \mathbf{0.026} \cr 1.703 & \mathbf{0.050} \cr 1.926 & \mathbf{0.136} \cr 1.363 & \mathbf{0.072} \cr 0.533 & \mathbf{0.114} \cr 0.390 & \mathbf{0.086} \end{matrix} & \begin{matrix} \hline \text{MSE} & \text{+FractalLoss} \cr \hline 1.017 & \mathbf{0.087} \cr 0.781 & \mathbf{0.054} \cr 1.899 & \mathbf{0.081} \cr 1.577 & \mathbf{0.084} \cr 1.437 & \mathbf{0.142} \cr 0.474 & \mathbf{0.028} \cr 0.410 & \mathbf{0.121} \end{matrix} \cr \hline \end{matrix}$$ - All deterministic time series forecasting methods produce a single point forecast, which under MSE targets the conditional mean. Modeling the distribution of possible future realizations falls within probabilistic time series forecasting and is beyond the scope of this paper. # W6: Methodological and evaluation details -  **One-sided spectrum**: Using rFFT to obtain a one-sided spectrum is standard practice for real-valued time series because their Fourier coefficients satisfy conjugate symmetry, $X_{\tau-k}=\overline{X_k}$. Each negative-frequency bin therefore has the same energy as its positive-frequency counterpart. - **Mismatch between global energy thresholds and fractal structure in real data.** Many high-frequency local peaks lie well above the fitted fractal line despite their small share of spectral energy, while some low-frequency non-peak components carry more energy yet closely follow the line. Both patterns can be seen in **Figure 3(b)** of the manuscript.
Reviewer_hS6jOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes FractalLoss, a model-agnostic regularizer for time series forecasting. The authors use Parseval’s theorem to express MSE in the frequency domain and argue that high energy components receive greater emphasis at equal relative error. They then observe approximately power law spectra among background components, defined as frequencies that are not local energy maxima. FractalLoss encourages predicted background log energies to match a line fitted to the target background spectrum. Experiments across seven datasets and five backbones show generally improved forecasting accuracy and substantial improvements in the proposed spectral metrics.

Critical strengths
  1. Encouraging empirical breadth. Table 1 evaluates 35 dataset–backbone combinations and reports lower MSE in 33 cases. Improvements across substantially different backbones suggest that the method has practical potential beyond one architecture.

  2. A natural spectral formulation. Fourier analysis is motivated by the exact Parseval connection to MSE and by the observed spectral envelopes. FFTs and closed-form line fitting also give the method favorable computational complexity.

Critical weaknesses
  1. The contribution needs clearer positioning against existing loss-based approaches to spectral bias. Modifying the training objective to improve uneven spectral learning has substantial precedent. Please refer to [1]-[4]. These methods differ in assumptions and application domains, and do not invalidate the proposed approach. However, they establish that loss-based mitigation of spectral bias and spectral-energy supervision are existing directions. The paper should identify its specific advance as the combination of background selection, log-energy supervision, and a fitted power-law target. FreIE is already cited in the manuscript but is not evaluated among the alternative objectives in Table 2.

  2. The energy-preference explanation is not separated from spectral bias. Proposition 1 correctly establishes greater MSE sensitivity to high-energy components at equal relative error. However, MSE weights absolute squared Fourier errors uniformly, up to normalization. A relative-error reparameterization does not establish preferential convergence during network training. When low frequencies carry more energy, frequency-dependent learning and energy-dependent error contributions can produce similar observations. Controlled experiments that vary amplitude and frequency independently are needed to distinguish these explanations. The paper should also clarify the connection between frequency over a learned function’s input domain and temporal frequency within the forecast output.

  3. The formal definitions do not establish a low-energy background. Section 4.1 defines peaks through local energy maximality and backgrounds as all remaining components. There is no energy threshold defining “low energy.” A background component can have greater energy than a peak elsewhere in the spectrum. The theoretical argument concerns energy, while the method selects components using local prominence.

  4. The structural metrics support a narrower conclusion than temporal-detail recovery. FractalLoss supervises energy rather than phase. Matching a spectral envelope does not ensure correct timing of fluctuations. FSE and FFE also omit the fitted intercept, so they measure selected spectral properties rather than complete structural fidelity. Additional phase-sensitive or temporal-difference metrics would better support claims about recovering abrupt events and fine temporal dynamics.

[1] Liming Jiang, Bo Dai, Wayne Wu, and Chen Change Loy. Focal Frequency Loss for Image Reconstruction and Synthesis. ICCV, 2021. [2] Annan Yu, Yunan Yang, and Alex Townsend. Tuning Frequency Bias in Neural Network Training with Nonuniform Data. ICLR, 2023. [3] Dibyajyoti Chakraborty, Arvind T. Mohan, and Romit Maulik. Binned Spectral Power Loss for Improved Prediction of Chaotic Systems. arXiv:2502.00472, 2025. [4] Jialong Sun, Xinpeng Ling, Jiaxuan Zou, Jiawen Kang, and Kejia Zhang. FreIE: Low-Frequency Spectral Bias in Neural Networks for Time-Series Tasks. arXiv:2510.25800, 2025.

Questions for authors
  1. How does it compare with FreLE [4] and established adaptive frequency or spectral-energy objectives [1,3]?
  2. Do stronger background power law fits predict larger forecasting improvements? What are the principal failure cases?
6 comments

Part 1/6

We apologize for submitting our rebuttal late due to health issues. We thank Reviewer hS6j for the insightful feedback. Below, we address each weakness and question in turn.

W1 & Q1: Contribution clarification and comparison with existing work [1-4]

Thank you for pointing us to these work, I would like to clarify our contribution differs in two respects.

  • Different Spectral bias definition. In [1–4], spectral bias refers to a neural network’s tendency to learn low-frequency components more readily than high-frequency ones. The papers describe this preference as follows: -[1] Jiang et al describe "a learning bias of neural networks towards low-frequency functions" (page 1) -[2] Yu et al. state that gradient-based algorithms "minimize the low-frequency misfit before reducing the high-frequency residuals" (page 1) -[3] Chakraborty et al. describe the tendency to "learn lower frequencies faster and more effectively" (page 3, section 2.1) -[4] Sun et al. write that “models tend to fit low-frequency signals before high-frequency ones” (page 1) Our work focuses on MSE’s preference for high-energy frequencies in the forecast spectrum as shown in Proposition 1 of the manuscript. Thus, our work focus on energy-dependent while existing works solve frequency-dependent problem.

  • The fractal structure characteristic of real time series. A central contribution of our work is the finding that fractal structure underlying the real-world time series. This structure can even remain relatively stable even in nonstationary data as shown in Fig. 4(e–f) and Section 5.4 of the manuscript. Energy-independent supervision is an additional property based on the structure (Proposition 2). To the best of our knowledge, existing spectral learning work didn't discuss the fractal structure.

  • Additional training objectives comparison. Since [2] has no publicly official code, we compared FractalLoss with FFL [1], BSP [3], and FreIE [4]. FractalLoss achieves the lowest MSE, MAE, FSE, and FFE on all five datasets as shown in Table R1-1, we have updated the Table 2 and Section 5.2 (line 410-417) of the pdf.

Table R1-1. MSE, MAE, FSE, and FFE (all ↓\downarrow) for 4 training objectives across five datasets. Results are averaged across forecast horizons {96,192,336,720}\{96, 192, 336, 720\}.

\hline \text{Metric} & \text{MSE}\downarrow & \text{MAE}\downarrow & \text{FSE}\downarrow & \text{FFE}\downarrow \\\\ \begin{matrix} \hline \text{Dataset} \\\\ \hline \text{ETTh2} \\\\ \text{ETTm2} \\\\ \text{Weather} \\\\ \text{Electricity} \\\\ \text{Exchange} \end{matrix} & \begin{matrix} \hline \text{FFL} & \text{BSP} & \text{FreIE} & \text{FractalLoss} \\\\ \hline 0.380 & 0.371 & 0.373 & \mathbf{0.362} \\\\ 0.285 & 0.280 & 0.282 & \mathbf{0.276} \\\\ 0.251 & 0.248 & 0.249 & \mathbf{0.245} \\\\ 0.179 & 0.177 & 0.180 & \mathbf{0.176} \\\\ 0.356 & 0.349 & 0.351 & \mathbf{0.343} \end{matrix} & \begin{matrix} \hline \text{FFL} & \text{BSP} & \text{FreIE} & \text{FractalLoss} \\\\ \hline 0.401 & 0.398 & 0.400 & \mathbf{0.390} \\\\ 0.331 & 0.329 & 0.326 & \mathbf{0.319} \\\\ 0.274 & 0.282 & 0.289 & \mathbf{0.273} \\\\ 0.278 & 0.276 & 0.277 & \mathbf{0.274} \\\\ 0.406 & 0.403 & 0.403 & \mathbf{0.395} \end{matrix} & \begin{matrix} \hline \text{FFL} & \text{BSP} & \text{FreIE} & \text{FractalLoss} \\\\ \hline 0.199 & 0.173 & 0.195 & \mathbf{0.060} \\\\ 0.452 & 0.467 & 0.365 & \mathbf{0.160} \\\\ 0.237 & 0.239 & 0.231 & \mathbf{0.051} \\\\ 0.269 & 0.298 & 0.258 & \mathbf{0.100} \\\\ 0.163 & 0.147 & 0.144 & \mathbf{0.087} \end{matrix} & \begin{matrix} \hline \text{FFL} & \text{BSP} & \text{FreIE} & \text{FractalLoss} \\\\ \hline 0.287 & 0.268 & 0.381 & \mathbf{0.049} \\\\ 0.249 & 0.229 & 0.204 & \mathbf{0.127} \\\\ 0.246 & 0.396 & 0.132 & \mathbf{0.096} \\\\ 0.198 & 0.223 & 0.273 & \mathbf{0.079} \\\\ 0.168 & 0.159 & 0.137 & \mathbf{0.028} \end{matrix} \\\\ \hline \end{matrix}$$ > [1] Focal Frequency Loss for Image Reconstruction and Synthesis. Jiang et al., 2021. > [2] Tuning Frequency Bias in Neural Network Training with Nonuniform Data. Yu et al., 2023. > [3] Binned Spectral Power Loss for Improved Prediction of Chaotic Systems. Chakraborty et al., 2025. > [4] FreIE: Low-Frequency Spectral Bias in Neural Networks for Time-Series Tasks. Sun et al., 2025.

part 2/6

W2: Controlled experiments to distinguish frequency-dependent and energy-dependent

We thank the reviewer for highlighting the two possible explanations regarding the similar observations. To distinguish them, we generate two synthetic datasets with the same three peak frequencies but reversed the order of their amplitudes.

  • Dataset Design. Each dataset containing 2,000 samples of 192 time steps for 96→9696\rightarrow 96 forecasting task. Each sample is generated by the sum of three sinusoids and Gaussian white noise:
= \sum_{j=1}^{3} A_j^{(d)} \sin\!\left(\frac{2\pi k_j t}{96}+\phi_j\right) + \varepsilon_{n,t}^{(d)}, \qquad \varepsilon_{n,t}^{(d)} \overset{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0,0.5^2).$$ Here, $x_{n,t}^{(d)}$ denotes the value at time step $t \in \{0,\ldots,191\}$ of sample $n \in \{1,\ldots,2000\}$ in dataset $d \in \{1,2\}$. The three peak frequency component $(k_1,k_2,k_3)=(8,24,40)$ position are the same in both datasets while their amplitudes $(A_1^{(1)},A_2^{(1)},A_3^{(1)}) =(9,3,1)$ and $(A_1^{(2)},A_2^{(2)},A_3^{(2)})=(1,3,9)$, swapping the amplitudes order. The initial phase $\phi_{n,j}^{(d)}$ is draw from $\mathrm{Uniform}(0,2\pi)$. - **Model shows preferential convergence on high-energy components.** We trained Fredformer using MSE Loss on each dataset and measured the relative error at each peak after every epoch. Given prediction $\hat{y}$ and its ground truth $y$, the relative error at frequency bin $k$ as $$\Delta_k = \frac{\left|\widehat{Y}_k - Y_k\right|}{\left|Y_k\right|}, \qquad k \in \{8,24,40\},$$ where $\widehat{Y}_k$ and $Y_k$ are the $k$-th Fourier component of $\hat{y}$ and $y$, respectively. The relative error records are shown in **Table R2-1**, the peak with the largest amplitude has the lowest relative error at every epoch in both datasets. This holds whether the peak occurs at the lowest frequency ($k=8$ in Dataset 1) or the highest frequency ($k=40$ in Dataset 2), supporting an **energy-dependent preference during model training**. - **In this paper, frequency refers to temporal frequency within the forecast output** rather than frequency over the learned function’s input domain. **Table R2-1.** Relative error ($\downarrow$) at each peak bin across the first 10 training epochs. Forecasting horizon is 96. $$\begin{matrix} \hline \text{Dataset} & \text{Frequency bin} & \text{Amplitude} & & & & & \text{Epoch} & & & & & \\\\ & k & A & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 & 9 & 10 \\\\ \hline 1 & 8 & 9 & \mathbf{0.861} & \mathbf{0.368} & \mathbf{0.097} & \mathbf{0.092} & \mathbf{0.090} & \mathbf{0.089} & \mathbf{0.088} & \mathbf{0.088} & \mathbf{0.087} & \mathbf{0.085} \\\\ & 24 & 3 & 0.960 & 0.915 & 0.819 & 0.634 & 0.400 & 0.241 & 0.213 & 0.208 & 0.205 & 0.203 \\\\ & 40 & 1 & 1.001 & 0.999 & 0.956 & 0.922 & 0.889 & 0.850 & 0.802 & 0.757 & 0.714 & 0.674 \\\\ \hline 2 & 8 & 1 & 1.017 & 1.048 & 0.984 & 0.951 & 0.922 & 0.889 & 0.850 & 0.811 & 0.772 & 0.733 \\\\ & 24 & 3 & 0.962 & 0.924 & 0.818 & 0.638 & 0.410 & 0.251 & 0.213 & 0.208 & 0.205 & 0.202 \\\\ & 40 & 9 & \mathbf{0.851} & \mathbf{0.345} & \mathbf{0.098} & \mathbf{0.096} & \mathbf{0.092} & \mathbf{0.091} & \mathbf{0.090} & \mathbf{0.089} & \mathbf{0.089} & \mathbf{0.088} \\\\ \hline \end{matrix}$$

part 3/6

W3: local prominence vs. global energy threshold

Thank you for suggesting an alternative way to select peak components. In fact, we also considered this strategy before but choose local maxima due to two reasons.

  • Mismatch between global energy thresholds and fractal structure in real data. In real time series, many high-frequency local peaks lie well above the fitted fractal line despite their small share of spectral energy, while some low-frequency non-peak components carry more energy yet closely follow the line. Both patterns can be seen in Figure 3(b) of the manuscript. A global energy threshold can therefore miss peaks and misclassify background components, empirically motivates our use of local maxima.

  • The global energy threshold is not stable across datasets. As shown in Table R3-1 , the best σ\sigma differed across all four datasets, ranging from 0.5%0.5\% to 10%10\%, while r=5r=5 was best on three datasets and r=7r=7 on the fourth. The best radius was thus more consistent across datasets, and local peak selection achieved equal or lower MSE on all four.

Table R3-1. Hyperparameter sensitivity test of the two strategies. Search range σ∈{0.1,0.25,0.5,1,2,4,5,10,15,20}%\sigma \in \{0.1, 0.25, 0.5, 1, 2, 4, 5, 10, 15, 20\}\% and r∈{1,3,5,…,19}r \in \{1, 3, 5, \ldots, 19\}. Forecasting horizon is 96.

\hline \text{Selection} & \text{Threshold}\ \sigma & \text{Radius}\ r \\\\ \begin{matrix} \hline \text{Dataset} \\\\ \hline \text{ETTh2} \\\\ \text{ETTm2} \\\\ \text{Weather} \\\\ \text{Exchange} \end{matrix} & \begin{matrix} \hline \text{Best}\ \sigma & \text{MSE}\downarrow \\\\ \hline 2\\% & 0.284 \\\\ 0.5\\% & 0.176 \\\\ 4\\% & 0.163 \\\\ 10\\% & 0.084 \end{matrix} & \begin{matrix} \hline \text{Best}\ r & \text{MSE}\downarrow \\\\ \hline 7 & 0.279 \\\\ 5 & 0.175 \\\\ 5 & 0.163 \\\\ 5 & 0.080 \end{matrix} \\\\ \hline \end{matrix}$$

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.