- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes FIRE, a patch-based frequency-domain forecaster. Each look-back window is RevIN-normalized, patched and transformed by FFT; amplitude and phase are embedded and processed in two disjoint Branchformer-style branches with causal attention over patches. Two per-channel scalar statistics of the look-back window, the Amplitude Non-stationarity Index (ANI, the coefficient of variation of patch spectral energy) and the Phase Irregularity Index (PII, the circular dispersion of inter-patch phase increments), are mapped by small MLPs to multiplicative gates on the MLP and attention outputs of each branch. After cross-variable attention, the branches are recombined into a complex representation, a projection head predicts the future spectrum, and an inverse FFT gives the forecast. Training uses a Huber loss plus the FreDF frequency loss. Experiments cover seven standard long-term benchmarks (look-back 96) against nine baselines, five additional datasets against two baselines, a two-variant ablation, a loss ablation, gate visualizations, a channel-level analysis on eight selected channels, and efficiency numbers.
Critical strengths
-
A clearly stated and physically motivated design question. Asking whether amplitude and phase should be given separate representations, and whether window-level spectral statistics can condition the network, is a reasonable hypothesis, and the architecture is a clean instantiation of it (cos/sin phase coordinates, masking of ill-conditioned phases, disjoint branches). This matters to my assessment because the idea is testable; my concerns are about whether the paper has tested it.
-
Breadth of the experimental surface and candour in places. Twelve datasets, a loss ablation, four sensitivity studies and an efficiency table are provided, and the paper acknowledges that WPMixer is cheaper and that gains vary by dataset. The appendix tables are what allowed me to check the main claims, which I credit.
Critical weaknesses
-
The evidence does not attribute the reported gains to the two claimed contributions (decoupling and ANI/PII modulation).
- Loss confound. FIRE is trained with Huber + FreDF loss; most baselines are reported under their own objectives. Table A5 shows that FIRE with plain MSE obtains average MSE 0.442 / 0.383 / 0.247 on ETTh1 / ETTh2 / Weather, which is worse than WPMixer (0.435 / 0.371 / 0.244), TimeMixer (ETTh2 0.364, Weather 0.240) and DistDF in Table 1. With Huber only it is 0.441 / 0.373 / 0.245. So on the three datasets where this was checked, the architecture alone does not beat the baselines; the margin comes from the borrowed training objective. No baseline is retrained with the same loss.
- Ablation. Removing modulation changes average MSE by 0.001–0.005 on six of seven datasets (Table 4), with no seeds or standard deviations. On Electricity, the dataset the paper highlights as its largest win (4.7% over APDNet), the full model, "w/o Mod." and "Joint Branch" score 0.162 / 0.163 / 0.163: neither claimed contribution matters there, so the gain must come from something else (cross-variable attention, loss, or tuning). The only large Joint-Branch gap (ETTh1) is driven by two horizons (0.547, 0.551) that look like an unstable run rather than a systematic effect. There is no ANI-only / PII-only ablation and no control that feeds the gates a constant or a random scalar.
- The gates barely act. Figure 3 shows learned gate values within roughly 1.00–1.05, applied to sub-layer outputs that are then normalized. A ≤5% rescaling is hard to reconcile with "regime-adaptive" modeling.
- Tuning variance exceeds the margins. The sensitivity tables, which share the main protocol (T = 96), give ETTh1 averages of 0.428–0.441, Weather 0.244–0.248 and Traffic 0.440–0.467, never reaching the main-table values (0.424, 0.241, 0.433). The headline numbers therefore come from a per-dataset/per-horizon search over patch length, batch size and learning rate whose spread (0.005–0.03) is larger than most gaps to the second-best baseline (often 0.001–0.006). Only Fredformer and WPMixer were re-run; it is not stated that baselines received a comparable search.
-
The central analytical claim, that FIRE's advantage "scales progressively with the severity of non-stationarity", rests on eight hand-selected channels, and the indices themselves are weakly justified.
- Table 5 reports 4 of 21 Weather channels and 4 of 7 ETTm1 channels, described as "selected". Weather channels 13 and 12 have essentially identical indices (ANI 0.3446 vs 0.3456, PII 1.2977 vs 1.2996) yet gains of 8.9% and 16.8%, so the indices do not explain the gain even within the selected set. No correlation over all channels and datasets is given, and the comparison is against a single baseline.
- PII is nearly saturated: seven of the eight reported values lie in 1.14–1.31 rad, so it hardly discriminates between channels. It weights all above-threshold bins equally, so it is likely dominated by low-energy noisy bins.
- ANI is computed after RevIN and, by Parseval, the patch spectral norm equals the time-domain patch energy; ANI is therefore the coefficient of variation of patch RMS, a time-domain statistic, not a spectral diagnostic.
- The motivating Figure 1(a,b) is close to tautological (gain perturbations change amplitude, circular shifts change phase).
- The closest prior work on amplitude/phase modeling that the paper itself cites (APRNet, TimeAPN) is not compared against, so novelty relative to it is not established empirically.
Questions for authors
Clarifications that could change my assessment:
- Please retrain at least WPMixer, Fredformer and iTransformer with the same Huber + FreDF objective and the same hyperparameter search budget, or report FIRE with each baseline's native loss on all seven datasets. Does FIRE still lead?
- Report mean ± std over ≥3 seeds for Table 1 and Table 4. Which differences survive?
- On Electricity, all three ablation variants tie. What produces the gain over the baselines there?
- Provide the ablations described in A.5.2 (amplitude-only, phase-only, ANI-only, PII-only) and a control where the gate input is replaced by a constant or shuffled across samples.
- Replace Table 5 with the gain-versus-ANI/PII relationship over all channels of all seven datasets (scatter plus rank correlation), against more than one baseline.
- What are the loss weights, and the selected patch length / batch size / learning rate for each dataset and horizon? Was the search validated on the validation set for every reported cell?
- How does FIRE compare with APRNet and TimeAPN, and with baselines at longer look-backs (Table A9 shows FIRE benefits from T up to 512)?
Suggestion for future work (does not affect my rating): per-frequency or per-patch conditioning rather than one scalar per channel.
3 comments
Q7. APRNet, TimeAPN, and longer contexts.
Table 6. Comparison at look-back 96 (average MSE/MAE).
| Dataset | FIRE | APRNet | F+T | P+T | S+T |
|---|---|---|---|---|---|
| ETTh1 | .424/.427 | .443/.432 | .441/.448 | .438/.430 | .447/.437 |
| ETTh2 | .362/.392 | .375/.397 | .384/.411 | .367/.395 | .374/.402 |
| ETTm1 | .371/.387 | .385/.395 | .373/.406 | .377/.395 | .391/.404 |
| ETTm2 | .276/.319 | .281/.324 | .277/.328 | .279/.324 | .278/.326 |
| Weather | .241/.265 | .256/.280 | .261/.317 | .245/.287 | .237/.287 |
| Electricity | .162/.255 | .171/.263 | .170/.274 | .161/.259 | .164/.270 |
| Best count | 4/6 | 0/0 | 0/0 | 1/0 | 1/0 |
- F/P/S denote FEDformer/PatchTST/S_Mamba. T denotes TimeAPN. All entries use look-back 96. Results average four horizons, best counts cover 24 dataset–horizon settings. FIRE achieves the best counts in average MSE(4)/MAE(6). For all cases, Fire also wins 14/24 for MSE and 22/24 for MAE, we'll add full results in the revised PDF.
Table 7. Traffic comparison at look-back 96 (average MSE/MAE).
| Dataset | FIRE | APRNet | F+T |
|---|---|---|---|
| Traffic | .433/.266 | .464/.293 | .479/.304 |
| Best count | 4/4 | 0/0 | 0/0 |
-
On Traffic, FIRE achieves the lowest MSE and MAE at all four horizons among the eligible models. Against APRNet, FIRE achieves lower average MSE and MAE on all seven datasets.
-
Appendix Table A.9 evaluates contexts up to 512: Weather’s average MSE decreases from .244 (96) to .227 (512). This supports using longer histories, without establishing superiority under matched longer-context comparisons.
The supplementary PDF will document full results and settings, with claims revised to match the evidence.
Thank the authors for the extensive rebuttal. While these additions substantially improve the evaluation, the new results still reinforce my core concern regarding component attribution: outside of Traffic, ANI/PII modulation yields virtually identical MSE to w/o Mod. across six of the seven datasets. Moreover, without the requested constant- or shuffled-scalar gating control, it remains unclear whether the Traffic gain stems from genuine spectral conditioning or simply added MLP capacity. Therefore, I maintain my current score.
We appreciate the detailed examination of attribution and evaluation settings. We clarify these issues with additional comparisons and ablations.
Q1. Training objective.
Table 1. Shared-objective results (average MSE/MAE).
Q2–Q3. Variability and component contributions.
Table 2. Average MSE over 10 seeds.
Table 3. Additional modulation ablations (MSE, 10 seeds).
Q4. Individual modulation and gates.
Table 4. Individual modulation (average MSE, 10 seeds).
Q5. Analysis across channels.
Table 5(a). ETTm1 gains (%).
Table 5(b). ETTm2 gains (%).
Table 5(c). Weather gains (%).
Table 5(d). Traffic gains (%).
Higher-index regions generally benefit more, most clearly on ETTm1/Traffic. We will revise “substantial complementary gains” to reflect these general trends.
ANI relates to patch energy variation, which whole-window RevIN need not eliminate. It does not uniquely identify spectral-shape changes. Similar channel-average PII values alone do not establish instance-level saturation. Figure 1 illustrates the Fourier motivation, not independent evidence of forecasting gains.
Q6. Tuning protocol.
Both FIRE and reproduced baselines were tuned using model-specific search spaces. We followed baselines’ official ranges and reference configurations, selecting on validation data and reserving test data for evaluation (Appendix A.4).
Appendix A.5.4 uses one-factor sweeps with other settings fixed, distinct from configuration selection and seed variability. We will clarify reference settings and report loss weights and selected patch length/batch size/learning rate for each dataset/horizon.