- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies frequency-domain objectives for multivariate time-series forecasting, using FreDF as the main example. The authors decompose such objectives into three factors: the basis in which the residual is represented, the norm applied to it, and the weighting of residual coordinates.
The paper argues that a unitary basis change is inert under a squared norm, and then uses a time-domain L1 loss as a control to study how much of FreDF's gain is due to the norm rather than the Fourier basis. It further introduces WHITEDF, a residual-whitening objective based on estimated temporal and cross-variable second moments, and connects the analysis to classical GLS results.
The empirical study mainly uses iTransformer, with additional experiments on PatchTST and DLinear, and reports a large number of diagnostic and follow-up experiments on ETT, Weather, ECL, and Traffic.
Critical strengths
- The paper asks a useful diagnostic question.
Separating the effects of basis, norm, and residual weighting is a reasonable way to analyze frequency-domain forecasting objectives. In particular, the time-domain L1 control is simple and useful, and on the main iTransformer/FreDF experiments it reproduces a substantial part of FreDF's improvement.
- The authors are transparent about many negative and unstable results.
The paper explicitly reports failed experiments, multiple-testing corrections, test-period sensitivity, implementation issues, and cases where the proposed whitening objective hurts performance. This transparency is valuable, even though in my view these results ultimately weaken the broader claims of the paper.
Critical weaknesses
- The baseline and dataset coverage is not sufficient for the broad claim made by the paper.
The paper is framed as an analysis of frequency-domain forecasting losses in general, but the central norm-versus-basis conclusion is primarily established for FreDF on iTransformer. The most systematic experiments are concentrated on the ETT subsets and Weather, while several closely related methods discussed by the paper itself, such as Time-o1, OLMA, SFALoss, and QDF, are either not evaluated or only appear in limited auxiliary experiments.
This is particularly problematic because the ETT subsets do not provide strong independent domain diversity. The paper itself notes that the held-out WHITEDF gains on iTransformer are concentrated on datasets sharing the development cell's station and test months. ECL and Traffic broaden the whitening analysis, but they are not used to provide an equally systematic test of the main norm-versus-basis claim.
As a result, I think the evidence supports a much narrower statement: on the authors' main iTransformer/FreDF setup, a time-domain L1 term explains much of the gain. I do not think it supports the broader framing that frequency-domain forecasting losses are "mostly the norm."
- The evaluation is heavily exploratory/post-hoc, and the defensive presentation does not resolve this issue.
The WHITEDF configuration is selected using the test error of the ETTh1, T=96 development cell. Several later experiments are explicitly motivated after observing previous results: additional candidate criteria are introduced after earlier comparisons, some follow-up cells are selected after observing harm, and the Holm correction families are defined after seeing the results. The authors disclose these choices, but disclosure does not turn them into independent confirmatory evidence.
This issue is amplified by the structure of the manuscript. Many sections read as responses to anticipated reviewer objections: checkpoint-rule controls, scale matching, rotations, shifted test periods, alternative criteria, and extensive caveats about what each result does or does not establish. These controls can be useful diagnostically, but they do not substitute for a pre-specified evaluation on genuinely unseen methods, datasets, and time periods.
The instability of the whitening results illustrates the problem. The largest PatchTST gain does not survive the Holm correction and disappears when the test period is shifted forward. The paper ultimately acknowledges that it cannot predict when whitening will help. In my view, this leaves the weighting part of the paper as an exploratory investigation rather than a validated general result.
Questions for authors
- What exactly is the intended scope of the "mostly the norm" conclusion?
If the claim is specifically about FreDF on iTransformer, I think the title and framing should be narrowed substantially. If the intended claim is about frequency-domain forecasting losses more generally, why are closely related objectives such as Time-o1, OLMA, SFALoss, and QDF not evaluated systematically under the same norm-versus-basis controls across multiple backbones and datasets?
- Can the authors provide a genuinely held-out evaluation in which all loss definitions, hyperparameters, and hypotheses are fixed before looking at the evaluation results?
In particular, the current WHITEDF setting is chosen using the test error of the development cell, and several later controls are explicitly introduced after observing earlier outcomes. A validation-only selection protocol followed by evaluation on unseen datasets or later non-overlapping time periods would materially affect my assessment.
- Can the authors clarify which experiments and hypotheses were specified before observing test results and which were added post hoc?
The current manuscript mixes confirmatory and exploratory analyses, and I found it difficult to determine which conclusions should be interpreted as independently validated.