- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper introduces excess spectral suppression (ES), a mode-wise prediction-skill gap relative to a Bayes or attainable reference. Through synthetic experiments and eight forecasting benchmarks, it studies how capacity and training budgets affect predictable tail modes. It also investigates operator effective rank and proposes predictability-aware loss reweighting to reduce tail errors.
Critical strengths
Evaluating mode-wise errors against an attainable reference provides a useful alternative to diagnosing forecasting bias from output smoothness alone.
The controlled synthetic experiments and interventions on width, regularization, and head rank provide informative evidence about the relationship between capacity constraints and tail prediction errors.
Critical weaknesses
ES does not uniquely identify suppression. Underprediction and over-amplification can produce identical ES and its decomposition, weakening the interpretation of “complete sacrifice” on real data.
The noise-floor calibration is not statistically convincing. Appendix B acknowledges that the DLinear-based floor mainly reflects architecture performance gaps, so falling below it cannot establish the absence of suppression in modern forecasters.
Questions for authors
For a centered mode, predicting or yields identical ES and “unrealized skill.” What additional evidence distinguishes actual suppression from miscalibration?
Do the conclusions about modern forecasters hold when uncertainty is estimated using repeated strong reference models and independent reference selection, rather than the weakest member’s performance gap?
3 comments
Thank you for the response. Most of my concerns have been addressed, so I am considering raising my score by one point. However, the manuscript’s overall quality, including its explanation of the underlying mechanisms, may still fall short of the standard required for acceptance.
Thank you for your reply and for considering raising your score.
Since our initial response focused on your two soundness questions, we briefly summarize how the revised manuscript establishes the underlying mechanism (§§3.3–5):
- Why MSE sacrifices predictable low-energy modes first (§3.3): Under MSE, both a hard rank ceiling (Proposition 1) and gradient-descent learning speed (Proposition 2, learning time ) prioritize modes by , where is mode energy and is predictability. Because spans orders of magnitude while , low-energy modes receive tiny MSE gradients and are dropped first — even when highly predictable.
- Separating hard capacity limits from optimization/objective bias (§3.4, §5): Comparing trained networks against closed-form Reduced-Rank Regression (RRR, the exact MSE-optimal rank- linear predictor) separates the two on real data. At rank , tail loss is mandated by the rank ceiling (RRR-8 also drops the tail). At , however, RRR-32 recovers over half the tail skill while trained networks leave 2.0–3.5× more tail skill unrealized — an optimization/objective effect that fixed-capacity inverse-energy loss reweighting (§5) recovers (closing 31–64% of the gap while also improving overall MSE).
- Pinpointing the bottleneck inside the model (§4): Prior work attributes spectral loss to collapse of internal feature rank, but §4 shows feature rank is only a proxy: dropout crushes internal feature rank without hurting tail skill, whereas inference-time SVD truncation of the output head sacrifices tail skill without touching internal features. Across all four intervention families, what actually governs tail sacrifice is the effective rank of the end-to-end input-to-prediction operator (Spearman with unrealized tail skill, vs. for feature rank).
We appreciate your engagement and are glad to clarify any point further.
Response to Reviewer 1m2X: Direction Diagnostics, Cross-Fitted Reference & Seed-Noise Calibration
We thank the reviewer for two precise soundness objections. Both pointed at real gaps, and the resulting experiments changed the paper. Sections and tables refer to the revised PDF.
W1 / Q1. Can ES tell suppression from miscalibration (predicting 0 vs. 2f⋆(x))?
The reviewer is right that R2-based ES cannot: on a centered mode, both predictions give Rk2=0. The two cases differ in information content, so we now report two scale-free diagnostics per tail mode (re-run measurement study, 26 skill-bearing channels, modes 25–48):
The attainable reference is R⋆2=0.185. At r=8 the tail predictions are uncorrelated with the target, and no recalibration recovers any skill. The MLP's tail energy (amplitude 0.55) is uncorrelated with the target, which is exactly our "injected error" term. This mirrors the amplitude/decorrelation split of Subich et al. (ICML 2025), now cited. (App. C.3, Table 9.)
One related change: the closed-form RRR-8 baseline shows that the r=8 endpoint is budget-mandated, so "complete sacrifice" is no longer a headline. The new headline is that trained r=32 networks leave 2.0–3.5× more tail skill unrealized than the rank-32 optimum (§3.4, Table 2). Their tail predictions are also nearly uncorrelated with the target (MLP-32: 0.017, vs. 0.135 for RRR-32), so the same diagnosis holds.
W2 / Q2. Noise floor; do the modern-forecaster conclusions survive strong repeated references and independent reference selection?
We agree the DLinear-twin floor mostly measured an architecture gap, and adopted the two controls the reviewer proposes:
Modern forecasters under this calibration (8 benchmarks; new: shared-weight PatchTST, dmodel∈{16,…,128}). Values are tail ES / unrealized component, all measured channels; seed noise is 0.007 (Traffic) and 0.006 (Electricity):
So yes: the conclusion does not rest on a loose floor. On Traffic these models beat the reference; on Electricity the unrealized part is ≤ 0.001 for the shared-weight and cross-channel models, and the remaining ES is injected error. The per-channel PatchTST is above seed noise on Traffic (0.027) but at about a tenth of the attainable 0.248: small, not absent. For the zero-shot foundation models (already in [−0.028,+0.046] under the stricter un-cross-fitted reference), we likewise drop the floor comparison and state "within ±0.05" (§6).
The tighter calibration also cuts the other way. On Traffic and Electricity, full-rank (r=96) networks leave 0.129–0.209 of tail skill unrealized, 18–35× seed noise. Previously "at floor level", this is now claimable.
Changes: §3.1 ("lower bound" withdrawn; both bias directions stated); Table 1 (cross-fitted, decomposed, seed-noise column); App. C.1, C.4.
Presentation. We restructured the paper: