- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies structured forecasting losses to test whether they preserve error correction and exact-fit optimality. For DBLoss, the authors show that exact seasonal fitting can eliminate trend-correction gradients and derive a direction-equivalent scalar potential. For RI-Loss, they find a sharp auxiliary-weight threshold. Below this, it has the unique global minimizer of the expected objective and above it, it becomes a saddle point. The theoretical findings are validated through numerical gradient checks, controlled optimization experiments, and neural-parameter perturbations. There are also analyses of Time-o1 and PS Loss testing representation invariance and batch-statistic sensitivity.
Critical strengths
The paper has some theoretical characterization of failure mechanisms in structured forecasting objectives. These results provide concrete mathematical conditions that can guide future loss design and hyperparameter selection. It has validated its theoretical results, including source-level derivative checks and controlled SGD/Adam comparisons.
Critical weaknesses
One of the limitations is that several central failure modes are shown under certain conditions rather than ordinary training trajectories. The DBLoss experiments use controlled residual interpolation and frozen-model bias adapters, while the RI-Loss curvature experiments construct exact agreement using model-generated targets. Moreover, Adam can reduce much of weak nonzero correction signals. Thus, the problems in its theoretical framework may not always translate into real optimization problems. The results also have uneven experimental depth, and the RI-Loss analysis focuses on an explicitly activated helper rather than the default training path.
Questions for authors
-
How frequently does the DBLoss stationary-bias or near-zero correction regime occur during ordinary end-to-end forecasting training, particularly when using Adam?
-
Can the authors demonstrate whether selecting RI-Loss auxiliary weights below the derived threshold consistently improves optimization stability or forecasting accuracy on real datasets?
-
How representative is the explicitly activated RI-Loss helper of the configurations used to obtain the original method's reported results, and do the theoretical conclusions extend to those configurations?
-
For Time-o1 and PS Loss, how frequently do the identified representation and batch-statistic sensitivities arise in standard implementations without deliberately constructed interventions?
Response to Reviewer rxPu
Thank you for helping us examine the connection to ordinary training. We appreciate your emphasis on checking how the analysis applies to the training process. Below we summarize the additional evidence and its limits. References are to the revised manuscript.
Q1. DBLoss frequency and Adam
No exactly zero coefficient was recorded in 386,400 native updates across five runs per dataset. The marker cT=0.5S/(T+ϵ)<0.1 and T/τ≥0.25 occurred in 45,600/45,600 Exchange updates (100%) and 21,398/340,800 Solar updates (6.28%); ϵ=10−8 and τ is persistence trend MAE on training windows (Table 12).
We agree that Adam's behavior also needs to be checked directly. Separate five-seed probes fixed parameters, batch and Adam moments, floored cT at 0.1, and matched native displacement lengths. Median vector differences divided by native step length were 51.70% and 17.57%; steps reducing trend MAE increased from 99/120 to 113/120 and 102/120 to 114/120, respectively (Table 13). Adam usually continued correcting error. The marker is not a failure criterion, and these separate cohorts do not yield the fraction of all updates materially affected.
Q2. Below-threshold forecasting results
Half-critical RI (λcrit/2) improved test MSE over RI-10 in all five paired seeds per principal setting. To address your consistency question, we report both the paired-seed result and the plain-MSE baseline. Each objective used learning rates selected on separate development seeds and 60 complete epochs without early stopping; test MSE used validation-selected checkpoints (Table 2). Model configurations and loss weights were fixed.
Values are means; changes use unrounded means. Plain MSE was lowest among the four RI comparison arms before rounding. Earlier 30-epoch ETTh1 results slightly favored RI-10 (+0.06% for half-critical RI), and the weight sweep showed no clear threshold transition (Table 19). This supports the tested comparisons, not a universal weight-selection rule or numerical-stability advantage; it does not isolate the threshold condition's causal effect.
Q3. Implementation and theoretical scope
We verified the README's activation path for the weight-10 DLinear helper; five inspected default entries use MSE. We found no run records in the inspected release linking this path to the original reported scores, and did not reconstruct the original end-to-end benchmark runs. On one ETTh2 batch with shared surrogate draws, the activated code and our implementation had identical prediction gradients and a maximum parameter-gradient difference of 9.31×10−10 (Appendix B.1).
Appendix A.4 extends the threshold to specified kernel scales and batch normalizations under the same biased-estimator and independence assumptions. Under Appendix A.5's assumptions, the standard signed, unclipped unbiased version instead preserves unique exact-fit optimality at every nonnegative weight. These conclusions do not cover unverified historical configurations.
Q4. Time-o1 and PS in ordinary pipelines
The natural-occurrence question remains partly open. We appreciate your distinction between a measured sensitivity and how often it occurs in ordinary training. Without manual sign flips, 0/384 observed Time-o1 residual batches had projected energy below 1% of scale-normalized residual energy (Table 22); this is not a sign-flip frequency estimate.
For PS, regrouping identical examples with fixed predictions, effective batch size and Adam history changed updates by more than 1% in 192/192 comparisons per dataset: median 4.01% on ETTh1 and 8.19% on ETTh2. Sharing pooled patch geometry and detached weights removed differences to numerical precision (Table 23). This measures intervention sensitivity, not natural incidence or test-error deterioration; both analyses remain in the appendix.