acceptodds
Under review as a conference paper at ICLR 2027

Not All Frequencies Are Harmonious: Conflict-Aware Multi-Task Spectral Training for Time-Series Forecasting

Abstract

Time-series forecasts can improve in some frequency bands while becoming less accurate in others. Aggregate error can conceal these trade-offs. Band loss values alone do not reveal how reducing one band's error affects the others during training. We therefore formulate time-series forecasting as a spectral multi-objective problem and propose Dyadic Spectral Alignment and Coordination (DySAC), a model-agnostic training framework. DySAC partitions the forecasting residual into non-overlapping dyadic bands. Each band defines a separate prediction objective for the same time series. It coordinates their gradients to seek compatible updates to the shared forecasting model. The resulting band coefficients reflect gradient directions, not just loss values or gradient norms. The aim is to distribute forecasting gains across frequencies rather than concentrate them in a few bands. The forecasting backbone and inference procedure remain unchanged. We evaluate prediction errors on the same bands to assess this balance. Across seven datasets and four backbones, DySAC yields a mean task-wise improvement of +3.85% over MSE training, with positive mean improvement in every dataset–backbone setting. The results support more balanced spectral performance, although individual tasks can still regress. Measured training-step times are 2.52-3.30 × MSE in a four-setting timing benchmark, with no change to inference.

Primary area:
learning on time series and dynamical systems
Keywords:
Time-series forecasting, gradient conflict, multi-objective optimization, frequency-domain learning, spectral bias, multi-task learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_NVCrOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper formulates time-series forecasting as a spectral multi-objective problem, and proposes Dyadic Spectral Alignment and Coordination (DySAC) to define the loss function and optimize it. From a high level of overview, the methodology is combing two established techniques (viz. frequency alignment and MOO) with application to time-series forecasting. As a model-agnostic approach, it does not change forecasting backbone and inference procedure. The effectiveness of the proposed DySAC approach is validated on standard time-series benchmarks.

Critical strengths

The ablation study is thorough, containing uncertainty bounds, which improves the technical quality of this study.

The paper is clearly written, and provides a main figure (Figure 1) to facilitate the readers to understand and grasp the core ideas swiftly.

The experiments cover several additional metrics beyond standard MAE/MSE, such as SMAPE, WAPE and OWA-style score.

Critical weaknesses

While the implementation code is provided, the code is not very complete. It does not contain details on dataset selection, data file, and parameter specification/tuning, which difficult to execute. The resulting logs are also not available.

The frequency loss in (1) and (3) has been defined in diverse literature (with or without MRA). Authors should discuss it in a separate preliminary section and refer to the literature that uses it (with or without MRA, e.g., OLMA), and then build up their contribution on this basis.

The multi-objective optimization problem is ubiquitous and has been discussed with rich literature. Similarly, authors should discuss it in a separate preliminary section, with clear and comprehensive reference, and then build up their contribution on this basis.

Once again, since the frequency loss and the multi-objective optimization has numerous established studies, including in the time-series domain, this paper could benefit from articulating (1) the evolution and contribution of existing studies, (2) the limitation of existing studies (not only possible methodology difference, since different methods could resolve the same problem), (3) the evidence that the proposed method could address it. While authors make some discussion on frequency modeling, the discussion is not sufficient and strong.

Theoretical analysis is not available, but essential to demonstrate the utility of frequency aliment and moo strategy.

Questions for authors

How does the proposed moo strategy improve over existing methods in this task? What is the critical problem in this task that cannot be addressed by the established moo approach, while suiting the proposed moo approach? Could you provide additional theoretical/evidence to support these limitations and the effectiveness? Similarly, how does the proposed method outperform existing frequency-domain loss? Could you please provide theoretical/evidence to support the existence of these limitations and the effectiveness of the proposed method to address them?

6 comments
AuthorsOpenReview ↗

Response to Reviewer NVCr (Part 1)

Many thanks to Reviewer NVCr for providing a detailed review and insightful questions.

We respectfully disagree that prior spectral losses and multi-objective optimization (MOO) solvers subsume our contribution. Our contribution is a spectral multi-objective formulation of forecasting, a gradient-coordinated training framework, and an evaluation protocol on the same frequency objectives (Sections 2.1–2.3). Prior spectral losses specify what to optimize, while MOO solvers combine gradients of given objectives. Our framework connects these two levels: it exposes frequency-specific forecasting errors as separate objectives, coordinates their shared update using ConFIG, and measures gains and regressions on those same bands. We adopt ConFIG’s coordination rule and do not claim a new aggregation algorithm.

The other reviewers’ comments identify this formulation and application as the contribution. Reviewer jV4U recognized the meaningful problem formulation, noting that frequency-specific objectives provide a reasonable way to consider optimization trade-offs. Reviewer PfQa identified the application of conflict-aware multi-task optimization to spectral forecasting losses as the main contribution. The responses below substantiate this distinction through matched comparisons, conditional analysis, and component ablations.

Q1: How does the proposed moo strategy improve over existing methods in this task?

A1: DySAC achieves the lowest mean MSE and MAE in three of four matched settings. Thank you for suggesting this comparison; we added the complete results to the main paper (Table 5 in Section 3.4). We compare PCGrad (Yu et al., 2020), MGDA (Sener and Koltun, 2018), CAGrad (Liu et al., 2021), Nash-MTL (Navon et al., 2022), GradNorm (Chen et al., 2018), and Dyadic + Mean, plus an MSE-trained reference. The spectral methods share identical dyadic objectives and matched training conditions.

The complete eight-method results below report three-seed mean ± sample SD; bold marks the lowest mean for each metric. Table 5 in Section 3.4 reports the same comparison.

Electricity / FreTS

MethodMSEMAE
MSE training0.19030 ± 0.000690.27951 ± 0.00052
Dyadic + Mean0.18785 ± 0.000510.27429 ± 0.00056
PCGrad0.18775 ± 0.000590.27386 ± 0.00042
MGDA0.19181 ± 0.000260.27761 ± 0.00064
CAGrad0.18804 ± 0.000270.27354 ± 0.00100
Nash-MTL0.18873 ± 0.001810.27433 ± 0.00174
GradNorm0.23762 ± 0.002050.33802 ± 0.00143
Ours0.18771 ± 0.000920.27333 ± 0.00135

Weather / FreTS

MethodMSEMAE
MSE training0.21400 ± 0.003600.27575 ± 0.00535
Dyadic + Mean0.20998 ± 0.003790.26080 ± 0.00784
PCGrad0.21036 ± 0.003850.26302 ± 0.00733
MGDA0.21226 ± 0.001870.26598 ± 0.00441
CAGrad0.20869 ± 0.003060.25824 ± 0.00591
Nash-MTL0.20829 ± 0.001510.25699 ± 0.00440
GradNorm0.21161 ± 0.001360.26252 ± 0.00521
Ours0.20783 ± 0.001800.25686 ± 0.00315

Electricity / TimeMixer

MethodMSEMAE
MSE training0.17691 ± 0.001800.27750 ± 0.00245
Dyadic + Mean0.19269 ± 0.000910.29690 ± 0.00199
PCGrad0.18888 ± 0.002850.29305 ± 0.00423
MGDA0.19093 ± 0.005160.28413 ± 0.00541
CAGrad0.18439 ± 0.006670.28544 ± 0.00854
Nash-MTL0.17402 ± 0.001880.27293 ± 0.00255
GradNorm0.19413 ± 0.001500.29822 ± 0.00134
Ours0.17333 ± 0.000910.27136 ± 0.00160

Weather / TimeMixer

MethodMSEMAE
MSE training0.20887 ± 0.000640.25249 ± 0.00086
Dyadic + Mean0.20695 ± 0.004830.24833 ± 0.00588
PCGrad0.20586 ± 0.003280.24735 ± 0.00380
MGDA0.22787 ± 0.002050.26402 ± 0.00170
CAGrad0.20729 ± 0.005470.24825 ± 0.00520
Nash-MTL0.20868 ± 0.002490.24759 ± 0.00197
GradNorm0.20634 ± 0.001690.24816 ± 0.00186
Ours0.20974 ± 0.001970.24894 ± 0.00141

The strongest improvement over Mean is on Electricity / TimeMixer: 10.05% lower MSE and 8.60% lower MAE. PCGrad leads on Weather / TimeMixer; the comparison establishes the observed benefit of coordination and its dependence on the forecasting setting.

AuthorsOpenReview ↗

Response to Reviewer NVCr (Part 2)

Q2: What is the critical problem in this task that cannot be addressed by the established moo approach, while suiting the proposed moo approach?

A2: The critical problem is interference among frequency-specific objectives of the same forecast; established MOO solvers can address it once these objectives are defined. Our contribution is to expose this forecasting problem through band-wise objectives, coordinate their shared-parameter gradients, and evaluate which bands improve or regress (Sections 2.1–2.3; Appendix C). This brings gradient-conflict coordination studied in multi-task learning, including computer vision, into a spectral forecasting framework.

Existing MOO work mainly studies how to combine objectives that have already been defined. MGDA (Sener and Koltun, 2018) combines task gradients toward Pareto stationarity; PCGrad (Yu et al., 2020) projects conflicting gradients; CAGrad (Liu et al., 2021) balances average progress and conflict aversion. Nash-MTL (Navon et al., 2022) treats task combination as bargaining, while GradNorm (Chen et al., 2018) adapts task weights using gradient norms. ConFIG (Liu et al., 2025) coordinates gradients for physics-informed neural networks; we adopt its direction construction unchanged (Appendix A.3).

Our distinction is the forecasting task definition and its evaluation. We construct separate objectives from the frequency bands of a single forecast, coordinate their shared update, and measure gains and regressions on those same bands (Sections 2.1–2.3; Appendix C). Existing solvers can operate on these objectives: Table 5 in Section 3.4 compares them under identical band losses. The claim is a spectral forecasting framework and aligned evaluation, not a new MOO solver.

Q3: Could you provide additional theoretical/evidence to support these limitations and the effectiveness?

A3: Appendix B provides conditional coordination analysis, and the added matched comparisons test forecasting effectiveness. The evidence below addresses gradient conflict, the information omitted by scalar loss statistics, and the conditions for descent; our response to Reviewer jV4U, Question 2, reports additional training-stage diagnostics.

Question addressedEvidencePaper location
Do band gradients conflict?Mean pairwise conflict of 10.89%–44.00% across four Fred-md backbones.Table 22 in Appendix K
What do scalar loss statistics omit?Two configurations have identical losses, gradient norms, and loss histories but different gradient geometry.Appendix B.3
What does the coordination analysis establish?Conditions for common descent; sufficiently small direct-step descent; an O(T^(-1/2)) projected-residual best-iterate bound under the stated deterministic smoothness assumptions.Appendices B.1–B.2

Forecasting effectiveness is tested by the controlled results in Q1. The analysis concerns direction geometry and conditional direct updates. Raw band-gradient conflict is a different diagnostic from the merged update and forecasting accuracy; our response to jV4U Q2 discusses the 96-run training-stage measurements.

AuthorsOpenReview ↗

Response to Reviewer NVCr (Part 3)

Q4: Similarly, how does the proposed method outperform existing frequency-domain loss?

A4: DySAC achieves the highest mean task-wise spectral improvement and the best average task rank among the evaluated objectives. We compare MSE and six established objectives across seven datasets and four backbones, including FreDF (Wang et al., 2025), DBLoss (Qiu et al., 2025), and PS-Loss (Kudrat et al., 2025). The compact comparison below retains the strongest competing objective separately for each measure.

Spectral evaluation measureOursBest competing objective
Mean task-wise improvement over MSE ↑+3.85%DBLoss: +2.85%
Worst-band mean improvement ↑+2.39%FreDF: +1.60%
Average task rank ↓2.01FreDF: 3.42

DySAC also has the best average task rank in 19 of 28 settings, including ties. Figure 3 in Section 3.3 and Table 21 in Appendix I report the complete spectral comparison; Table 1 in Section 3.2 and Appendix E report aggregate MSE/MAE. These results assess both forecasting accuracy and how gains are distributed across bands.

Q5: Could you please provide theoretical/evidence to support the existence of these limitations and the effectiveness of the proposed method to address them?

A5: Figure 3 in Section 3.3 shows that improvements in some frequency bands can coexist with deterioration in others. For example, TILDE-Q improves the first two bands by +5.59% and +3.37%, while its fourth- and fifth-band means are -1.81% and -3.60%. This motivates evaluating gains and regressions on individual bands.

Table 4 in Section 3.4 separates the effects of spectral supervision, dyadic decomposition, and gradient coordination. The three-seed FreTS means below show the component comparison; Table 4 in Section 3.4 provides standard deviations and the additional L2 variants.

Training variantElectricity MSE / MAEWeather MSE / MAE
Single full-spectrum L10.1910 / 0.27690.2101 / 0.2637
Dyadic + Mean0.1879 / 0.27430.2100 / 0.2608
Ours0.1877 / 0.27330.2078 / 0.2569

Both decomposition and coordination improve all four means in this ablation. Appendix B supplies the conditional coordination analysis summarized in Q3, while Table 6 in Section 3.4 separately tests the frequency partition with loss and coordinator fixed. Together, these comparisons support the proposed framework and its design choices.

Code and reproducibility. Appendices D.2–D.3 document datasets, splits, preprocessing, configurations, and training settings. We commit to releasing the full code alongside the final version of this paper, together with complete reproduction details including dependencies, dataset selection and preprocessing, per-setting configurations, seeds, training/evaluation commands, run logs, and checkpoint identifiers.

Reviewer_PfQaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DySAC, a conflict-aware multi-task training framework for time-series forecasting. It divides the Fourier spectrum of prediction residuals into dyadic frequency bands and treats the amplitude L1 loss of each band as a separate task. ConFIG is then used to combine gradients across bands. Experiments on seven datasets and four backbone models show consistent improvements over MSE under the proposed band-wise metric, although the training cost increases by 2.5–3.3 times. The main contribution is the application of conflict-aware multi-task optimization to spectral forecasting losses, rather than a new gradient aggregation algorithm.

Critical strengths

1: the empirical results are reasonably consistent across different backbones. DySAC outperforms MSE in 52 of the 56 reported MSE/MAE columns, suggesting that treating frequency bands as separate tasks may be useful beyond a single architecture.

2: the experiments are reported with sufficient detail to allow cross-checking of the main tables and summary statistics. The reported numbers are largely consistent, which increases my confidence in the empirical results. The approach also offers a straightforward way to incorporate gradient coordination into existing forecasting models.

Critical weaknesses

the central contribution is not sufficiently established. DySAC uses the existing ConFIG solver with a standard dyadic frequency partition. More importantly, the ablation results show only 0.11–1.5% improvements over simple gradient averaging. The paper does not compare ConFIG with alternatives such as PCGrad, MGDA, or Nash-MTL under the same band-wise objectives. As a result, it remains unclear whether conflict-aware coordination is necessary or offers a meaningful advantage over simpler methods.

Questions for authors

1: How does DySAC compare with PCGrad, MGDA, Nash-MTL, and GradNorm when all methods use the same band-wise losses? This comparison is important for determining whether ConFIG

2: Can the authors provide results for prediction horizons of 96, 192, 336, and 720, comparing DySAC with MSE and strong baselines such as DBLoss and PS-Loss? In particular, does the gap between Dyadic+ConFIG and Dyadic+Mean increase at longer horizons?

3: How often is the ConFIG fallback mechanism triggered during training? A high fallback rate would raise the question of whether the reported gains mainly come from simple gradient averaging.

6 comments
AuthorsOpenReview ↗

Response to Reviewer PfQa (Part 1)

Many thanks to Reviewer PfQa for the valuable comments and insightful suggestions. We address the requests on matched baselines, prediction horizons, and the ConFIG fallback below

Q1: How does DySAC compare with PCGrad, MGDA, Nash-MTL, and GradNorm when all methods use the same band-wise losses?

A1: DySAC has the lowest mean MSE and MAE in three of four matched settings; PCGrad leads on Weather / TimeMixer. As suggested, we compare PCGrad, MGDA, Nash-MTL, and GradNorm, and also include CAGrad, Dyadic + Mean, and an MSE-trained reference. The spectral methods use identical dyadic objectives and matched training conditions. The complete three-seed results below are also reported in Table 5 in Section 3.4.

Electricity / FreTS

MethodMSEMAE
MSE training0.19030 ± 0.000690.27951 ± 0.00052
Dyadic + Mean0.18785 ± 0.000510.27429 ± 0.00056
PCGrad0.18775 ± 0.000590.27386 ± 0.00042
MGDA0.19181 ± 0.000260.27761 ± 0.00064
CAGrad0.18804 ± 0.000270.27354 ± 0.00100
Nash-MTL0.18873 ± 0.001810.27433 ± 0.00174
GradNorm0.23762 ± 0.002050.33802 ± 0.00143
Ours0.18771 ± 0.000920.27333 ± 0.00135

Weather / FreTS

MethodMSEMAE
MSE training0.21400 ± 0.003600.27575 ± 0.00535
Dyadic + Mean0.20998 ± 0.003790.26080 ± 0.00784
PCGrad0.21036 ± 0.003850.26302 ± 0.00733
MGDA0.21226 ± 0.001870.26598 ± 0.00441
CAGrad0.20869 ± 0.003060.25824 ± 0.00591
Nash-MTL0.20829 ± 0.001510.25699 ± 0.00440
GradNorm0.21161 ± 0.001360.26252 ± 0.00521
Ours0.20783 ± 0.001800.25686 ± 0.00315

Electricity / TimeMixer

MethodMSEMAE
MSE training0.17691 ± 0.001800.27750 ± 0.00245
Dyadic + Mean0.19269 ± 0.000910.29690 ± 0.00199
PCGrad0.18888 ± 0.002850.29305 ± 0.00423
MGDA0.19093 ± 0.005160.28413 ± 0.00541
CAGrad0.18439 ± 0.006670.28544 ± 0.00854
Nash-MTL0.17402 ± 0.001880.27293 ± 0.00255
GradNorm0.19413 ± 0.001500.29822 ± 0.00134
Ours0.17333 ± 0.000910.27136 ± 0.00160

Weather / TimeMixer

MethodMSEMAE
MSE training0.20887 ± 0.000640.25249 ± 0.00086
Dyadic + Mean0.20695 ± 0.004830.24833 ± 0.00588
PCGrad0.20586 ± 0.003280.24735 ± 0.00380
MGDA0.22787 ± 0.002050.26402 ± 0.00170
CAGrad0.20729 ± 0.005470.24825 ± 0.00520
Nash-MTL0.20868 ± 0.002490.24759 ± 0.00197
GradNorm0.20634 ± 0.001690.24816 ± 0.00186
Ours0.20974 ± 0.001970.24894 ± 0.00141

DySAC has the lowest mean MSE and MAE in three of four settings; PCGrad leads on Weather / TimeMixer. Its largest gain over Mean is on Electricity / TimeMixer (10.05% MSE and 8.60% MAE reduction). These are three-seed means, not a claim of universal superiority.

AuthorsOpenReview ↗

Response to Reviewer PfQa (Part 2)

Q2: Can the authors provide results for prediction horizons of 96, 192, 336, and 720, comparing DySAC with MSE and strong baselines such as DBLoss and PS-Loss? In particular, does the gap between Dyadic+ConFIG and Dyadic+Mean increase at longer horizons?

Q2.1: Can the authors provide results for prediction horizons of 96, 192, 336, and 720, comparing DySAC with MSE and strong baselines such as DBLoss and PS-Loss?

A2.1: As suggested, we evaluated all five methods at horizons 96, 192, 336, and 720 on Electricity using FreTS and iTransformer. Methods share the same training conditions within each setting, and DySAC and Mean use identical dyadic objectives.

DySAC leads both mean metrics at all four Electricity/FreTS horizons and at Electricity/iTransformer horizons 192 and 336. At Electricity/iTransformer horizon 720, PS-Loss leads MSE and DBLoss leads MAE.

Each cell reports MSE / MAE (mean ± standard deviation over three seeds). Bold marks the lowest mean for each horizon and metric.

FreTS

MethodH=96H=192H=336H=720
MSE training0.18388 ± 0.00132 / 0.27321 ± 0.002070.19030 ± 0.00069 / 0.27951 ± 0.000520.20826 ± 0.00127 / 0.30002 ± 0.001710.24427 ± 0.00303 / 0.33115 ± 0.00186
Dyadic + Mean0.18359 ± 0.00100 / 0.26722 ± 0.001360.18785 ± 0.00051 / 0.27429 ± 0.000560.20381 ± 0.00181 / 0.29245 ± 0.002320.25121 ± 0.00293 / 0.34001 ± 0.00337
DBLoss0.18539 ± 0.00066 / 0.26741 ± 0.001220.19163 ± 0.00080 / 0.27605 ± 0.001180.20687 ± 0.00079 / 0.29298 ± 0.000900.24284 ± 0.00214 / 0.32512 ± 0.00211
PS-Loss0.18234 ± 0.00083 / 0.26710 ± 0.000820.19014 ± 0.00037 / 0.27582 ± 0.000850.20552 ± 0.00053 / 0.29335 ± 0.000680.24272 ± 0.00319 / 0.32688 ± 0.00279
Ours0.18214 ± 0.00084 / 0.26499 ± 0.001250.18771 ± 0.00092 / 0.27333 ± 0.001350.20019 ± 0.00063 / 0.28759 ± 0.000720.23548 ± 0.00113 / 0.31929 ± 0.00081

iTransformer

MethodH=96H=192H=336H=720
MSE training0.15262 ± 0.00017 / 0.24513 ± 0.000480.16584 ± 0.00039 / 0.25595 ± 0.000390.18049 ± 0.00117 / 0.27228 ± 0.000920.21786 ± 0.00209 / 0.30627 ± 0.00052
Dyadic + Mean0.15263 ± 0.00069 / 0.24300 ± 0.001020.16704 ± 0.00034 / 0.25672 ± 0.000110.18432 ± 0.00093 / 0.27582 ± 0.000750.23253 ± 0.00507 / 0.32350 ± 0.00462
DBLoss0.15162 ± 0.00069 / 0.24047 ± 0.000760.16666 ± 0.00027 / 0.25362 ± 0.000320.18245 ± 0.00120 / 0.26983 ± 0.001140.21509 ± 0.00103 / 0.29819 ± 0.00044
PS-Loss0.15052 ± 0.00040 / 0.24165 ± 0.000330.16526 ± 0.00004 / 0.25431 ± 0.000140.18072 ± 0.00099 / 0.27055 ± 0.001120.21237 ± 0.00128 / 0.29851 ± 0.00133
Ours0.15065 ± 0.00059 / 0.24035 ± 0.000610.16421 ± 0.00024 / 0.25252 ± 0.000150.18024 ± 0.00120 / 0.26919 ± 0.000970.21353 ± 0.00060 / 0.29860 ± 0.00034

Q2.2: In particular, does the gap between Dyadic+ConFIG and Dyadic+Mean increase at longer horizons?

A2.2: No, the coordination gain does not increase uniformly with horizon. On Electricity/iTransformer, MSE reductions over Mean increase from 1.30% to 1.69%, 2.22%, and 8.17%; the FreTS gains dip at horizon 192 before increasing at 336 and 720. The table reports relative MSE / MAE reductions as 100 × (Mean error − ConFIG error) / Mean error; positive values favor DySAC.

Dataset / backboneH=96H=192H=336H=720
Electricity / FreTS+0.79% / +0.83%+0.08% / +0.35%+1.77% / +1.66%+6.26% / +6.09%
Electricity / iTransformer+1.30% / +1.09%+1.69% / +1.64%+2.22% / +2.40%+8.17% / +7.70%

Q3: How often is the ConFIG fallback mechanism triggered during training? A high fallback rate would raise the question of whether the reported gains mainly come from simple gradient averaging.

A3: The fallback was triggered 0 times in 205,140 monitored optimizer steps, including 0 of 138,909 steps through the selected best-checkpoint epochs. Fallback means returning the arithmetic mean of the per-band gradients when the ConFIG Gram-system computation raises a runtime or linear-algebra error, or when its direction or scale is non-finite or its scale is non-positive. These counts cover 12 monitored Electricity/Weather × FreTS/TimeMixer runs with three seeds each.

DatasetBackboneSeedsFallback / all optimizer stepsFallback / steps through best epoch
WeatherFreTS30 / 68,6100 / 59,462
WeatherTimeMixer30 / 68,6100 / 20,583
ElectricityFreTS30 / 33,9600 / 28,300
ElectricityTimeMixer30 / 33,9600 / 30,564
Total120 / 205,1400 / 138,909

I thank the authors for the thorough rebuttal and supplementary experiments. All my concrete requests have been addressed, including comparisons against existing multi‑objective optimizers, multi‑horizon evaluations, and fallback trigger statistics.

Still, key concerns remain partially unresolved. Gains over the simple Dyadic+Mean baseline are setting‑dependent and sometimes modest. The gradient‑coordination solver is from prior work; novelty mainly comes from the spectral forecasting formulation and evaluation protocol, not a new optimizer. As noted by Reviewer jV4U, the causal link between gradient‑conflict diagnostics and test‑set performance under Adam is not fully established, and training overhead is substantial.

The manuscript would need to better explain the practical value of the formulation despite inconsistent gains, and clarify limitations of gradient‑coordination under adaptive optimizers like Adam.

Reviewer_jV4UOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DySAC, a model-agnostic training framework for time-series forecasting that treats prediction errors in different frequency bands as separate optimization objectives. The method partitions forecasting residuals into non-overlapping dyadic frequency bands and adopts the existing ConFIG algorithm to coordinate their gradients based on geometric relationships. DySAC does not modify the forecasting backbone or inference procedure. Experiments on seven datasets and four forecasting backbones show improvements over standard MSE training.

Critical strengths

1. Meaningful problem formulation.

The paper addresses an important limitation of conventional forecasting objectives: improvements in aggregate prediction error may hide performance degradation in individual frequency bands. Formulating frequency-specific errors as separate objectives provides a reasonable way to explicitly consider these trade-offs during optimization.

2. Broad experimental evaluation.

The experiments cover seven datasets and four different forecasting architectures. In addition to standard MSE and MAE, the paper evaluates frequency-specific improvements and task rankings. These results provide useful evidence that the proposed framework can improve forecasting performance across multiple frequency bands.

Critical weaknesses

1. Insufficient comparison with existing multi-objective optimization methods.

The proposed framework directly adopts the existing ConFIG algorithm for gradient coordination, but does not compare it with other established approaches such as PCGrad, MGDA, or CAGrad under the same spectral objectives. The current ablation study only compares ConFIG-based coordination with uniform gradient averaging.

2. Limited analysis of the gradient conflict mechanism.

Although the paper reports gradient conflict statistics, the analysis is restricted to the final training epoch on the Fred-md dataset. Furthermore, pairwise gradient conflicts and conflicts between individual gradients and the merged gradient are different measurements, rather than a direct before-and-after comparison. Without matched diagnostics for alternative gradient aggregation strategies, the results do not sufficiently establish how much the proposed coordination mechanism contributes to the observed forecasting improvements.

3. Relatively high computational overhead compared with the additional gains.

DySAC requires computing separate gradients for multiple frequency bands, increasing training-step time to approximately 2.52–3.30 times that of standard MSE training in the reported benchmark. However, the improvements over simple gradient averaging are relatively small in some settings. For example, the MSE decreases only from 0.1879 to 0.1877 on Electricity and from 0.2100 to 0.2078 on Weather.

Questions for authors
  1. Could the authors compare DySAC with PCGrad, MGDA, or CAGrad using exactly the same dyadic spectral objectives?

  2. Could the authors provide gradient conflict statistics throughout training and compare them with those of alternative gradient aggregation strategies?

  3. Given the substantial training overhead, could the authors provide a more detailed analysis of the performance–efficiency trade-off, particularly compared with the simpler Dyadic + Mean baseline?

6 comments
AuthorsOpenReview ↗

Response to Reviewer jV4U (Part 1)

Many thanks to Reviewer jV4U for providing a detailed review and insightful questions. We address the questions on matched comparisons, training-stage gradient diagnostics, and computational cost below.

Q1: Could the authors compare DySAC with PCGrad, MGDA, or CAGrad using exactly the same dyadic spectral objectives?

Weakness 1: Insufficient comparison with existing multi-objective optimization methods.

A1: Thank you for this suggestion. We added the requested controlled comparison to the main paper (Table 5 in Section 3.4); DySAC achieves the lowest mean MSE and MAE in three of four settings. Besides PCGrad, MGDA, and CAGrad, we include Nash-MTL, GradNorm, Dyadic + Mean, and an MSE-trained reference. The spectral methods use identical dyadic objectives and matched training conditions within each setting.

The tables below report the mean and sample standard deviation of MSE and MAE over three seeds for all eight methods across four dataset-backbone settings. Bold indicates the lowest mean for each metric. The same results appear in Table 5 in Section 3.4.

Electricity / FreTS

MethodMSEMAE
MSE training0.19030 ± 0.000690.27951 ± 0.00052
Dyadic + Mean0.18785 ± 0.000510.27429 ± 0.00056
PCGrad0.18775 ± 0.000590.27386 ± 0.00042
MGDA0.19181 ± 0.000260.27761 ± 0.00064
CAGrad0.18804 ± 0.000270.27354 ± 0.00100
Nash-MTL0.18873 ± 0.001810.27433 ± 0.00174
GradNorm0.23762 ± 0.002050.33802 ± 0.00143
Ours0.18771 ± 0.000920.27333 ± 0.00135

Weather / FreTS

MethodMSEMAE
MSE training0.21400 ± 0.003600.27575 ± 0.00535
Dyadic + Mean0.20998 ± 0.003790.26080 ± 0.00784
PCGrad0.21036 ± 0.003850.26302 ± 0.00733
MGDA0.21226 ± 0.001870.26598 ± 0.00441
CAGrad0.20869 ± 0.003060.25824 ± 0.00591
Nash-MTL0.20829 ± 0.001510.25699 ± 0.00440
GradNorm0.21161 ± 0.001360.26252 ± 0.00521
Ours0.20783 ± 0.001800.25686 ± 0.00315

Electricity / TimeMixer

MethodMSEMAE
MSE training0.17691 ± 0.001800.27750 ± 0.00245
Dyadic + Mean0.19269 ± 0.000910.29690 ± 0.00199
PCGrad0.18888 ± 0.002850.29305 ± 0.00423
MGDA0.19093 ± 0.005160.28413 ± 0.00541
CAGrad0.18439 ± 0.006670.28544 ± 0.00854
Nash-MTL0.17402 ± 0.001880.27293 ± 0.00255
GradNorm0.19413 ± 0.001500.29822 ± 0.00134
Ours0.17333 ± 0.000910.27136 ± 0.00160

Weather / TimeMixer

MethodMSEMAE
MSE training0.20887 ± 0.000640.25249 ± 0.00086
Dyadic + Mean0.20695 ± 0.004830.24833 ± 0.00588
PCGrad0.20586 ± 0.003280.24735 ± 0.00380
MGDA0.22787 ± 0.002050.26402 ± 0.00170
CAGrad0.20729 ± 0.005470.24825 ± 0.00520
Nash-MTL0.20868 ± 0.002490.24759 ± 0.00197
GradNorm0.20634 ± 0.001690.24816 ± 0.00186
Ours0.20974 ± 0.001970.24894 ± 0.00141

As can be seen, DySAC has the lowest mean MSE and MAE in three of four settings, supporting the effectiveness of ConFIG-based coordination for the shared spectral objectives in those settings. The largest improvement over Mean is on Electricity / TimeMixer: 10.05% lower MSE and 8.60% lower MAE. PCGrad leads on Weather / TimeMixer, indicating that relative performance depends on the dataset and backbone.

AuthorsOpenReview ↗

Response to Reviewer jV4U (Part 2)

Q2: Could the authors provide gradient conflict statistics throughout training and compare them with those of alternative gradient aggregation strategies?

Weakness 2: Limited analysis of the gradient conflict mechanism.

A2: As suggested, we added training-stage diagnostics for all eight methods across 96 verified runs on Electricity/Weather and FreTS/TimeMixer. Methods are evaluated throughout training on matched fixed probes. The table reports final-training mean raw pairwise conflict rates for DySAC, Mean, and the best-MSE competitor; the full trajectories accompany this response, and Table 5 in Section 3.4 reports forecasting errors.

Dataset / backboneOursDyadic + MeanBest-MSE competitor
Electricity / FreTS28.9%24.4%PCGrad 23.9%
Weather / FreTS12.2%12.8%Nash-MTL 8.9%
Electricity / TimeMixer3.9%17.8%Nash-MTL 3.3%
Weather / TimeMixer1.7%1.1%PCGrad 0.6%

Your distinction is correct: raw pairwise conflict and merged-direction conflict are separate measurements. DySAC does not minimize raw pairwise conflict in every setting. This rate counts negative cosines but omits gradient magnitudes, conflict severity, and the quality of the merged update. DySAC coordinates a shared direction and restores its scale (Eq. 8 in Section 2.2); raw pairwise conflict alone therefore cannot rank forecasting accuracy.

The matched merged-direction probes show that coordination can avoid negative band-direction inner products even when raw gradients remain in conflict. Along DySAC trajectories, Mean’s negative task-direction fraction reaches 13.9% in one Electricity/TimeMixer epoch, while ConFIG, PCGrad, MGDA, and CAGrad remain at zero on the same probes. This supports evaluating the merged direction separately from raw pairwise conflict. It does not establish a unique ConFIG advantage or a causal explanation of the test-error ranking; these candidate directions are measured before Adam’s update.

Q3: Given the substantial training overhead, could the authors provide a more detailed analysis of the performance–efficiency trade-off, particularly compared with the simpler Dyadic + Mean baseline?

Weakness 3: Relatively high computational overhead compared with the additional gains.

A3: As suggested, we provide a matched accuracy-cost comparison with Dyadic + Mean below. DySAC improves both mean errors in three of four settings, while its complete-run time is 0.2%-2.3% longer. Both methods use identical spectral objectives and matched training conditions; only the gradient merger changes. Positive error reductions favor DySAC. The separate MSE-relative training-step cost is 2.52-3.30x (Table 3 in Section 3.2; Table 18 in Appendix G).

Dataset / backboneMSE reductionMAE reductionFull-run time ratio
Electricity / FreTS+0.08%+0.35%1.002x
Weather / FreTS+1.02%+1.51%1.023x
Electricity / TimeMixer+10.05%+8.60%1.008x
Weather / TimeMixer-1.35%-0.25%1.015x

The benefit is setting-dependent: DySAC lowers both mean errors in three of four settings, most clearly on Electricity/TimeMixer (10.05% MSE and 8.60% MAE reductions), but the gain on Electricity/FreTS is only 0.08%/0.35%. Mean is both more accurate and faster on Weather/TimeMixer. The complete probe-instrumented runs take 0.2%-2.3% longer with DySAC; they include shared diagnostics and validation, so these ratios cannot be compared directly with the MSE-relative step costs. We claim neither a universal accuracy gain nor a runtime advantage. Inference is unchanged.

I appreciate the author’s diligent response to the review comments. However, I still have the following concerns:

  1. The initial gradient conflict rate of DySAC is not consistently lower than that of other multi-objective optimization methods. Furthermore, the diagnosis of combined directional conflicts lacks a systematic comparison across different algorithms, datasets, and the entire training process.

  2. Currently, gradient diagnosis is primarily performed before the Adam update; however, Adam’s momentum and coordinate-wise preconditioning may alter the actual direction of parameter updates. It has not been verified whether the objectives for each frequency band are still effectively improved following the Adam update.

  3. While the authors demonstrate both gradient conflict diagnosis and improvements in predictive performance, the link between the two has not been sufficiently established.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

We sincerely thank all reviewers for their thoughtful reviews and constructive suggestions. In particular, the reviewers recognized the following strengths:

  • The paper presents a meaningful problem formulation: treating frequency-specific errors as separate objectives provides a reasonable way to explicitly consider optimization trade-offs (Reviewer jV4U).
  • The experimental evaluation is broad (Reviewer jV4U), and the empirical results are reasonably consistent across different backbones (Reviewer PfQa). The ablation study is thorough and includes uncertainty bounds that improve the study's technical quality (Reviewer NVCr).
  • The paper is clearly written (Reviewer NVCr), and the experiments are reported with sufficient detail to allow cross-checking of the main tables and summary statistics (Reviewer PfQa).

We made four revisions to clarify our contribution and strengthen the evidence, including new results on how gradient conflict under Adam relates to joint training progress and forecasting accuracy:

1. Contribution and Theoretical Support [NVCr]

Reviewer NVCr notes that the frequency loss and the multi-objective optimization has numerous established studies, including in the time-series domain. We respectfully clarify a possible misunderstanding: our novelty claim concerns the forecasting formulation and training framework built on these foundations, rather than the spectral loss or MOO solver itself. To our knowledge, DySAC is the first forecasting framework to formulate dyadic frequency-band errors of the same forecast as interacting objectives and coordinate their gradients to address cross-band conflicts. This introduces a gradient-conflict perspective on how different frequency components compete during forecasting training.

We further propose an evaluation protocol aligned with these objectives, revealing which bands improve and which regress beyond aggregate MSE/MAE. Sections 2.1–2.3 describe the framework; Appendices A.2–A.3 distinguish its contribution from prior work. Appendix B and Table 4 in Section 3.4 provide conditional analysis and component evidence.

2. Matched Comparison with Multi-Objective Optimization (MOO) Baselines [jV4U, PfQa, NVCr]

We added a controlled comparison under identical dyadic objectives with PCGrad (NeurIPS 2020), MGDA (NeurIPS 2018), CAGrad (NeurIPS 2021), Nash-MTL (ICML 2022), GradNorm (ICML 2018), and uniform averaging, plus an MSE reference. DySAC has the lowest mean MSE/MAE in three of four settings; PCGrad leads on Weather/TimeMixer. Table 5 in Section 3.4 reports the complete three-seed results, with solver settings in Appendix J.1.

3. Gradient Conflict, Joint Optimization, and Forecasting [jV4U, PfQa]

Reviewers jV4U and PfQa asked how gradient conflict under Adam relates to joint progress across frequency bands and to forecasting accuracy. Because Adam can change the proposed direction, we measured conflict along actual Adam steps sampled across all training epochs and compared the six band losses before and after each step. Across FreTS and TimeMixer on Weather and Electricity, DySAC had the lowest conflict rate in three of four settings and ranked among the top three of eight methods in all four by the number of updates that reduced all six losses on the update batch. Table 23 in Appendix L.1 reports directional conflict; Tables 24–25 in Appendix L.2 report the observed loss decreases on the update and fixed validation batches. Appendix B.4 explains why joint progress on spectral L₁ training losses does not guarantee lower test MSE, which depends on total squared residual energy on held-out data.

In matched runs with diagnostic probes, DySAC reduced both mean errors in three of four settings and took 0.2%–2.3% longer than Dyadic + Mean. A separate benchmark reports a training-step cost of 2.52–3.30× that of MSE training (Table 3 in Section 3.2 of the main text; Table 18 in Appendix G). Inference is unchanged.

4. Additional Validation and Reproducibility [PfQa, NVCr]

Four-horizon comparisons on FreTS and iTransformer show that the gain over Mean depends on the dataset and backbone and does not increase uniformly with horizon. The fallback mechanism was triggered 0 times in 205,140 optimizer steps across 12 monitored FreTS/TimeMixer runs; its triggering conditions and mean-gradient replacement are explicitly defined. Appendices D.2–D.3 document the experimental settings. If published, we will release the complete implementation and reproduction details, including configurations, commands, available logs, and checkpoint identifiers.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.