acceptodds
Under review as a conference paper at ICLR 2027

Deep Time-series Forecasting Needs Kernelized Moment Balancing

Abstract

Deep time-series forecasting can be formulated as a distribution balancing problem aimed at aligning the distribution of the forecasts and ground truths. According to Imbens’ criterion, balancing two distributions requires that their first moments be identical with respect to any balancing function. We demonstrate that existing objectives enforce moment matching only for one or two predefined balancing functions, thus failing to achieve full distribution balance. To address this limitation, we propose direct forecasting with kernelized moment balancing (KMB-DF). Unlike existing objectives, KMB-DF adaptively selects the most informative balancing functions from a reproducing kernel Hilbert space (RKHS) to enforce sufficient distribution balancing. A tractable and differentiable formulation is derived to enable estimation from empirical samples and seamless integration into gradient-based training pipelines. Extensive experiments across multiple models and datasets show that KMB-DF consistently improves forecasting accuracy and achieves state-of-the-art performance. Code is available at https://anonymous.4open.science/r/KMB-DF-403C.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Forecasting, Reproducing Kernel Hilbert Space, Label Autocorrelation

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_bwAcOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

KMB-DF supplements MSE with kernel moment matching between joint history–label and history–forecast distributions. It selects sample-anchored kernels with large empirical discrepancies and proposes a soft-margin penalty. A conditional distribution-identification result motivates the method. Experiments cover seven datasets, multiple backbones and horizons, ablations, seed sensitivity, and computational measurements.

Critical strengths
  1. The common-backbone comparison is reasonably broad: Tables 2/9 compare ten alternative objectives on seven datasets, including QDF and DistDF. Reported average MSE improves on the strongest alternative in each dataset, suggesting a potentially useful training modification.
  2. The appendix materially strengthens the evaluation: Table 10 varies history length and includes PatchTST; Table 11 reports five seeds on two datasets; Table 12 measures loss computation. Released code clarifies implementation, and the loss adds no inference-time computation.
Critical weaknesses
  1. The statistical target and guarantee are insufficiently justified. Lemma 2.1's likelihood mismatch does not establish bias of the squared-error-optimal conditional mean: this remains E[Y|X] under correlated labels (Gneiting, 2011, §3.1). The deterministic forecast in §3.2 cannot generally equal a nondegenerate conditional label distribution, even with perfect optimization. Theorem 3.3/B.3 is a valid sufficient implication under its exact witness-span assumption, but does not establish that adaptive top-K selection satisfies that assumption or control the finite-sample, soft-margin approximation. The p.5 caveat about finite functions and p.9 regularizer limitation are welcome, but do not support the stronger claims of sufficient distribution balance and unbiased training in the abstract/introduction.
  2. Evidence does not isolate the added value over standard kernel matching. The discrepancy in Eq. (6) is an empirical MMD witness evaluation, up to normalization (Gretton et al., 2012, §2.3); informative finite kernel locations also have established precedent (Jitkrittum et al., 2016). A matched MSE-plus-MMD baseline is absent from Tables 2/9. Improvements over the strongest alternatives are only approximately 0.4–2.6% in average MSE. Table 11 usefully includes seeds, but only compares DF, and its ETTh1 KMB average is 0.429 versus 0.424 in Table 2. Thus the small advantage over strong competing objectives needs comparable uncertainty estimates and clarification of main-table aggregation.
Questions for authors
  1. Which configurations generated Tables 2/9, and were baseline tuning budgets comparable? The released ETTh1 script uses batch size 24 at horizon 192 and a learning rate outside §C.2's grid. Please reconcile the protocol and document bandwidth and discrepancy normalization.
  2. What population forecasting target is optimized when P(Y|X) has irreducible noise? Can the theoretical claim be connected to the actual finite-K, soft-margin objective, or narrowed to an empirically useful regularizer?
  3. Under matched validation/tuning budgets, how does KMB compare with MSE plus ordinary joint MMD? Please clarify main-table aggregation and provide paired multi-seed comparisons with the strongest competing objectives.
5 comments

Thank you for your constructive comments and recognition of our empirical evaluation, supporting appendices and presentation. Below are our responses to your specific concerns.


[W1] The statistical target and guarantee are insufficiently justified: likelihood mismatch does not establish conditional-mean bias, and deterministic forecasts cannot generally reproduce nondegenerate conditional distributions. Theorem 3.3/B.3 does not establish that adaptive top-K selection satisfies its witness-span assumption or control the finite-sample, soft-margin approximation. These limitations do not support the stronger claims of sufficient distribution balance and unbiased training in the abstract/introduction.

Response. Thank you for your insightful and meticulous comment. Our clarifications and corresponding revisions are detailed below.

  • First, we agree that likelihood mismatch does not imply conditional-mean bias. Nevertheless, our focus is autocorrelation bias, which Lemma 2.1 adopts from the NeurIPS paper [1, Theorem 3.1] to denote the bias of MSE loss from negative log-likelihood (NLL). For residual e=y−y^\mathbf{e}=\mathbf{y}-\hat{\mathbf{y}}, this bias is e⊤(Σ−1−I)e\mathbf{e}^{\top}(\mathbf{\Sigma}^{-1}-\mathbf{I})\mathbf{e}. It does not change the unrestricted MSE-optimal target E[y∣x]\mathbb{E}[\mathbf{y}|\mathbf{x}]. We also agree that a deterministic forecaster cannot represent all conditional randomness, while our target is to improve point forecasts through richer kernel-moment supervision. We have specified the Gaussian likelihood interpretation in Lemma 2.1 and Appendix B.1 and clarified the objective-level meaning of autocorrelation bias in Section 2.2.

  • Second, we clarify the connection between Top-K selection and the witness-span assumption. Top-K targets the worst-case discrepancies within the candidate kernel set. Under the witness-span assumption, reducing these discrepancies to zero in the population formulation satisfies the condition for distribution balancing in Theorem 3.3. And empirical soft-margin matching provides a tractable finite-data relaxation for training point forecasters. We have made span membership explicit in Theorem 3.3 and retained the finite-function approximation qualification in Section 3.3.

[1] Wang et al. Time-o1: Time-series Forecasting Needs Transformed Label Alignment. NeurIPS, 2025.

[W2,Q3] Evidence does not isolate the added value over standard kernel matching: Eq. (6) is an empirical MMD witness evaluation, and informative kernel locations have precedent. Under matched validation/tuning budgets, how does KMB compare with MSE plus ordinary joint MMD? Please clarify main-table aggregation, including the ETTh1 discrepancy between Tables 2 and 11 (0.424 versus 0.429), and provide paired multi-seed comparisons with the strongest competing objectives to assess the small average-MSE gains.

Response. Thank you for your thoughtful and actionable comment. Our comparisons and aggregation clarifications are detailed below.

  • First, we add experiments comparing KMB-DF with MSE plus ordinary joint MMD using the same exponential kernel (denoted as MMD@Exp) to assess its added value over standard kernel matching. The results below show that KMB-DF achieves lower MSE and MAE on all four datasets, supporting the benefit of informativeness-based selection with soft-margin balancing over ordinary MMD regularization. We have added these comparisons and analysis to Appendix D.8.
LossETTm1-MSEETTm1-MAEETTh1-MSEETTh1-MAEWeather-MSEWeather-MAEM5-MSEM5-MAE
MSE0.3780.3940.4360.4300.2410.2670.1680.327
MMD@Exp0.3740.3920.4330.4290.2410.2660.1560.315
KMB-DF0.3720.3910.4240.4260.2390.2650.1520.309
  • Second, we clarify the main-table aggregation: ETTh1 MSE 0.424 in Table 2 averages the four horizon results in Table 9, whereas 0.429 in Table 11 is the mean over five seeds. These are different aggregation strategies, not inconsistent results. We have further revised the table captions to clarify these aggregation strategies and specified the seed protocol in Appendix C.2.

  • Finally, we add paired multi-seed comparisons with QDF, DistDF, Time-o1, FreDF, DBLoss, and MSE over the same five seeds (1-5) on Weather and M5. The results below demonstrate that KMB-DF achieves the lowest mean MSE and MAE on both datasets. One-sided paired t-tests show significantly lower MSE than all six baselines on Weather and five on M5 (p<0.05p<0.05, marked with ∗^*), supporting its forecasting benefits across repeated runs. We have added these comparisons and analysis to Appendix D.10.

DatasetSeedKMB-DF-MSEKMB-DF-MAEQDF-MSEQDF-MAEDistDF-MSEDistDF-MAETime-o1-MSETime-o1-MAEFreDF-MSEFreDF-MAEDBLoss-MSEDBLoss-MAEMSE-MSEMSE-MAE
Weather10.2390.2650.2400.2670.2410.2690.2390.2650.2400.2660.2450.2660.2410.267
20.2390.2660.2410.2670.2420.2700.2400.2650.2430.2670.2530.2700.2410.267
30.2390.2650.2410.2670.2420.2710.2410.2660.2400.2660.2430.2650.2400.266
40.2390.2650.2410.2670.2410.2680.2400.2650.2400.2650.2470.2670.2410.268
50.2390.2660.2410.2670.2420.2710.2410.2660.2440.2680.2470.2670.2430.269
Mean0.2390.2650.241$^*$0.267$^*$0.242$^*$0.270$^*$0.240$^*$0.2650.242$^*$0.2660.247$^*$0.2670.241$^*$0.268$^*$
Std0.0000.0000.0000.0000.0010.0010.0010.0010.0020.0010.0040.0020.0010.001
M510.1510.3090.1530.3100.1510.3100.1560.3140.1620.3180.1580.3170.1630.320
20.1520.3100.1550.3130.1540.3120.1640.3210.1640.3230.1590.3180.1650.323
30.1520.3090.1540.3110.1600.3160.1590.3170.1740.3300.1650.3210.1730.328
40.1520.3120.1510.3100.1520.3110.1550.3140.1600.3200.1570.3160.1590.319
50.1480.3030.1550.3120.1530.3100.1570.3160.1630.3210.1590.3180.1620.319
Mean0.1510.3090.154$^*$0.3110.1540.3120.158$^*$0.316$^*$0.165$^*$0.322$^*$0.160$^*$0.318$^*$0.164$^*$0.322$^*$
Std0.0020.0030.0020.0010.0040.0020.0030.0030.0050.0050.0030.0020.0050.004

[Q1] Which configurations generated Tables 2/9, and were baseline tuning budgets comparable? The released ETTh1 script uses batch size 24 at horizon 192 and a learning rate outside §C.2's grid. Please reconcile the protocol and document bandwidth and discrepancy normalization.

Response. Thank you for your meticulous and observant comment. Our clarifications and corresponding revisions are detailed below.

  • First, tuning budgets for the common training parameters are comparable across baselines. All methods use the same learning-rate and batch-size search spaces, with configurations selected by validation MSE. As specified in Appendix C.2, the learning-rate grid is {5×10−3,2×10−3,10−3,5×10−4,2×10−4,10−4,5×10−5}\{5\times10^{-3},2\times10^{-3},10^{-3},5\times10^{-4},2\times10^{-4},10^{-4},5\times10^{-5}\} and the batch-size candidates are {8,16,32,128}\{8,16,32,128\}. Method-specific parameters follow their default settings. Tables 2/9 use CFPT backbone, history length 96, and seed 2023. The batch size 24 and off-grid learning rate came from an outdated debug script, not the configuration used for the reported results. We have corrected the batch-size search description in Appendix C.2 and will replace that script with the experimental configuration after the rebuttal stage.

  • Second, bandwidth is selected by validation MSE, using γ=1/(2σ2)\gamma=1/(2\sigma^2) and the search grid {0.05,0.1,0.2,0.3,0.5,1,2,3,5}\{0.05,0.1,0.2,0.3,0.5,1,2,3,5\}. We have completed the bandwidth search configuration in Appendix C.2.

  • Finally, we clarify the discrepancy normalization, as implemented in our released code (exp/exp_long_term_forecasting.py, lines 347--349). Kernel differences are averaged over the batch before applying the soft-margin hinge, and the resulting penalties are summed over selected anchors:

    loss_auxi = torch.mean(loss_auxi, dim=0)
    loss_auxi = F.relu(loss_auxi - self.args.C) + F.relu(-loss_auxi - self.args.C)
    loss_auxi = loss_auxi.sum()
    

    The batch mean prevents discrepancy magnitudes from scaling directly with batch size.

[Q2] What population forecasting target is optimized when P(Y∣X)\mathbb P(Y|X) has irreducible noise? Can the theoretical claim be connected to the actual finite-K, soft-margin objective, or narrowed to an empirically useful regularizer?

Response. Thank you for your constructive and insightful comment. Our clarifications and detailed responses are organized below.

  • First, we clarify the population forecasting target under irreducible noise. Consider noisy labels y~=y+ε\tilde{\mathbf{y}}=\mathbf{y}+\boldsymbol{\varepsilon}, where y\mathbf{y} is the underlying true label and ε\boldsymbol{\varepsilon} is the noise. The population forecasting target is

    Lnoise=(1−α)E(x,y~)∼P[∥y~−g(x)∥2]+αR,\mathcal{L}_{\text{noise}}=(1-\alpha)\mathbb{E}_{(\mathbf{x},\tilde{\mathbf{y}})\sim\mathbb{P}}\left[\|\tilde{\mathbf{y}}-g(\mathbf{x})\|^2\right]+\alpha R,

    where RR is the expected sum of selected soft-margin penalties between the observed joint samples (x,y~)(\mathbf{x},\tilde{\mathbf{y}}) and forecasts (x,y^)(\mathbf{x},\hat{\mathbf{y}}). Irreducible noise is present in the observations regardless of the loss used. KMB-DF balances prediction accuracy with kernel-moment agreement for the observed labels, rather than eliminating the noise itself.

  • Second, we clarify how the finite-K, soft-margin objective relates to Theorem 3.3. Top-K prioritizes the largest absolute discrepancies within the candidate kernel set. Under the theorem's span assumption, a zero exact population-balancing objective on the selected functions is necessary for perfect balance. The empirical soft-margin objective provides a tractable relaxation of exact moment matching, rather than equating a zero training penalty with perfect balance.

Reviewer_86mmOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes KMB-DF, a model-agnostic regularization objective for deep time-series forecasting. It constructs joint history–label and history–forecast trajectories, evaluates discrepancies in kernel moments at data-dependent anchors, and selects a small number of anchors with the largest reported imbalance. The resulting balancing penalty is combined with MSE, with a proposed soft-margin mechanism intended to tolerate small discrepancies. The theoretical analysis connects exact balancing of selected kernel moments to equality of distributions under an assumption on the span of the optimal MMD witness function. Experiments compare forecasting objectives on seven datasets and include component ablations, alternative kernels, multiple forecasting backbones, varying history lengths, selected five-seed experiments, and computational measurements.

Critical strengths
  1. The empirical evaluation provides encouraging evidence across several forecasting settings. Table 2 reports the lowest average MSE among the compared objectives on all seven datasets, including a reduction from 0.168 to 0.152 on M5 relative to MSE training. The additional experiments examine multiple backbones, history lengths, and random seeds. This breadth supports the potential practical usefulness of kernel-based regularization, even though some improvements over the strongest competing objectives are small.

  2. The proposed regularization mechanism is straightforward to integrate into existing forecasting models. Selecting data-dependent kernel anchors offers an interpretable way to focus the penalty on currently large moment discrepancies. The method does not require architectural changes or additional inference computation, and the paper includes ablations of anchor selection and the proposed margin mechanism, together with computational measurements.

Critical weaknesses
  1. The stated training objective is inconsistent with the proposed soft-margin formulation. Equation (7) imposes |δ_k| ≤ C + ε_k with ε_k ≥ 0. Eliminating the slack variable gives the penalty [|δ_k| − C]+, equivalently [δ_k − C]+ + [−δ_k − C]+. However, Equation (8) and Algorithm 1 instead use [−C − δ_k]+ + [δ_k + C]_+, which equals |δ_k + C|. This penalty is minimized at δ_k = −C rather than allowing an unpenalized interval around zero. Algorithm 1 also specifies Top-K selection using δ_k, whereas the surrounding text specifies |δ_k|. These may be transcription errors, but they affect the central objective and the interpretation of the ablation and margin-sensitivity experiments. Without clarification of the implementation used for the reported results, I cannot reliably assess the proposed mechanism.

  2. The theoretical justification does not establish the main claims for the actual forecasting algorithm. Correlation among future targets does not by itself make squared-error minimization biased for the conditional mean: the population minimizer remains E[Y|X], regardless of the off-diagonal conditional covariance. A mismatch with a particular likelihood model should therefore be distinguished from bias in point forecasting. Furthermore, a deterministic forecaster generally cannot reproduce a non-degenerate conditional distribution. Theorem 3.3 is a valid conditional statement under its witness-span assumption, but the paper does not show that selecting a few anchors by the proposed Top-K rule satisfies this assumption. Kernel universality alone does not establish it, and the theorem concerns exact population moment equality rather than finite-sample soft-margin matching. Consequently, the claims of sufficient distribution balancing and mitigation of autocorrelation bias are stronger than the established results. A comparison with standard MMD regularization using the same kernel would also help establish what the proposed selection mechanism contributes beyond existing kernel distribution matching.

Questions for authors
  1. Implementation clarification: Which penalty was actually used in the experiments: [|δ_k| − C]_+ or the expression |δ_k + C| printed in Equation (8)? Was Top-K applied to δ_k or |δ_k|? Please provide the relevant implementation excerpt and explain whether correcting the manuscript requires any experimental results to be updated. How was the “strict constraints” variant in the ablation implemented?

  2. Theoretical clarification: What precise notion of “bias” is intended in Lemma 2.1, and why would it imply biased conditional-mean forecasting under correlated labels? The main-text lemma refers to an identity covariance, while Appendix B.1 refers to a diagonal covariance; please reconcile these statements. What connection, if any, can be established between the actual Top-K anchor selection rule and the witness-span assumption in Theorem 3.3?

  3. Evidence needed to assess the contribution: Can the authors compare against MSE plus standard MMD using the same exponential kernel, and against random selection of the same number of anchors? These controls would help distinguish the benefits of kernel distribution matching from those of the proposed informativeness-based selection. Please also specify the kernel distance, bandwidth selection, loss normalization, multivariate aggregation, and whether competing objectives received comparable validation-based tuning.

  4. Experimental clarification: Were the strongest competing objectives, such as QDF and DistDF, evaluated over the same five seeds as KMB-DF? The repeated results in Table 11 are useful, but cover only the MSE baseline, while several main-table margins over stronger alternatives are small. Please clarify how the 50% confidence intervals in Figures 2 and 4 were constructed and explain the ETTm1, T=96 discrepancy between Table 3 (0.313/0.354) and Table 9 (0.315/0.352).

6 comments

Thank you for your encouraging assessment of our empirical evaluation and practical utility. Below are our responses to your specific concerns.


[W1,Q1] The stated training objective is inconsistent with the proposed soft-margin formulation, and Algorithm 1 specifies signed rather than absolute-score Top-K selection. Which penalty was actually used in the experiments: [∣δk∣−C]+[|\delta_k|-\mathrm{C}]_+ or ∣δk+C∣|\delta_k+\mathrm{C}|? Was Top-K applied to δk\delta_k or ∣δk∣|\delta_k|? Please provide the relevant implementation excerpt and explain whether correcting the manuscript requires any experimental results to be updated. How was the "strict constraints" variant in the ablation implemented?

Response. Thank you for your careful comment. Our clarifications and responses are detailed below.

  • First, the soft-margin expression in Eq. (8) contained a typographical error: the second hinge branch should be [δnk−C]+[\delta_{n_k}-\mathrm{C}]_+, not [δnk+C]+[\delta_{n_k}+\mathrm{C}]_+. The correct penalty is

    ξnk=[∣δnk∣−C]+=[−δnk−C]++[δnk−C]+.\xi_{n_k}=[|\delta_{n_k}|-\mathrm{C}]_+=[-\delta_{n_k}-\mathrm{C}]_++[\delta_{n_k}-\mathrm{C}]_+.

    This is the expression used in the released code, exp/exp_long_term_forecasting.py, lines 347--349:

    loss_auxi = torch.mean(loss_auxi, dim=0)
    loss_auxi = F.relu(loss_auxi - self.args.C) + F.relu(-loss_auxi - self.args.C)
    loss_auxi = loss_auxi.sum()
    

    After the mean reduction, loss_auxi contains the selected moment discrepancies. The next line computes their hinge penalties ξnk\xi_{n_k}, which are then summed. We have corrected the sign in Eq. (8) and Algorithm 1, step 5, to match this implementation.

  • Indeed, the corrected expression agrees with your derivation. In Eq. (7), ∣δnk∣≤C+εk|\delta_{n_k}|\leq\mathrm{C}+\varepsilon_k and εk≥0\varepsilon_k\geq0 give the minimum feasible slack εk⋆=[∣δnk∣−C]+\varepsilon_k^\star=[|\delta_{n_k}|-\mathrm{C}]_+. Thus, the penalty is zero throughout [−C,C][-\mathrm{C},\mathrm{C}], rather than minimized only at −C-\mathrm{C}.

  • Second, the Top-K is applied to ∣δk∣|\delta_k|, since discrepancies of either sign indicate imbalance. This is consistent in our released code: (utils/kernel_balaning.py, lines 221--222):

    _, topj_indices = torch.topk(torch.abs(alpha), k=actual_J)
    target_anchors = target_flat[topj_indices]
    

    Here, the code variable alpha denotes the moment difference δk\delta_k. We have corrected Algorithm 1, step 4, to fix the typo and explicitly rank with ∣δk∣|\delta_k|. Notably, both corrections above only concern the manuscript, the reported experiments used the correct implementations, so no results require updating.

  • Finally, "strict constraints" in KMB-DF$^\ddagger$ means setting C=0\mathrm{C}=0, yielding the penalty ∑k∣δnk∣\sum_k|\delta_{n_k}| so that any nonzero imbalance is penalized. Anchor selection is retained, and "strict" refers to zero-margin penalization rather than exact equality enforced by a constrained solver. We have explicitly added C=0\mathrm{C}=0 to the description of KMB-DF$^\ddagger$ in Section 4.3.

[W2,Q2] The theoretical justification does not establish the main claims for the actual forecasting algorithm: correlated targets do not imply conditional-mean bias, deterministic forecasts cannot generally reproduce non-degenerate conditional distributions, and exact population matching does not establish a finite-sample soft-margin guarantee. Consequently, the claims of sufficient distribution balancing and mitigation of autocorrelation bias are stronger than the established results. What precise notion of "bias" is intended in Lemma 2.1, and why would it imply biased conditional-mean forecasting under correlated labels? Please reconcile the main text's identity covariance with Appendix B.1's diagonal covariance. What connection, if any, can be established between the actual Top-K anchor selection rule and the witness-span assumption in Theorem 3.3?

Response. Thank you for your thoughtful and constructive comment. Our responses and corresponding revisions are detailed below.

  • First, we agree that correlated targets do not imply conditional-mean bias. Instead, our focus is autocorrelation bias, where the loss function is biased from the data likelihood. According to the NeurIPS paper [1, Theorem 3.1], this term can be formulated as the discrepancy between MSE and the negative log-likelihood (NLL). Let e=y−y^\mathbf{e}=\mathbf{y}-\hat{\mathbf{y}}; under a Gaussian model with fixed covariance matrix Σ\mathbf{\Sigma}, the NLL is

    LNLL=−log⁡p(y∣x)=12e⊤Σ−1e+12log⁡∣Σ∣+T2log⁡(2π).\mathcal{L}_{\mathrm{NLL}}=-\log p(\mathbf{y}|\mathbf{x})=\tfrac12\mathbf{e}^{\top}\mathbf{\Sigma}^{-1}\mathbf{e}+\tfrac12\log|\mathbf{\Sigma}|+\tfrac{\mathrm{T}}2\log(2\pi).

    Removing constants and using the same multiplicative scale gives LNLL=e⊤Σ−1e\mathcal{L}_{\mathrm{NLL}}=\mathbf{e}^{\top}\mathbf{\Sigma}^{-1}\mathbf{e}, whereas LMSE=e⊤e\mathcal{L}_{\mathrm{MSE}}=\mathbf{e}^{\top}\mathbf{e}. Then, the precise notion of "bias" in Lemma 2.1 is given by:

    LNLL−LMSE=e⊤(Σ−1−I)e\mathcal{L}_{\mathrm{NLL}}-\mathcal{L}_{\mathrm{MSE}}=\mathbf{e}^{\top}(\mathbf{\Sigma}^{-1}-\mathbf{I})\mathbf{e}

    which vanishes for all residuals when Σ=I\mathbf{\Sigma}=\mathbf{I}. It is a likelihood-objective mismatch, not statistical bias of a point estimator. We have aligned Lemma 2.1 and Appendix B.1 and specified that the comparison concerns the Gaussian negative log-likelihood, consistent with formulation in [1].

  • Second, "diagonal" in Appendix B.1 was a typographical error. The intended condition is Σ=I\mathbf{\Sigma}=\mathbf{I}, as in the main text and the derivation in [1]. We have replaced diagonal with identity in Appendix B.1 and aligned its statement with Lemma 2.1.

  • Finally, we clarify the connection between Top-K anchor selection and the witness-span assumption in Theorem 3.3. Top-K performs worst-case selection within the candidate kernel set by prioritizing the largest absolute moment discrepancies. Under the witness-span assumption, a zero exact population-balancing objective on the selected functions is necessary for perfect distribution balance. The empirical soft-margin objective in KMB-DF is a tractable relaxation of this exact matching criterion, as described in Section 3.3.

[1] Wang et al. Time-o1: Time-series Forecasting Needs Transformed Label Alignment. NeurIPS, 2025.

[W2,Q3] Can the authors compare against MSE plus standard MMD using the same exponential kernel, and against random selection of the same number of anchors? These controls would help distinguish the benefits of kernel distribution matching from those of the proposed informativeness-based selection. Please also specify the kernel distance, bandwidth selection, loss normalization, multivariate aggregation, and whether competing objectives received comparable validation-based tuning.

Response. Thank you for your actionable and meticulous suggestion. We have added both requested variants to improve the technical quality of this work. Our detailed responses and results are as follows.

  • First, we add experiments to compare KMB-DF with MSE plus standard MMD using the same exponential kernel (denoted as MMD@Exp). KMB-DF achieves lower MSE and MAE on all four datasets, supporting the effectiveness of its informativeness-based selection with soft-margin balancing compared with ordinary MMD regularization. We have added these comparisons and their analysis to Appendix D.8.
LossETTm1-MSEETTm1-MAEETTh1-MSEETTh1-MAEWeather-MSEWeather-MAEM5-MSEM5-MAE
MSE0.3780.3940.4360.4300.2410.2670.1680.327
MMD@Exp0.3740.3920.4330.4290.2410.2660.1560.315
KMB-DF0.3720.3910.4240.4260.2390.2650.1520.309
  • Second, we add experiments to compare informativeness-based selection (denoted as Selective) with random selection (denoted as Random) of the same number of anchors. Selective achieves better performance on all four datasets, substantiating the benefit of informativeness-based selection. We have added the results and analysis to Appendix D.7.
DatasetMethodMSEΔ\DeltaMSEMAEΔ\DeltaMAE
ETTm1DF0.378-0.394-
Random0.3750.8%↓\downarrow0.3930.4%↓\downarrow
Selective0.3721.6%↓\downarrow0.3910.7%↓\downarrow
ETTh1DF0.436-0.430-
Random0.4291.6%↓\downarrow0.4300.1%↓\downarrow
Selective0.4242.7%↓\downarrow0.4261.0%↓\downarrow
WeatherDF0.241-0.267-
Random0.2400.7%↓\downarrow0.2660.6%↓\downarrow
Selective0.2391.1%↓\downarrow0.2651.0%↓\downarrow
M5DF0.168-0.327-
Random0.1548.5%↓\downarrow0.3105.1%↓\downarrow
Selective0.1529.5%↓\downarrow0.3095.4%↓\downarrow
  • Finally, we clarify the kernel settings, loss reductions, and tuning protocol. We use the exponential kernel with non-squared Euclidean distance on flattened samples, K(z,z′)=exp⁡(−∥z−z′∥22σ2)\mathcal{K}(\mathbf{z},\mathbf{z}')=\exp(-\tfrac{\|\mathbf{z}-\mathbf{z}^{\prime}\|_2}{2\sigma^2}). We parameterize bandwidth by γ=1/(2σ2)\gamma=1/(2\sigma^2) and select γ\gamma from {0.05,0.1,0.2,0.3,0.5,1,2,3,5}\{0.05,0.1,0.2,0.3,0.5,1,2,3,5\} on validation data. For loss normalization, kernel differences are averaged over the batch before applying the soft-margin hinge, and the resulting penalties are summed over selected anchors. Multivariate aggregation uses an average across variables, without additional processing. Competing objectives retain their default method-specific parameters and use the same learning-rate and batch-size search spaces, with configurations selected by validation MSE. These common-parameter searches have been completed in Appendix C.2.
Reviewer_kDhCOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies learning objectives for deep time-series forecasting. It argues that (1) the standard MSE objective is biased under label autocorrelation (Lemma 2.1), and (2) existing objectives balance distributions with only one or two predefined balancing functions, falling short of Imbens' criterion. To this end, the paper proposes KMB-DF, which treats forecast training as a distribution balancing problem: the most informative kernel functions are adaptively selected from the RKHS of a universal kernel as balancing functions, and a soft-margin relaxation yields a tractable constrained objective. Theorem 3.3 provides an alignment guarantee conditioned on the optimal discrimination function lying in the span of the selected kernels. Experiments on seven datasets report consistent MSE/MAE gains over multiple competitive objectives.

Critical strengths

The paper identifies a genuine and under-explored limitation of existing forecasting objectives: the gap between matching moments of a few predefined functions and Imbens' criterion. It proposes a principled RKHS-based remedy that is both theoretically motivated and practically trainable via a differentiable soft-margin formulation. The empirical evaluation is also broad, spanning multiple datasets, backbones, and ablations of each designed component.

Critical weaknesses
  • The claim that the objective is fully differentiable overlooks the discrete top-K anchor-selection step. This optimization detail should be stated clearly.
  • The ablation does not adequately isolate the value of the proposed informativeness-based anchor selection. Comparison does not show that top-K selection by ∣δk∣|\delta_k| is superior to random or diversity-aware anchor subsets.
  • The authors could further strengthen the evaluation by comparing against direct MMD minimization as a discrepancy baseline.
  • The kernel bandwidth σ\sigma seems an important hyperparameter in the proposed method, but its sensitivity is not analyzed.

Minor fixes:

  • In line 274, the authors state that the K anchors with the largest ∣δk∣|\delta_k| are selected, while Algorithm 1 applies Top-K to the signed δk\delta_k.
  • The hinge expression in Eq. (8) appears inconsistent with the slack formulation in Eq. (7); please verify the signs and implementation.
  • In line 254 and line 279, the tolerance margin is typeset inconsistently, as italic CC in Eq. (7) but upright CC in Eq. (8).
  • In line 302, the efficiency analysis is referred to as In Appendix without pointing to the specific appendix section. Moreover, phrases of in Appendix would hamper the self-containness of the main text.
Questions for authors
  • In Theorem 3.3, is f∗f^{\ast} intended to belong to the linear span of the kernel functions K(⋅,zk)\mathcal{K}(\cdot, \mathbf{z}_k) for k=1,…,Kk=1,\ldots,K, rather than to the set of individual kernel functions?
  • How is κ\kappa in Eq. (7) related to α\alpha in Eq. (8)? Are they tuned independently?
  • In Algorithm 1, does N represent a mini-batch or the full training set? Moreover, are the anchors re-selected at every iteration? Clarifying these points would make the algorithm easier to understand.
5 comments

Thank you for your positive feedback and appreciation of our motivation, methodology, and empirical evaluation. Below are our responses to your specific concerns.


[W1] The claim that the objective is fully differentiable overlooks the discrete top-K anchor-selection step. This optimization detail should be stated clearly.

Response. Thank you for your constructive comment. We agree that the discrete top-K anchor-selection step is not differentiable. Nevertheless, the objective is piecewise differentiable and supports gradient-based optimization through the selected kernel penalties, with the anchor indices fixed during gradient computation. We have replaced "differentiable" with "piecewise differentiable" in the abstract and introduction. In Section 3.4, immediately after Eq. (8), we have specified that optimization operates "over the selected piecewise-differentiable penalties while treating the discrete top-K selection as fixed".

[W2] The ablation does not adequately isolate the value of the proposed informativeness-based anchor selection. Comparison does not show that top-K selection by ∣δk∣|\delta_k| is superior to random or diversity-aware anchor subsets.

Response. Thank you for your helpful suggestion. We agree that comparisons with random or diversity-aware anchor subsets are needed to assess the value of informativeness-based anchor selection beyond reducing the anchor count. Please kindly see our detailed response as follows.

  • Additional Experiments. We add experiments comparing informativeness-based selection (denoted as Selective) with random (denoted as Random) and diversity-aware (denoted as Diversity) selection using the same number of anchors on ETTm1, ETTh1, Weather, and M5. Diversity uses greedy kernel k-center selection on target samples, without using predictions or prediction errors.
DatasetMethodMSEΔ\DeltaMSEMAEΔ\DeltaMAE
ETTm1DF0.378-0.394-
Random0.3750.8%↓\downarrow0.3930.4%↓\downarrow
Diversity0.3741.1%↓\downarrow0.3930.4%↓\downarrow
Selective0.3721.6%↓\downarrow0.3910.7%↓\downarrow
ETTh1DF0.436-0.430-
Random0.4291.6%↓\downarrow0.4300.1%↓\downarrow
Diversity0.4311.0%↓\downarrow0.4300.0%↓\downarrow
Selective0.4242.7%↓\downarrow0.4261.0%↓\downarrow
WeatherDF0.241-0.267-
Random0.2400.7%↓\downarrow0.2660.6%↓\downarrow
Diversity0.2390.8%↓\downarrow0.2660.7%↓\downarrow
Selective0.2391.1%↓\downarrow0.2651.0%↓\downarrow
M5DF0.168-0.327-
Random0.1548.5%↓\downarrow0.3105.1%↓\downarrow
Diversity0.1557.9%↓\downarrow0.3134.4%↓\downarrow
Selective0.1529.5%↓\downarrow0.3095.4%↓\downarrow
  • Result Analysis. There are two primary observations. (i) Random and Diversity reduce MSE relative to DF on all four datasets, suggesting that kernel balancing can benefit forecasting even without informativeness-based selection. (ii) Selective achieves the lowest MSE and MAE on all four datasets. With the anchor count held constant, these results provide empirical evidence for the additional value of informativeness-based anchor selection over the tested random or diversity-aware anchor subsets.

  • Revision. We have added the above table and result analysis to Appendix D.7.

[W3] The authors could further strengthen the evaluation by comparing against direct MMD minimization as a discrepancy baseline.

Response. Thank you for your actionable and meticulous suggestion. We agree that direct MMD minimization is an important baseline for assessing the value of selective soft-margin balancing.

  • Additional Experiments. We add experiments with direct MMD minimization as an additional baseline. For fair comparison, we replace the discrepancy term in KMB-DF with the MMD terms, using Gaussian radial basis function (denoted as MMD@RBF) and exponential (denoted as MMD@Exp) kernels. Similar to other baselines, an MSE term is incorporated in the loss function. The results on ETTm1, ETTh1, Weather, and M5 are presented in the following table.
LossETTm1-MSEETTm1-MAEETTh1-MSEETTh1-MAEWeather-MSEWeather-MAEM5-MSEM5-MAE
MSE0.3780.3940.4360.4300.2410.2670.1680.327
MMD@RBF0.3750.3910.4330.4290.2410.2650.1570.316
MMD@Exp0.3740.3920.4330.4290.2410.2660.1560.315
KMB-DF0.3720.3910.4240.4260.2390.2650.1520.309
  • Result Analysis. While MMD is effective to improve forecasting performance, KMB-DF achieves lower MSE and MAE than both MMD variants on all four datasets. These results substantiate the benefit of selective soft-margin balancing over ordinary MMD regularization in time-series forecasting.
  • Revision. We have added the above table and result analysis to Appendix D.8.

[W4] The kernel bandwidth σ\sigma seems an important hyperparameter in the proposed method, but its sensitivity is not analyzed.

Response. Thank you for your insightful and constructive suggestion. We agree that evaluating the sensitivity of kernel bandwidth is beneficial to improve the technical quality.

  • Additional Experiments. We add experiments to assess bandwidth sensitivity on ETTm1 and ETTh1 by varying γ=1/(2σ2)\gamma=1/(2\sigma^2) from 0.05 to 5. Thus, increasing γ\gamma decreases the kernel bandwidth σ\sigma. The results are presented in the following table.
γ\gammaETTm1-MSEETTm1-Δ\DeltaMSEETTm1-MAEETTm1-Δ\DeltaMAEETTh1-MSEETTh1-Δ\DeltaMSEETTh1-MAEETTh1-Δ\DeltaMAE
DF0.378-0.394-0.436-0.430-
0.050.3731.4%↓\downarrow0.3910.8%↓\downarrow0.4340.3%↓\downarrow0.4300.1%↓\downarrow
0.10.3721.5%↓\downarrow0.3910.6%↓\downarrow0.4340.3%↓\downarrow0.4300.1%↓\downarrow
0.20.3750.9%↓\downarrow0.3930.4%↓\downarrow0.4350.3%↓\downarrow0.4300.1%↓\downarrow
0.30.3750.8%↓\downarrow0.3920.4%↓\downarrow0.4421.3%↑\uparrow0.4300.0%↑\uparrow
0.50.3750.8%↓\downarrow0.3930.4%↓\downarrow0.4350.1%↓\downarrow0.4290.2%↓\downarrow
10.3750.8%↓\downarrow0.3930.3%↓\downarrow0.4320.8%↓\downarrow0.4290.3%↓\downarrow
20.3750.8%↓\downarrow0.3930.3%↓\downarrow0.4301.2%↓\downarrow0.4300.2%↓\downarrow
30.3760.6%↓\downarrow0.3940.1%↓\downarrow0.4252.5%↓\downarrow0.4261.0%↓\downarrow
50.3750.8%↓\downarrow0.3930.3%↓\downarrow0.4311.2%↓\downarrow0.4290.2%↓\downarrow
  • Result Analysis. There are two primary observations. (i) All tested values reduce MSE relative to DF on ETTm1, and eight of nine do so on ETTh1, indicating forecasting gains across a broad bandwidth range. (ii) The lowest MSE occurs at γ=0.1\gamma=0.1 on ETTm1 and γ=3\gamma=3 on ETTh1. These results show that bandwidth sensitivity is dataset-dependent, rather than uniformly negligible.
  • Revision. We have added the bandwidth sensitivity results and analysis to Appendix D.9.

[M1] In line 274, the authors state that the K anchors with the largest ∣δk∣|\delta_k| are selected, while Algorithm 1 applies Top-K to the signed δk\delta_k.

Response. Thank you for your careful and observant comment. This was a typographical error in Algorithm 1: both the algorithm and its implementation select anchors by ∣δk∣|\delta_k|, not the signed δk\delta_k. We have corrected step 4 in Algorithm 1 to argTopK⁡j∣δj∣\operatorname{argTopK}_j|\delta_j|. The reported experiments used absolute-score selection, so no results require updating.

[M2] The hinge expression in Eq. (8) appears inconsistent with the slack formulation in Eq. (7); please verify the signs and implementation.

Response. Thank you for your thoughtful comment. This was a typographical error in the hinge expression in Eq. (8). The correct expression, consistent with the slack formulation in Eq. (7), is

ξnk=[∣δnk∣−C]+=[−δnk−C]++[δnk−C]+.\xi_{n_k}=[|\delta_{n_k}|-\mathrm{C}]_{+}=[-\delta_{n_k}-\mathrm{C}]_{+} + [\delta_{n_k}-\mathrm{C}]_{+}.

We have corrected Eq. (8) and the corresponding penalty computation in Algorithm 1, step 5.

[M3] In line 254 and line 279, the tolerance margin is typeset inconsistently, as italic CC in Eq. (7) but upright C\mathrm{C} in Eq. (8).

Response. Thank you for your meticulous comment. The italic CC was a typographical error, as we use upright notation for constants such as the tolerance margin C\mathrm{C}. We have corrected Eq. (7) and its accompanying definition to use C\mathrm{C} consistently.

[M4] In line 302, the efficiency analysis is referred to as In Appendix without pointing to the specific appendix section. Moreover, phrases of in Appendix would hamper the self-containness of the main text.

Response. Thank you for your helpful and constructive comment. We have added the specific reference to Appendix D.6 (Complexity Analysis) in the efficiency research question at the beginning of Section 4 and clarified that KMB-DF adds no inference computation. Apart from the experimental research questions, we have removed other appendix-referring statements from the main text to balance clarity and self-contained presentation.

[Q1] The notation f∗∈{K(⋅,zk)}k=1Kf^\ast \in \{\mathcal{K}(\cdot,{\mathbf{z}_k})\}_{k=1}^{K} in Theorem 3.3 is ambiguous. Does it imply f∗∈span {K(⋅,zk)}k=1Kf^\ast \in \text{span }\{\mathcal{K}(\cdot,\mathbf z_k)\}_{k=1}^{K}?

Response. Thank you for your insightful and meticulous comment. This was a typographical error: the intended condition is membership in the linear span of the selected kernel functions, meaning f∗(⋅)=∑k=1KαkK(⋅,zk)f^\ast(\cdot)=\sum_{k=1}^{\mathrm{K}} \alpha_k \mathcal{K}(\cdot,\mathbf{z}_k). Our proof uses this linear combination, rather than requiring f∗f^* to equal a single selected kernel function. By linearity, matching the selected kernel moments also matches the moment of f∗f^\ast. We have added the span notation in Theorem 3.3 and its restatement in Appendix B.

[Q2] How does κ\kappa in Eq. (7) relates to α\alpha in Eq. (8)? Are they independently tuned?

Response. Thank you for your thoughtful and observant comment. The two weights need not be independently tuned, because they parameterize the same trade-off under the sum normalization in Eqs. (7)--(8). Let

S=∑n=1N∥yn−y^n∥22,R=∑k=1K[∣δnk∣−C]+.S=\sum_{n=1}^{\mathrm{N}}\|\mathbf{y}_n-\hat{\mathbf{y}}_n\|_2^2,\qquad R=\sum_{k=1}^{\mathrm{K}}[|\delta_{n_k}|-\mathrm{C}]_+.

  • In Eq. (7), the minimum feasible slack is εk⋆=[∣δnk∣−C]+\varepsilon_k^\star=[|\delta_{n_k}|-\mathrm{C}]_+. Substituting it into the objective gives S+κRS+\kappa R.
  • Corrected Eq. (8) is (1−α)S+αR(1-\alpha)S+\alpha R. For 0<α<10<\alpha<1, dividing by the positive constant 1−α1-\alpha gives S+α1−αRS+\frac{\alpha}{1-\alpha}R. The objectives therefore have the same minimizers when κ=α1−α\kappa=\frac{\alpha}{1-\alpha}, equivalently α=κ1+κ\alpha=\frac{\kappa}{1+\kappa}.

We tune α\alpha rather than κ\kappa independently, and Section 4.6 reports its sensitivity analysis. At α=0\alpha=0, the balancing penalty is removed; at α=1\alpha=1, MSE is removed and no finite equivalent κ\kappa exists. We have added α=κ/(1+κ)\alpha=\kappa/(1+\kappa) directly to the definition of α\alpha following Eq. (8) in Section 3.4.

[Q3] In Algorithm 1, does N\mathrm N represent a mini-batch or the full training set? Moreover, do you re-select the anchors at every iteration? Clarify these points could make the algorithm table more clear.

Response. Thank you for your constructive comment. Algorithm 1 operates on the current mini-batch, not the full training set. The kernel discrepancies are recomputed from the current batch and predictions, and the anchors are re-selected at every iteration using the largest absolute discrepancies. We have introduced B\mathrm{B} for the batch size in Section 3.4, Algorithm 1, and Eq. (8). The dataset-size notation N\mathrm{N} in the preceding formulation remains unchanged.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Author Final Remarks

Dear AC and all reviewers,

We sincerely appreciate your commitment to evaluating our paper despite your busy schedules. We are encouraged that all three reviewers have raised their ratings, with scores 4 with conf 3, 3 with conf 2, and 3 with conf 3, recognizing our paper

  • "identifies a genuine and under-explored limitation of existing forecasting objectives" and "proposes a principled RKHS-based remedy" (Reviewer kDhC),
  • "provides encouraging evidence across several forecasting settings" with a mechanism that is "straightforward to integrate into existing forecasting models" (Reviewer 86mm),
  • reports average MSE that "improves on the strongest alternative in each dataset", with a loss that "adds no inference-time computation" (Reviewer bwAc).

We are grateful to all reviewers for their helpful comments regarding our paper. Below is a summary of our responses:

  • To Reviewer kDhC: We have clarified the discrete Top-K selection step, added comparisons with random and diversity-aware anchor selection and direct MMD minimization, and included a kernel-bandwidth sensitivity analysis. We have also corrected the algorithm and soft-margin notation.
  • To Reviewer 86mm: We have provided implementation excerpts for the soft-margin penalty and absolute-score anchor selection, clarified the conditional scope of the witness-span guarantee and the distinction between likelihood mismatch and conditional-mean bias, and added MMD, anchor-selection, and paired multi-seed comparisons.
  • To Reviewer bwAc: We have clarified the population forecasting objective under noisy labels, the scope of the theoretical guarantee, tuning protocols, and table aggregation. We have also added matched-kernel MMD comparisons and paired multi-seed tests against strong competing objectives. Our added experiments support the value of informativeness-based selection with soft-margin balancing over standard MMD regularization.

We are pleased that all reviewers appreciate the efforts above to improve the manuscript. We sincerely thank all reviewers for their effort, thoughtfulness, and generosity in evaluating this paper.

Many thanks,

Authors of #54166

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.