- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
KMB-DF supplements MSE with kernel moment matching between joint history–label and history–forecast distributions. It selects sample-anchored kernels with large empirical discrepancies and proposes a soft-margin penalty. A conditional distribution-identification result motivates the method. Experiments cover seven datasets, multiple backbones and horizons, ablations, seed sensitivity, and computational measurements.
Critical strengths
- The common-backbone comparison is reasonably broad: Tables 2/9 compare ten alternative objectives on seven datasets, including QDF and DistDF. Reported average MSE improves on the strongest alternative in each dataset, suggesting a potentially useful training modification.
- The appendix materially strengthens the evaluation: Table 10 varies history length and includes PatchTST; Table 11 reports five seeds on two datasets; Table 12 measures loss computation. Released code clarifies implementation, and the loss adds no inference-time computation.
Critical weaknesses
- The statistical target and guarantee are insufficiently justified. Lemma 2.1's likelihood mismatch does not establish bias of the squared-error-optimal conditional mean: this remains E[Y|X] under correlated labels (Gneiting, 2011, §3.1). The deterministic forecast in §3.2 cannot generally equal a nondegenerate conditional label distribution, even with perfect optimization. Theorem 3.3/B.3 is a valid sufficient implication under its exact witness-span assumption, but does not establish that adaptive top-K selection satisfies that assumption or control the finite-sample, soft-margin approximation. The p.5 caveat about finite functions and p.9 regularizer limitation are welcome, but do not support the stronger claims of sufficient distribution balance and unbiased training in the abstract/introduction.
- Evidence does not isolate the added value over standard kernel matching. The discrepancy in Eq. (6) is an empirical MMD witness evaluation, up to normalization (Gretton et al., 2012, §2.3); informative finite kernel locations also have established precedent (Jitkrittum et al., 2016). A matched MSE-plus-MMD baseline is absent from Tables 2/9. Improvements over the strongest alternatives are only approximately 0.4–2.6% in average MSE. Table 11 usefully includes seeds, but only compares DF, and its ETTh1 KMB average is 0.429 versus 0.424 in Table 2. Thus the small advantage over strong competing objectives needs comparable uncertainty estimates and clarification of main-table aggregation.
Questions for authors
- Which configurations generated Tables 2/9, and were baseline tuning budgets comparable? The released ETTh1 script uses batch size 24 at horizon 192 and a learning rate outside §C.2's grid. Please reconcile the protocol and document bandwidth and discrepancy normalization.
- What population forecasting target is optimized when P(Y|X) has irreducible noise? Can the theoretical claim be connected to the actual finite-K, soft-margin objective, or narrowed to an empirically useful regularizer?
- Under matched validation/tuning budgets, how does KMB compare with MSE plus ordinary joint MMD? Please clarify main-table aggregation and provide paired multi-seed comparisons with the strongest competing objectives.
5 comments
[W2,Q3] Evidence does not isolate the added value over standard kernel matching: Eq. (6) is an empirical MMD witness evaluation, and informative kernel locations have precedent. Under matched validation/tuning budgets, how does KMB compare with MSE plus ordinary joint MMD? Please clarify main-table aggregation, including the ETTh1 discrepancy between Tables 2 and 11 (0.424 versus 0.429), and provide paired multi-seed comparisons with the strongest competing objectives to assess the small average-MSE gains.
Response. Thank you for your thoughtful and actionable comment. Our comparisons and aggregation clarifications are detailed below.
- First, we add experiments comparing KMB-DF with MSE plus ordinary joint MMD using the same exponential kernel (denoted as MMD@Exp) to assess its added value over standard kernel matching. The results below show that KMB-DF achieves lower MSE and MAE on all four datasets, supporting the benefit of informativeness-based selection with soft-margin balancing over ordinary MMD regularization. We have added these comparisons and analysis to Appendix D.8.
| Loss | ETTm1-MSE | ETTm1-MAE | ETTh1-MSE | ETTh1-MAE | Weather-MSE | Weather-MAE | M5-MSE | M5-MAE |
|---|---|---|---|---|---|---|---|---|
| MSE | 0.378 | 0.394 | 0.436 | 0.430 | 0.241 | 0.267 | 0.168 | 0.327 |
| MMD@Exp | 0.374 | 0.392 | 0.433 | 0.429 | 0.241 | 0.266 | 0.156 | 0.315 |
| KMB-DF | 0.372 | 0.391 | 0.424 | 0.426 | 0.239 | 0.265 | 0.152 | 0.309 |
-
Second, we clarify the main-table aggregation: ETTh1 MSE 0.424 in Table 2 averages the four horizon results in Table 9, whereas 0.429 in Table 11 is the mean over five seeds. These are different aggregation strategies, not inconsistent results. We have further revised the table captions to clarify these aggregation strategies and specified the seed protocol in Appendix C.2.
-
Finally, we add paired multi-seed comparisons with QDF, DistDF, Time-o1, FreDF, DBLoss, and MSE over the same five seeds (1-5) on Weather and M5. The results below demonstrate that KMB-DF achieves the lowest mean MSE and MAE on both datasets. One-sided paired t-tests show significantly lower MSE than all six baselines on Weather and five on M5 (, marked with ), supporting its forecasting benefits across repeated runs. We have added these comparisons and analysis to Appendix D.10.
| Dataset | Seed | KMB-DF-MSE | KMB-DF-MAE | QDF-MSE | QDF-MAE | DistDF-MSE | DistDF-MAE | Time-o1-MSE | Time-o1-MAE | FreDF-MSE | FreDF-MAE | DBLoss-MSE | DBLoss-MAE | MSE-MSE | MSE-MAE |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Weather | 1 | 0.239 | 0.265 | 0.240 | 0.267 | 0.241 | 0.269 | 0.239 | 0.265 | 0.240 | 0.266 | 0.245 | 0.266 | 0.241 | 0.267 |
| 2 | 0.239 | 0.266 | 0.241 | 0.267 | 0.242 | 0.270 | 0.240 | 0.265 | 0.243 | 0.267 | 0.253 | 0.270 | 0.241 | 0.267 | |
| 3 | 0.239 | 0.265 | 0.241 | 0.267 | 0.242 | 0.271 | 0.241 | 0.266 | 0.240 | 0.266 | 0.243 | 0.265 | 0.240 | 0.266 | |
| 4 | 0.239 | 0.265 | 0.241 | 0.267 | 0.241 | 0.268 | 0.240 | 0.265 | 0.240 | 0.265 | 0.247 | 0.267 | 0.241 | 0.268 | |
| 5 | 0.239 | 0.266 | 0.241 | 0.267 | 0.242 | 0.271 | 0.241 | 0.266 | 0.244 | 0.268 | 0.247 | 0.267 | 0.243 | 0.269 | |
| Mean | 0.239 | 0.265 | 0.241$^*$ | 0.267$^*$ | 0.242$^*$ | 0.270$^*$ | 0.240$^*$ | 0.265 | 0.242$^*$ | 0.266 | 0.247$^*$ | 0.267 | 0.241$^*$ | 0.268$^*$ | |
| Std | 0.000 | 0.000 | 0.000 | 0.000 | 0.001 | 0.001 | 0.001 | 0.001 | 0.002 | 0.001 | 0.004 | 0.002 | 0.001 | 0.001 | |
| M5 | 1 | 0.151 | 0.309 | 0.153 | 0.310 | 0.151 | 0.310 | 0.156 | 0.314 | 0.162 | 0.318 | 0.158 | 0.317 | 0.163 | 0.320 |
| 2 | 0.152 | 0.310 | 0.155 | 0.313 | 0.154 | 0.312 | 0.164 | 0.321 | 0.164 | 0.323 | 0.159 | 0.318 | 0.165 | 0.323 | |
| 3 | 0.152 | 0.309 | 0.154 | 0.311 | 0.160 | 0.316 | 0.159 | 0.317 | 0.174 | 0.330 | 0.165 | 0.321 | 0.173 | 0.328 | |
| 4 | 0.152 | 0.312 | 0.151 | 0.310 | 0.152 | 0.311 | 0.155 | 0.314 | 0.160 | 0.320 | 0.157 | 0.316 | 0.159 | 0.319 | |
| 5 | 0.148 | 0.303 | 0.155 | 0.312 | 0.153 | 0.310 | 0.157 | 0.316 | 0.163 | 0.321 | 0.159 | 0.318 | 0.162 | 0.319 | |
| Mean | 0.151 | 0.309 | 0.154$^*$ | 0.311 | 0.154 | 0.312 | 0.158$^*$ | 0.316$^*$ | 0.165$^*$ | 0.322$^*$ | 0.160$^*$ | 0.318$^*$ | 0.164$^*$ | 0.322$^*$ | |
| Std | 0.002 | 0.003 | 0.002 | 0.001 | 0.004 | 0.002 | 0.003 | 0.003 | 0.005 | 0.005 | 0.003 | 0.002 | 0.005 | 0.004 |
[Q1] Which configurations generated Tables 2/9, and were baseline tuning budgets comparable? The released ETTh1 script uses batch size 24 at horizon 192 and a learning rate outside §C.2's grid. Please reconcile the protocol and document bandwidth and discrepancy normalization.
Response. Thank you for your meticulous and observant comment. Our clarifications and corresponding revisions are detailed below.
-
First, tuning budgets for the common training parameters are comparable across baselines. All methods use the same learning-rate and batch-size search spaces, with configurations selected by validation MSE. As specified in Appendix C.2, the learning-rate grid is and the batch-size candidates are . Method-specific parameters follow their default settings. Tables 2/9 use CFPT backbone, history length 96, and seed 2023. The batch size 24 and off-grid learning rate came from an outdated debug script, not the configuration used for the reported results. We have corrected the batch-size search description in Appendix C.2 and will replace that script with the experimental configuration after the rebuttal stage.
-
Second, bandwidth is selected by validation MSE, using and the search grid . We have completed the bandwidth search configuration in Appendix C.2.
-
Finally, we clarify the discrepancy normalization, as implemented in our released code (
exp/exp_long_term_forecasting.py, lines 347--349). Kernel differences are averaged over the batch before applying the soft-margin hinge, and the resulting penalties are summed over selected anchors:loss_auxi = torch.mean(loss_auxi, dim=0) loss_auxi = F.relu(loss_auxi - self.args.C) + F.relu(-loss_auxi - self.args.C) loss_auxi = loss_auxi.sum()The batch mean prevents discrepancy magnitudes from scaling directly with batch size.
[Q2] What population forecasting target is optimized when has irreducible noise? Can the theoretical claim be connected to the actual finite-K, soft-margin objective, or narrowed to an empirically useful regularizer?
Response. Thank you for your constructive and insightful comment. Our clarifications and detailed responses are organized below.
-
First, we clarify the population forecasting target under irreducible noise. Consider noisy labels , where is the underlying true label and is the noise. The population forecasting target is
where is the expected sum of selected soft-margin penalties between the observed joint samples and forecasts . Irreducible noise is present in the observations regardless of the loss used. KMB-DF balances prediction accuracy with kernel-moment agreement for the observed labels, rather than eliminating the noise itself.
-
Second, we clarify how the finite-K, soft-margin objective relates to Theorem 3.3. Top-K prioritizes the largest absolute discrepancies within the candidate kernel set. Under the theorem's span assumption, a zero exact population-balancing objective on the selected functions is necessary for perfect balance. The empirical soft-margin objective provides a tractable relaxation of exact moment matching, rather than equating a zero training penalty with perfect balance.
Thank you for your constructive comments and recognition of our empirical evaluation, supporting appendices and presentation. Below are our responses to your specific concerns.
Response. Thank you for your insightful and meticulous comment. Our clarifications and corresponding revisions are detailed below.
First, we agree that likelihood mismatch does not imply conditional-mean bias. Nevertheless, our focus is autocorrelation bias, which Lemma 2.1 adopts from the NeurIPS paper [1, Theorem 3.1] to denote the bias of MSE loss from negative log-likelihood (NLL). For residual e=y−y^, this bias is e⊤(Σ−1−I)e. It does not change the unrestricted MSE-optimal target E[y∣x]. We also agree that a deterministic forecaster cannot represent all conditional randomness, while our target is to improve point forecasts through richer kernel-moment supervision. We have specified the Gaussian likelihood interpretation in Lemma 2.1 and Appendix B.1 and clarified the objective-level meaning of autocorrelation bias in Section 2.2.
Second, we clarify the connection between Top-K selection and the witness-span assumption. Top-K targets the worst-case discrepancies within the candidate kernel set. Under the witness-span assumption, reducing these discrepancies to zero in the population formulation satisfies the condition for distribution balancing in Theorem 3.3. And empirical soft-margin matching provides a tractable finite-data relaxation for training point forecasters. We have made span membership explicit in Theorem 3.3 and retained the finite-function approximation qualification in Section 3.3.
[1] Wang et al. Time-o1: Time-series Forecasting Needs Transformed Label Alignment. NeurIPS, 2025.