- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper motivates itself by arguing that probabilistic forecasts of the same history at different horizons should agree on their shared future prefix (horizon-wise projective consistency). It shows that (sota) generative models, trained separately per horizon, are not consistent. They propose PrefixFlow, with Prefix-masked conditioning and a left-causal velocity network, to make the prefix computation independent of the suffix, which together with a factorized Gaussian source guarantees consistency. PrefixFlow is trained jointly across horizons.
Critical strengths
- They define the problem of consistency clearly and point out that current metric do not capture it.
- The consistency evaluation is well done.
- The single jointly trained model is better in metric than the horizon-specific baselines.
Critical weaknesses
- It is unclear if the inconsistency is a horizon effect. The analysis compares separately trained models per horizon. However, two independently trained models are expected to disagree, even at the same horizon. The paper does not controll for training variance (seeds etc). The authors’ own Separate Training variant (Excess ED 1.59 with an identical architecture) is consistent with this concern.
- The practical relevance of consistency is not demonstrated. For the true conditional distribution the marginals are consistent by definition, but it does not follow that disagreement between imperfect models is harmful. Consistency is not correctness: a model can be perfectly consistent and poorly calibrated, and two inconsistent forecasts can be equally well calibrated. MOSES (Yalavarthi et al., 2026) shows how this can be tested: it measures the cost of inconsistency by comparing marginal predictive accuracy. An analogous experiment here, scoring the baselines’ Direct-h and Crop-from-H forecasts against ground truth on the shared prefix, would show whether their inconsistency actually hurts. Also I find the paper to not adequately compare qualitatively and quantitatively to the MOSES paper.
- Joint training across horizons appears to do most of the work. In table 3 (ETTh1 96 720) wo both almost matches the causal coinstrained one, i.e., the causal constraints only remove this small residual and actually worsen the CRPS. Further, a baseline is missing: a single model trained on all horizons with no causal structure and an explicit horizon encoding.
- Limited theoretical novelty given MOSES.
Questions for authors
See waeknesses.
3 comments
Response to Reviewer LU2r (2/2)
3. Joint training and explicit horizon conditioning [W3]
We agree that joint multi-horizon training already produces strong cross-horizon alignment. We view this as a soft consistency bias: sharing one model and training it across horizons encourages the corresponding predictive distributions to agree, even without the structural constraints. This is also why w/o Both performs well in Table 3 and slightly outperforms Full in CRPS.
The reviewer also suggested a shared noncausal model with an explicit horizon encoding. We added this baseline by augmenting w/o Both with an MLP embedding of the requested horizon, while keeping the model noncausal and without prefix masking.
ETTh1 results for jointly trained shared models. Forecasting columns report CRPS.
| Model | H96 | H192 | H336 | H720 | Excess ED 96–192 | Excess ED 96–720 |
|---|---|---|---|---|---|---|
| PrefixFlow | 0.2972 | 0.3145 | 0.3235 | 0.3375 | 0.0010 | 0.0013 |
| w/o Both | 0.2953 | 0.3112 | 0.3197 | 0.3342 | 0.0560 | 0.1697 |
| + Horizon Encoding | 0.2962 | 0.3120 | 0.3199 | 0.3323 | 0.0563 | 0.1903 |
Explicit horizon conditioning leaves forecasting accuracy essentially unchanged, but does not reduce the cross-horizon discrepancy. Thus, joint training can learn substantial empirical alignment, but neither parameter sharing nor explicit knowledge of the requested horizon is sufficient to make the resulting distributions projectively consistent.
4. Relation to MOSES and theoretical novelty [W2, second part + W4]
We agree that MOSES is closely related and should have been discussed more clearly in the original submission.
MOSES focuses on probabilistic forecasting for irregular time series, where future queries may contain arbitrary time points and channels. To remain consistent under arbitrary subqueries, MOSES uses separable flow transformations, with dependence across queried variables mainly handled by the Gaussian source and mixture components. The paper also discusses the expressivity trade-off introduced by this separability.
Our problem is different: we consider regular-grid probabilistic forecasting with variable horizons. The query sets are not arbitrary; they are nested prefixes. PrefixFlow only requires the velocity field on an existing prefix to remain unchanged when a suffix is appended. This keeps temporal interactions within the prefix while preserving consistency across horizons.
So the novelty is not the idea of consistency itself, but how to enforce it for nested forecasting horizons without making the flow fully separable. We additionally show that this prefix property is preserved by the Euler discretization used at inference.
We also compared MOSES and PrefixFlow at on three regular-grid benchmarks.
Forecasting results at . Each entry is MSE / CRPS.
| Dataset | PrefixFlow | MOSES |
|---|---|---|
| ETTh1 | 0.3760 / 0.2972 | 1.3877 / 0.6428 |
| ETTm1 | 0.2789 / 0.2468 | 1.0918 / 0.6274 |
| Weather | 0.1458 / 0.1430 | 0.5076 / 0.4282 |
PrefixFlow performs substantially better on all three datasets in this regular-grid forecasting setting.
Thanks a lot for the detailed rebuttal and providing most of the experiments that I asked for. I especially appreciate providing results that did not promote the method. However, these results do mostly confirm my concerns that joint training already captures most of the inconsistency and especially when considering the seed variance, most remaining differences are within run variances. At the same time forcing a causal structure removes most of the benefits that joint modelling provides.
Overall, I see this more as a investigative report of cross-horizon consistency showing very limited impact and keep my score.
One last remark: the moses results seem implausibly poor to me, maybe the setup was misconfigured.
Response to Reviewer LU2r (1/2)
Thank you for the detailed feedback. We added several controlled experiments and clarified the relation to MOSES.
1. Separately trained models vs. horizon sensitivity [W1]
We agree that the experiment in Sec. 3.2 changes both the forecast horizon and the trained model, so it does not isolate horizon sensitivity. The experiment was intended to reflect the usual horizon-specific setup of these baselines, but independent training can clearly contribute to the disagreement.
We repeated the test with a single jointly trained checkpoint, varying only the requested horizon.
Excess ED from the same checkpoint evaluated at different horizons; 95% bootstrap confidence intervals are shown.
With the same checkpoint, w/o Both still shows clear cross-horizon disagreement, while PrefixFlow remains near zero.
2. Does inconsistency have a predictive cost? [W2, first part]
We agree with the reviewer that consistency is not correctness: exact consistency alone does not imply better calibration or lower CRPS. Following the suggested MOSES-style test, we score the same H96 target using either a direct H96 forecast or the H96 prefix cropped from an H720 forecast.
CRPS for the same H96 target obtained through Direct-H96 or Crop-from-H720, using 256 predictive samples.
The baseline results can change substantially between Direct and Crop, but the direction varies across models and datasets. We therefore do not claim that inconsistency always lowers predictive accuracy. For these baselines, Direct and Crop come from separately trained horizon-specific models, so the difference is not purely a horizon effect.
We also test whether the structural constraint helps when shorter horizons receive no direct supervision. PrefixFlow and w/o Both are trained only at H720, using the same coverage-aware positional weighting; H96/H192/H336 are never used for training. Direct and Crop use the same checkpoint and the same H720-valid test origins.
H720-only training on ETTh1. Each entry is MSE / CRPS.
Under H720-only supervision, PrefixFlow gives better direct forecasts than w/o Both at all shorter horizons. In PrefixFlow, the distribution queried at a shorter horizon is exactly the corresponding marginal of the supervised H720 forecast, so H720 training constrains those shorter-horizon predictions as well. This link is absent in w/o Both.