acceptodds
Under review as a conference paper at ICLR 2027

DiverseFlow: Jointly Modeling Forecasting Diversity and Uncertainty for Time Series Forecasting

Abstract

Time-series forecasting is fundamental to many real-world applications, however, temporal observations often exhibit non-stationary dynamics and inherent uncertainty. Existing end-to-end forecasters typically learn a single deterministic mapping from historical observations to future values, which can limit forecast diversity and fails to characterize multiple plausible future outcomes. We propose DiverseFlow, it is a unified framework that jointly models forecast diversity and predictive uncertainty. DiverseFlow employs multiple structurally aligned predictors to capture complementary predictive views and introduces conditional residual flow matching to model flexible predictive distributions around deterministic forecasts. To stabilize distribution learning across heterogeneous temporal dynamics, we further develop conditional scale normalization, which decouples residual magnitude from distributional shape and allows the flow model to learn normalized residual dynamics more effectively. Crucially, DiverseFlow exploits complementary information from flow-generated forecasts to diversify deterministic predictors, enabling uncertainty modeling to directly improve point forecasting rather than serving only as an auxiliary probabilistic objective. Extensive experiments on ten real-world benchmark datasets show that DiverseFlow consistently improves point-forecasting accuracy over competitive baselines and achieves superior probabilistic forecasting performance over conditional Gaussian modeling across multiple metrics. Ablation studies further verify the effectiveness of forecast diversification, conditional scale normalization, and residual flow matching.

Primary area:
learning on time series and dynamical systems
Keywords:
Forecast Diversity, Predictive Uncertainty, Flow Matching

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Gyg5OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose DiverseFlow, a method for jointly modeling forecast diversity and predictive uncertainty in time-series forecasting by centering multiple plausible horizon forecasts from conditional residual flow matching around a mean horizon.

The mean horizon forecast is obtained by combining two deterministic forecasters, sharing the same backbone but operating on different segments of the time-series history, and the mean of the samples generated from the residual flow process. The weights assigned to each forecast are determined by solving a convex quadratic optimization problem on the validation set.

For horizon generation, DiverseFlow learns residuals relative to the weighted center of the forecast horizons obtained from the two deterministic forecasters (which are pre-trained separately and kept frozen during the flow optimization). To address magnitude variation across datasets and different time periods, which can make optimization harder, residuals are scale-normalized by an element-wise scale parameter s obtained from a conditional scale prediction model that outputs a positive element-wise scale. Scale normalization is applied during training, and the magnitude is restored (by multiplying by s) during sampling.

The authors evaluate the proposed method against deterministic baselines and probabilistic time-series models on eight and six datasets, respectively (with some dataset overlap between the two comparisons). DiverseFlow obtains lower MSE/MAE compared to the reported deterministic baselines and lower CRPS compared to the reported probabilistic baselines in the majority of cases. In addition, the authors quantify the contribution of scale normalization by showing a substantial and consistent degradation in results when removing it. They also show that the ensemble of the flow center and the two deterministic forecasts obtains a lower forecasting error compared to the deterministic ensemble alone.

Critical strengths
  1. The proposed model outperforms pointwise and probabilistic baselines in the majority of cases across contemporary time-series forecasting benchmarks. The contributions of scale normalization and the flow-based center are empirically demonstrated through ablations.
  2. The paper is well-structured: the motivation, method, and discussion are clear and overall easy to follow.
Critical weaknesses
  1. The paper mentions closely related work on probabilistic models for residual generation, such as RDIT, CGFM, and WrapFlow, but these are not included in the empirical comparison. Since these methods appear to be the closest to the proposed work, it is important to benchmark against them or explicitly justify why they are excluded.
  2. Following from Weakness 1, it is not completely clear what constitutes the core novelty compared to previous work: is it the residual flow matching formulation, its combination with deterministic forecasters, or the introduction of scale normalization? While the authors discuss related work, they do not explicitly expain their conceptual and architectural differentiators.
Questions for authors
  1. Can the authors please provide an empirical comparison with baselines employing residual generation (e.g., RDIT, CGFM, WrapFlow), or justify why such comparisons are not feasible (see Weakness 1)?
  2. Could the authors explicitly distill the core technical novelty of their proposed method relative to existing residual-based generative models (see Weakness 2)?
4 comments
AuthorsOpenReview ↗

Part 1/3 - RDIT probabilistic comparisons and a shared-center control

Response to Weakness 1 and the RDIT comparison in Question 1.

Thank you for suggesting comparisons with the closest residual-generation methods. Appendix G of the revision now includes quantitative comparisons with RDIT, CGFM and WrapFlow. Our three-part response covers RDIT, CGFM/WrapFlow, and methodological distinctions with component evidence.

Comparison with published full-RDIT results. Table G.1 in Appendix G.1 reports CRPS at H=96H=96 on eight datasets. RDIT values come from Appendix G, Table 7 of its public paper. DiverseFlow uses the local fixed-center probabilistic configuration and reports three-seed means. Its CRPS on ETTh1 and Solar-Energy is 0.304 and 0.195, respectively, versus published RDIT values of 0.309 and 0.217. RDIT has lower values on the remaining six datasets. The table note specifies data splits, sample counts and integration settings.

Shared-center residual-generation control. To further examine residual generation around point forecasts, we adapt the public RDIT residual network to the same frozen center as DiverseFlow. Both methods share the equal-weight center from two independently trained L=96L=96 PatchTST predictors, with matched H=96H=96, data splits, 64 samples and empirical CRPS. We report three-seed means.

The adaptation combines the shared PatchTST center with ResDiff, training-residual RMS, sample normalization and EAE calibration, using the public reverse-update implementation. Table G.2 reports all three datasets: DiverseFlow and the RDIT adaptation obtain 0.304 and 0.305 on ETTh1, 0.204 and 0.202 on ETTm2, and 0.163 and 0.216 on Weather. The note specifies the adaptation procedure and validation-based checkpoint selection.

These comparisons address two levels of the question. Table G.1 compares DiverseFlow's probabilistic performance with the published full RDIT method. Table G.2 compares residual-generation pipelines with the point-forecast center held fixed. The latter shows a clear CRPS improvement on Weather and provides direct evidence for analyzing the generative component beyond the forecast center. Together, the comparisons establish an empirical connection to this closely related residual-generation method.

AuthorsOpenReview ↗

Part 2/3 - Quantitative comparisons with CGFM and WrapFlow

Further response to Weakness 1 and Question 1.

CGFM: All four auxiliary-predictor variants. Tables G.3 and G.4 in Appendix G.2 compare MSE and MAE at H=96H=96. CGFM values come from Table 1 of its public paper and cover RLinear, iTransformer, TimeDiff and MultiPatchFormer combined with CGFM. DiverseFlow reports three-seed means from the unified configuration. Published CGFM uses L=96L=96 with ten repetitions; DiverseFlow uses two predictors with L=720L=720/L=96L=96.

DiverseFlow has lower MSE and MAE than all four CGFM variants on ETTm1 and ETTm2. For example, ETTm1 MSE/MAE is 0.281/0.339, while the four CGFM variants range from 0.307 to 0.313 in MSE and 0.351 to 0.362 in MAE. DiverseFlow's ETTm2 MSE/MAE is 0.153/0.250. On Traffic, its MSE of 0.372 is also lower than the four CGFM variants' 0.388 to 0.423. The two tables provide complete results on all seven datasets.

WrapFlow: Additional irregular-series tasks. We completed five-seed DiverseFlow adaptation experiments on HumanActivity, USHCN and PhysioNet, using seeds 2024 through 2028. Table G.5 lists their MSE/MAE alongside the means reported in Table 1 of the public WrapFlow paper.

The adaptation interpolates historical observations onto 96 regular grid points. Two independent PatchTST predictors use 96 and 48 points to construct the forecast center. Future targets retain their actual query times, the residual decoder receives future-time information, and losses and metrics are computed at observed targets. Training retains independent scale supervision, residual Flow with the scale frozen, and validation-weighted point ensembling. Appendix G.3 details preprocessing, history lengths and forecast targets.

On HumanActivity, DiverseFlow and WrapFlow both have MSE 0.044 at three-decimal precision, with MAE 0.118 and 0.115, respectively, showing similar MSE performance. Their MSE values are 0.223 and 0.140 on USHCN, and 0.316 and 0.304 on PhysioNet; WrapFlow performs better on these two datasets.

Appendix G thus covers quantitative comparisons with all three related methods: CRPS for RDIT, and the MSE/MAE reported by CGFM and WrapFlow. The CGFM comparison provides a reference for forecast correction on regularly sampled series, while the WrapFlow comparison extends evaluation to irregular series. Captions and notes identify the sources of local experiments, adaptations and published results.

AuthorsOpenReview ↗

Part 3/3 - Core technical distinctions and their validation

Response to Weakness 2 and Question 2.

DiverseFlow's core design independently supervises an input-dependent residual scale, freezes it to learn the normalized residual distribution, and uses that distribution's sample mean in a validation-weighted point ensemble. This explicitly connects scale modeling, probabilistic generation and point-forecast complementarity. Table G.6 in Appendix G.4 compares the concepts and architectures. The relationships to the three methods are as follows.

Relation to RDIT. RDIT models point estimation and residual diffusion separately, combining residual normalization with distribution calibration. DiverseFlow uses a conditional network to produce elementwise scales from the current history and forecast center, fits them with independent reference CRPS, then freezes them before training the residual vector field in fixed normalized coordinates. Its specific design concerns input-dependent scaling, the scale-supervision objective, and the training order of scaling and residual generation.

Relation to CGFM. CGFM uses an auxiliary predictive distribution as its source, combining historical conditioning and affine probability paths to learn forecast corrections. DiverseFlow starts from noise in the residual space defined by the frozen center mm and conditional scale sηs_\eta, reconstructing predictions as Y~=m+sη⊙r~\widetilde Y=m+s_\eta\odot\widetilde r. Its sample mean joins two frozen point predictors in a three-member convex combination with weights fitted on validation data.

Relation to WrapFlow. WrapFlow builds continuous-time residual Flow around base predictions and uses masked-history auxiliary training for irregular observations. DiverseFlow emphasizes independently supervised conditional scaling, a frozen two-predictor center, and a single generated distribution supplying both probabilistic samples and a point-ensemble candidate. The irregular-series adaptation in Part 2 further evaluates this framework on related tasks.

Controlled validation of these designs. Three sets of results in Appendix E address these components:

  1. Predictive value of conditional scaling: Table E.2. With Flow architecture and training settings fixed, compare unit scale, fixed RMS per channel, fixed RMS per step/channel, and conditional scale. Relative to the finer fixed RMS, conditional scaling lowers CRPS on 8/8 datasets and all 24 dataset-seed combinations, with mean improvements of 2.26%2.26\% to 22.24%22.24\%.
  2. Point-forecasting value of generated means: Table E.1. Match conditioning inputs, encoder/decoder, scale, parameter count, initialization and maximum update budget to compare direct MSE residual regression with the Flow sample mean. Flow lowers candidate MSE on 6/8 datasets, and its ensemble lowers MSE on 7/8 under the same validation-weighted procedure.
  3. Both outputs within one system: Tables E.3-E.5. Samples from the same predictors, center, scale and Flow checkpoints provide probabilistic scores, and their mean enters the point ensemble. Seven datasets simultaneously improve point MSE over the two-predictor center and CRPS over fixed-RMS Flow.

These results connect technical distinctions to evidence: fixed-normalization controls test conditional scaling, matched-capacity residual-regression controls test the complementarity of generated means, and unified evaluation demonstrates their combination within one trained system.

I thank the authors for their detailed response. The additional clarifications and experimental results have largely addressed my concerns. While the new analysis confirms that RDIT is a strong baseline, the proposed method offers a distinct design with convincing overall performance. Accordingly, I will maintain my original positive score.

Reviewer_83gEOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DiverseFlow, a framework that aims to jointly improve deterministic and probabilistic time series forecasting by leveraging complementary forecasting information from conditional generative modeling. Specifically, the method employs two independently trained PatchTST predictors with different historical context lengths to provide complementary point forecasts and construct a deterministic forecasting center. A conditional residual Flow Matching model is then introduced to generate probabilistic forecasts around this center.

To facilitate residual distribution learning under heterogeneous temporal dynamics, the authors propose Conditional Scale Normalization, which uses a learnable positive scale to normalize prediction residuals before Flow Matching and restore their magnitudes during sampling. Furthermore, the mean of the Flow-generated samples is incorporated as an additional point-forecasting candidate, which is combined with the two deterministic predictors through validation-optimized convex ensembling.

Critical strengths

S1. A meaningful research motivation that connects probabilistic modeling with point-forecasting improvement.

S2. Promising empirical effectiveness of Conditional Scale Normalization.

The proposed Conditional Scale Normalization provides a practical approach to handling heterogeneous residual magnitudes during Flow Matching. The controlled ablation in Table 3 shows consistent CRPS improvements over the unit-scale variant across all eight evaluated datasets, with particularly substantial gains on ETTh2, Electricity, and Traffic. These results provide meaningful empirical support for the effectiveness of the proposed scale parameterization, even though its statistical interpretation and optimization behavior require further clarification.

S3. Evidence that Flow-generated forecasts can provide complementary information for point prediction.

Critical weaknesses

W1. Potentially inconsistent experimental protocols in probabilistic forecasting comparisons.

Table 2 reports probabilistic forecasting results with a prediction horizon of 96. However, the reported results for all five baseline methods exactly match those in Table 2 of the published D3U paper, where the prediction horizon is 192. Since forecasting horizon substantially affects prediction difficulty, these results do not appear to constitute a comparison under matched experimental settings. Unless this discrepancy is due to an error in the reported configuration, it undermines the validity of the main probabilistic forecasting comparisons.

W2. Insufficient justification of the scale–distribution disentanglement achieved by Conditional Scale Normalization.

Although the ablation results demonstrate that Conditional Scale Normalization improves CRPS, its statistical interpretation and optimization behavior remain insufficiently established. The decomposition R=s⊙rR=s\odot r is not uniquely identifiable without additional constraints, as changes in the positive scale can be absorbed into the normalized residual distribution. The manuscript does not specify an independent scale-supervision objective or clearly describe mechanisms that constrain the learned scales during joint flow-matching training. Consequently, the empirical improvements support the usefulness of the proposed parameterization but do not establish that it achieves a statistically meaningful or stable separation of residual magnitude and distributional shape.

W3. Lack of unified evaluation of point and probabilistic forecasting.

The paper aims to jointly model forecasting diversity and predictive uncertainty. However, its main point-forecasting experiments use a multi-predictor ensemble, whereas probabilistic forecasting is evaluated using a separate fixed-center configuration. As a result, the reported experiments provide evidence for point-forecasting improvements and probabilistic modeling performance under different configurations, rather than directly demonstrating both benefits within the same complete model. This limits the experimental support for the paper's central joint-forecasting claim.

Questions for authors

Q1. Table 2 reports results for a prediction horizon of 96, whereas all baseline scores appear identical to those reported in Table 2 of the published D3U paper, which uses a prediction horizon of 192. Could the authors clarify whether these baseline results were adopted from the D3U paper or independently evaluated? More importantly, were DiverseFlow and all baselines evaluated with the same forecasting horizon and data-processing protocol? If not, how should the reported performance comparisons be interpreted?

Q2. The proposed Conditional Scale Normalization is an important component of the framework, and Table 3 demonstrates its empirical effectiveness. However, since R=s⊙rR=s\odot r admits non-unique scale–distribution decompositions, it is unclear whether the learned (s) necessarily represents meaningful residual magnitude. Could the authors clarify the architecture, positivity constraints, and training mechanisms of the Scale Network? It would be particularly informative to visualize or statistically characterize the learned scales across datasets, variables, and training epochs, potentially comparing them with empirical residual variability. Such analyses could help determine whether scale inflation or other degenerate behaviors occur during joint optimization.

Q3. The main point-forecasting experiments and probabilistic forecasting experiments appear to use different model configurations. Could the authors evaluate point accuracy (MSE/MAE) and probabilistic quality (CRPS/CRPSsum) under the same complete DiverseFlow configuration, at least for the 96-step forecasting horizon? This would provide more direct evidence that the proposed framework can simultaneously deliver the two benefits emphasized in the paper.

Q4. Section 4.1 describes an independent fixed-center probabilistic configuration with a 96-step historical context. Could the authors specify how the deterministic center is constructed in this setting? In particular, does it use a single PatchTST predictor or a combination of predictors, and how does this configuration differ from the two-PatchTST setting used for point forecasting? Clarifying the center construction and the associated model settings would improve the interpretability and reproducibility of the probabilistic experiments.

Q5. Table 4 demonstrates that including the Flow sample mean as an additional ensemble candidate can improve point-forecasting accuracy. However, it remains unclear whether these improvements specifically arise from learning the predictive distribution or mainly from introducing an additional predictor with a different architecture and conditioning information. Could the authors compare the Flow-based candidate against a deterministic residual-correction model trained directly with an MSE objective, ideally using comparable model capacity and conditioning information? Such a comparison would help clarify the unique contribution of probabilistic distribution learning to point forecasting.

5 comments
AuthorsOpenReview ↗

Part 1/4 - Forecast horizons and additional probabilistic comparisons

Response to Weakness 1 and Question 1.

Thank you for identifying the forecast-horizon issue. The five baselines in the original Table 2 came from the D3U paper's H=192H=192 results, while DiverseFlow used H=96H=96. The revision replaces this comparison with completed local H=96H=96 experiments; our comparative probabilistic-forecasting conclusions use these results under a common protocol. We describe the preprocessing, evaluation settings and results below.

Local comparison at H=96H=96. We evaluate DiverseFlow, D3U, TMDM, TimeDiff, CSDI and TimeGrad on ETTm1, ETTm2, Weather, Solar-Energy, Electricity and Traffic. All methods use history length L=96L=96, horizon H=96H=96, 64 generated samples, seed 2 and all test windows. Revised Table 2 and Table F.1 in Appendix F.2 report marginal CRPS; Table F.2 reports aggregate scores.

Preprocessing and evaluation protocol. Methods share the same training, validation and test boundaries on each dataset. Standardization statistics are estimated only from the training interval. Training-window targets stay within that interval; validation and test windows use historical observations preceding their forecast origins. Test origins have stride 1, covering the full test range and retaining the final incomplete batch. All methods' generated samples use the same metric definitions: Table F.1 computes marginal empirical CRPS in the space standardized using training statistics; Table F.2 uses the original-unit aggregate score defined below.

DiverseFlow has the lowest marginal CRPS on all six datasets and the lowest aggregate score on five. For example, marginal CRPS is 0.162 versus D3U's 0.186 on Weather, and 0.253 versus 0.282 on Traffic. CSDI has the lowest aggregate score on Weather. The metrics assess per-variable and channel-aggregate predictive distributions, respectively.

Aggregate-score calculation. Table F.2 uses normalized empirical CRPS in original units. First, restore samples and targets to original units using training-set statistics. Sum channels within each joint sample at each forecast time, compute empirical CRPS for the aggregate samples, accumulate over the full test set, and divide by the sum of absolute aggregate targets. This preserves channel correspondence within each joint sample. Appendix F.1, Eqs. (29)-(30), gives the formulas.

Additional longer-horizon results. Tables F.3/F.4 in Appendix F.3 also list local DiverseFlow results at H=192H=192 on six datasets alongside the same-horizon published values from the D3U paper, with sources identified by column. DiverseFlow's aggregate metric there uses standardized values and 19 quantiles, as defined in Appendix F.1, Eq. (31).

AuthorsOpenReview ↗

Part 2/4 - Conditional-scale training and diagnostics

Response to Weakness 2 and Question 2.

Our scale is an input-dependent parameter independently estimated by reference-distribution CRPS, with the fixed forecast center as the location parameter. Once fitted, it is held fixed while Flow learns the residual distribution normalized by that scale. Scale/shape separation thus has a concrete procedure: an independent scoring objective establishes residual-normalization coordinates, then the generative distribution is trained in those coordinates. Both the local fixed-center probabilistic implementation and new unified experiments follow this order. Section 4.1 corrects the scale-update procedure to match the implementation; Appendix E.1 gives the full objectives, and Appendix E.5 reports scale diagnostics.

Scale supervision and constraints. The network uses 13 history/center features, two width-32 SiLU hidden layers and a learnable base bb per forecast step/channel:

b+\tanh\left(\operatorname{MLP}_\eta(\operatorname{features}(X,m))\right),-6,4\right]\right).$$ Its output satisfies $s_\eta\in[e^{-6},e^4]$. The local fixed-center configuration uses empirical CRPS over 64 Student-t quantile nodes, fitting $\nu\in\{3,5\}$ separately and selecting by validation CRPS. The unified configuration uses analytical reference CRPS. Appendix E.1, Eqs. (23)/(24), gives both objectives. During scale fitting, predictors and center are fixed, and the full scoring objective updates $\eta$. The unified configuration uses AdamW, learning rate $3\times10^{-4}$, weight decay $10^{-3}$, at most 40 epochs and validation early-stopping patience 5. We then freeze the scale, construct $r=(Y-m)\oslash s_\eta$ without gradient tracking, and train the encoder and vector field with Flow Matching plus four-sample Fair-CRPS weighted by 0.03. The Flow optimizer contains only these components' trainable parameters. Normalized targets, interpolation and sample scoring all use the frozen scale. Independent scoring thus determines the scale, and Flow learns residual distributions in those coordinates. **Relation to empirical residual magnitude.** **Table E.7** reports eight datasets and three seeds. Within each channel, we compute the Spearman correlation between scale RMS and empirical residual RMS across windows, then summarize valid channel/seed records. We also report median and maximum scale-RMS/residual-RMS ratios. Channel statistics use up to 256 fixed, uniformly spaced forecast origins per split. Dataset medians of channel correlations range from **0.118 to 0.410**, and median RMS ratios from **0.666 to 1.094**, showing varying degrees of positive association between predicted scales and residual magnitudes across windows. Electricity's maximum RMS ratio, 758.375, corresponds to channel 182 in one seed: residual RMS is approximately $5.71\times10^{-5}$ and scale RMS approximately 0.043. The large ratio reflects the small residual denominator. **Changes during training.** Each run records scale quantiles and boundary saturation at every epoch on the same 32 validation origins. Across eight datasets and three seeds, the selected checkpoints' median scales are below initialization in all 24 runs, ranging from **0.386 to 0.935 times** their respective initial medians. Median scales at every recorded epoch are also no greater than initialization. Across all epochs and runs, the largest ratio of the 99th scale percentile to its initial value is **1.958**. These records characterize changes in the scale distribution's center and upper tail. Table E.7 reports maximum upper/lower saturation rates of **$0\%$** across epochs and seeds. Following independent scale fitting, scale-network parameters stay fixed during Flow training. **Predictive gains.** **Table E.2 in Appendix E.3** compares unit scale, fixed channel RMS, fixed step/channel RMS and conditional scale under the same Flow architecture and training settings. Conditional scaling lowers CRPS relative to the finer fixed RMS on all eight datasets and all 24 dataset-seed combinations, with three-seed mean improvements of $2.26\%$ to $22.24\%$. Independent CRPS defines the scale objective; epoch records characterize optimization; cross-window statistics assess its relation to residual magnitude; and fixed-RMS controls test the predictive benefit of input-dependent scaling.
AuthorsOpenReview ↗

Part 3/4 - Center construction and both outputs from one system

Response to Weakness 3, Question 3 and Question 4.

We specify the local fixed-center probabilistic implementation's center and add point/probabilistic evaluation from the same checkpoints. Table E.8 lists centers and scale settings. The local probability comparison is in Table 2 and Tables F.1/F.2; unified evaluation is in Tables E.3-E.6.

Fixed probabilistic center. The local configuration uses two independently trained L=96L=96 PatchTST implementations (simplified/official), with their equal-weight mean as the residual center, i.e., αm=0.5\alpha_m=0.5. Conditional scale is fitted with Student-t reference CRPS and then frozen. Both predictors read 96-step histories and are trained separately to jointly provide this fixed center.

Difference from the main-text point setting. The main point setting uses two PatchTST predictors with 720- and 96-step histories and adds the Flow sample mean as a third candidate in a validation-weighted ensemble. The fixed-center probabilistic configuration uses the equal-weight center of two 96-step predictors and generates probabilistic samples around it. The settings thus specify distinct history lengths, center construction and final outputs.

New unified system. The new configuration uses frozen L=720L=720 and L=96L=96 predictors with center

m=αmg+(1−αm)f,αm∈[0,1].m=\alpha_m g+(1-\alpha_m)f,\qquad \alpha_m\in[0,1].

The center weight minimizes full-validation MSE. Conditional scale is then independently fitted with analytical reference CRPS, frozen, and followed by Flow training. The residual decoder explicitly receives the center and log scale. Appendix E.1 specifies center selection and training. This configuration uses both long and short histories and the same residual distribution for a point candidate and probabilistic output.

Each test window generates K=64K=64 samples from the same Flow checkpoint. Samples directly provide probabilistic scores; their mean joins the two predictors in a three-member convex ensemble with validation-fitted weights. Both outputs therefore share the predictors, center, scale and Flow checkpoints, and the actual samples generated for each window.

Complete unified results. Tables E.3/E.4 in Appendix E.4 report MSE/MAE and CRPS/CRPSsum\mathrm{CRPS}_{\mathrm{sum}} on eight datasets with three seeds. The aggregate metric uses standardized values: sum channels within joint samples, compute the 19-quantile score at each time, and normalize by the sum of absolute aggregate targets over the full test set. Appendix F.1, Eq. (31), defines it.

For example, this configuration's MSE, MAE, CRPS and CRPSsum\mathrm{CRPS}_{\mathrm{sum}} on Electricity are 0.134, 0.233, 0.175 and 0.112, respectively. MSE/MAE assess the three-member point ensemble; the probabilistic metrics assess the same batch of joint generated samples.

Table E.5 directly tests whether both gains coexist. Point MSE improves over the two-predictor center on seven datasets; CRPS improves over fixed-RMS Flow on eight. Thus seven datasets improve both outputs within one system. On Electricity, MSE falls from 0.141 to 0.134 and CRPS from 0.184 to 0.175, improvements of 4.43%4.43\% and 4.82%4.82\%.

Table E.6 reports this configuration's inference cost: on an RTX 3090, with batch size 4, 64 samples and four Euler steps, median model latency across five datasets ranges from 13.870 to 67.490 ms per batch. Tables F.5/F.6 in Appendix F.4 compare its point output with existing forecasting methods in MSE/MAE.

Reviewer_YR3DOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes DiverseFlow for point and probabilistic time-series forecasting. It combines two independently trained PatchTST predictors using different history lengths with a conditional flow-matching model that learns residual distributions around a fixed forecast center. A learned scale network normalizes residuals during training and restores their magnitudes during sampling. The generated sample mean is combined with the two deterministic forecasts using validation-fitted weights. Experiments evaluate point forecasting on eight datasets across four horizons and probabilistic forecasting on six datasets at horizon 96. Ablation studies assess the contributions of conditional scaling and the flow sample mean.

Critical strengths
  • The experiments provide useful evidence for conditional scale normalization. Table 3 reports lower mean CRPS on all eight evaluated datasets over three seeds, while the main probabilistic comparison achieves the best scores on 10 of 12 dataset–metric pairs. These results support the practical value of the proposed residual modeling approach.

  • The framework can extend pretrained point predictors while keeping their parameters frozen. Table 4 compares two- and three-member ensembles using the same deterministic predictors, providing a controlled demonstration that the flow sample mean can contribute useful corrections to point forecasts.

Critical weaknesses
  • The methodological contribution appears incremental, and its specific benefit is insufficiently established. The framework combines existing forecasting backbones, residual flow matching, and validation-fitted convex ensembling. Its main distinctions are conditional scaling and the inclusion of the generated sample mean as an ensemble candidate. However, comparison against a deterministic residual regressor with comparable conditioning and capacity is missing, leaving unclear whether distribution learning provides a specific advantage for point forecasting. The scaling ablation also compares only against unit scale, without establishing the benefit of conditional scaling over simpler fixed normalization. Furthermore, the main point and probabilistic results use different configurations, limiting evidence that both reported benefits coexist in one trained system.

  • The training specification for the conditional scale network is incomplete. The scale determines both the normalized residual target and the interpolation used in the flow objective, yet the paper does not clearly specify its complete training objective, gradient propagation through these quantities, or any scale supervision or constraints. Since learning the scale changes the regression problem itself, these details are necessary to assess the optimization procedure and interpret the claimed separation of residual magnitude from distributional shape. This omission affects a central contribution and limits reproducibility.

Questions for authors
  1. What is the complete objective and update procedure for the scale network? Do gradients propagate through the normalized residual target and interpolation in Equation (9)? Please specify any supervision or constraints and explain how the residual-center coefficient αm\alpha_m is selected.

  2. Under matched inputs and comparable capacity, how does the flow candidate compare with a deterministic residual regressor? Likewise, how does conditional scaling compare with fixed residual normalization estimated from training data? These comparisons would help establish the value of the proposed components beyond simpler alternatives.

  3. Can you provide point and probabilistic forecasting results from the same trained configuration at horizon 96, together with its inference cost? This would clarify whether the reported benefits coexist in a single operational setting.

10 comments
AuthorsOpenReview ↗

Part 1/4 - Component contributions and deterministic residual regression

Response to Weakness 1 and the deterministic residual-regression comparison in Question 2.

Thank you for suggesting simpler alternatives to test the components' value. We added controls on eight datasets with three seeds at H=96H=96, reported in Appendix E. Our four parts cover the point-forecasting value of distribution-based candidates, scale training, fixed-normalization controls, and both outputs and costs within one system.

Our specific design. DiverseFlow independently learns an input-dependent residual scale around a center constructed from frozen point predictors, then freezes the scale to learn the normalized residual distribution. Samples provide probabilistic forecasts, while their mean enters a validation-weighted point ensemble. This yields two separately testable components: residual-scale modeling and the predictive complementarity of generated means.

Testing the distribution-based candidate. We construct a deterministic residual head matching Flow in encoder, decoder, conditioning information, conditional scale, trainable parameter count, initialization and maximum update budget. Both receive 96-step histories, past and future temporal features, historical prompts and metadata, and pass the forecast center and log scale to the residual decoder. The deterministic head sets the residual state and Flow time to zero, retains these conditions, and learns a residual correction with prediction MSE:

Y^det=m+sη⊙hdet(X,m).\widehat Y_{\mathrm{det}}=m+s_\eta\odot h_{\mathrm{det}}(X,m).

Flow learns the normalized residual distribution, generates K=64K=64 samples and uses their mean as its candidate:

\overline Y_{\mathrm{Flow}}=\frac{1}{K}\sum_{k=1}^{K}\widetilde Y^{(k)}.$$ **Capacity and training budget.** The table lists trainable parameters in each residual head, including its conditional encoder and vector field. Frozen predictors and the prefitted scale network are maintained separately. | Dataset | Deterministic head | Flow head | | --- | ---: | ---: | | ETTh1, ETTh2, ETTm1, ETTm2 | 49,742 | 49,742 | | Electricity | 140,802 | 140,802 | | Traffic | 297,692 | 297,692 | | Weather | 53,802 | 53,802 | | Exchange | 50,032 | 50,032 | Both use AdamW, learning rate $10^{-4}$, weight decay $10^{-3}$, gradient-norm clipping at 1, and effective batch size 32. Each has at most 20 epochs and 128 updates per epoch (2,560 updates total), with validation early-stopping patience 5. Seeds are 2, 2021 and 2023; paired heads share initialization and training-window order. Checkpoints are selected each epoch on the same 192 uniformly spaced validation origins: MSE for the deterministic head and 16-sample Fair-CRPS for Flow. Final ensemble weights use the full validation set, with 64 Flow samples; evaluation covers all test windows. **Results and interpretation.** **Table E.1 in Appendix E.2** reports both candidate and ensemble MSE. Flow lowers candidate MSE on 6/8 datasets and ensemble MSE on 7/8. Ensemble improvements on Exchange, ETTm2 and Electricity are $3.87\%$, $3.66\%$ and $2.60\%$. Traffic illustrates complementarity: deterministic and Flow candidates have standalone MSE 0.375 and 0.379, while their ensembles obtain 0.373 and 0.372. The generated mean's value thus concerns both candidate accuracy and its combination with existing predictors. With matched conditioning, capacity and maximum update budgets, a candidate trained with Flow Matching plus Fair-CRPS and selected by validation Fair-CRPS can provide additional point-forecasting gains. Part 3 tests the independent contribution of conditional scaling against fixed normalization.
AuthorsOpenReview ↗

Part 2/4 - Scale objectives, gradient paths and center coefficient

Response to Weakness 2 and Question 1.

Both the local fixed-center probabilistic implementation and new unified experiments use three stages: fit point predictors, independently fit scales, then freeze scales and train Flow. Section 4.1 corrects the scale-update order to match the implementation; Appendix E.1 gives the objectives and gradient paths.

Configuration and resultsResidual centerIndependent scale supervision
Local fixed-center probability: Table 2, Tables F.1/F.2Equal-weight mean of two independently trained L=96L=96 PatchTST models; αm=0.5\alpha_m=0.5Student-t reference CRPS
Unified system and controls: Tables E.1-E.6Frozen L=720L=720/L=96L=96 PatchTST models; αm\alpha_m fitted by validation MSEAnalytical reference CRPS for the unified system and deterministic control; Table E.2 also tests unit and fixed-RMS scales

The unified system combines long/short histories, a validation-weighted center and conditional scaling, explicitly conditioning the residual decoder on the center and log scale. Its training is as follows.

1. Fix the center. Let g,fg,f denote frozen L=720,L=96L=720,L=96 predictions. Set m=αmg+(1−αm)fm=\alpha_m g+(1-\alpha_m)f. For each dataset and seed, minimize full-validation MSE:

\frac{\sum_{\mathrm{val}}(g-f)(Y-f)}{\sum_{\mathrm{val}}(g-f)^2},0,1\right).$$ Sums cover validation windows, forecast steps and channels. If the denominator is zero, set $\alpha_m=0.5$. Fix this coefficient during residual training; fit final three-member weights separately on validation data. See Eq. (21) and **Table E.8**. **2. Independently supervise scale.** The network uses 13 history/center features, two width-32 SiLU hidden layers and a learnable base $b$ per forecast step/channel: $$s_\eta(X,m)=\exp\left(\operatorname{clamp}\left[ b+\tanh\left(\operatorname{MLP}_\eta(\operatorname{features}(X,m))\right),-6,4\right]\right).$$ The 13 features are: historical and recent standard deviations; mean absolute adjacent difference; last-value distance from the historical mean; center-to-last distance; center magnitude; relative forecast step; seasonal sine and cosine; log ratio of recent to full-history standard deviations; seasonal-lag difference; change between adjacent center steps; and center distance from a seasonally repeated forecast. Magnitude/distance features use $\log(1+x)$. The period is the dataset's primary period; the seasonal lag is capped at half the history length. In the unified system, $b$ is initialized with log training-residual RMS per step/channel. Exponentiation and clipping give $s_\eta\in[e^{-6},e^4]$. The fixed-center probabilistic implementation supervises scale with empirical CRPS over 64 Student-t quantile nodes, fitting $\nu\in\{3,5\}$ separately and selecting by validation CRPS (Eq. (23)). The unified system uses analytical reference CRPS (Eq. (24)): $$\mathcal{L}_s = \operatorname{mean}_{\mathrm{train}} \left\\{ s_\eta \left[ \xi \left(2\Phi(\xi)-1\right) + 2\varphi(\xi) - \frac{1}{\sqrt{\pi}} \right] \right\\}, \qquad \xi=\frac{Y-m}{s_\eta}.$$ Here $\Phi,\varphi$ are the standard normal CDF and density. Gradients through both $\xi$ and the outer $s_\eta$ update $\eta$. Scale fitting uses AdamW, learning rate $3\times10^{-4}$, weight decay $10^{-3}$, gradient-norm clipping at 1, at most 40 epochs, and validation-CRPS checkpoint selection with patience 5. **3. Freeze scale and train the residual distribution.** The unified experiments set the main text's noise parameter $\tau=1$. With frozen scale, let $r=(Y-m)\oslash s_\eta$, $r_u=(1-u)z+ur$, $z\sim\mathcal N(0,I)$ and $u\sim\mathcal U(0,1)$. The full objective is $$\mathcal L_{\mathrm{head}}=\mathcal L_{\mathrm{FM}}+0.03\mathcal L_{\mathrm{FairCRPS}}.$$ Eq. (9) is the Flow Matching term, regressing target velocity $r-z$. Both the local fixed-center implementation and unified experiments include the Fair-CRPS term with weight 0.03 and four independent generated samples. Appendix E.1, Eqs. (25)-(27), expands the objective. **Gradients in Eq. (9): scales are computed without gradient tracking; the Flow optimizer contains only trainable conditional-encoder and residual-vector-field parameters.** The scale used in normalized targets, interpolation and sample scoring stays fixed while gradients update the encoder and vector field. Independent scale scoring establishes the coordinates in which Flow learns the residual distribution.
AuthorsOpenReview ↗

Part 3/4 - Conditional scale versus fixed normalization

Response to Weakness 1 and the fixed residual-normalization comparison in Question 2.

We expanded the scale ablation to four settings, directly comparing against fixed scales estimated from training residuals. Table E.2 uses the new unified configuration: frozen L=720L=720/L=96L=96 predictors and their validation-weighted center. Flow architecture, initialization, objective and the maximum update budget in Part 1 are fixed. We compare four scales on eight datasets, three seeds and H=96H=96:

  1. Unit scale: s=1s=1; Flow directly models residuals around the center.
  2. Fixed RMS per channel: pool squared residuals over training windows and forecast steps to estimate one scale per channel.
  3. Fixed RMS per forecast step/channel: estimate a separate scale for each step/channel, retaining magnitude differences across the horizon.
  4. Conditional scale: sη(X,m)s_\eta(X,m) produces elementwise scales from the current history and forecast center.

The fixed scale per step/channel is

\sqrt{\operatorname{mean}_{\mathrm{train}}[(Y[h,c]-m[h,c])^2]}\right).$$ Both fixed RMS scales use training residuals and remain constant across forecast windows. Conditional scaling learns the relationship between inputs and residual magnitude through independent CRPS supervision, then is frozen for Flow training. All four settings share residual-normalization, decoder-conditioning and sample-reconstruction interfaces; the decoder's scale condition takes the value specified by each variant. Main-text Table 3 provides the original unit-scale ablation; Table E.2 retrains all four variants in the unified configuration to directly test conditional scaling against fixed normalization. **Complete results are in Appendix E.3, Table E.2.** Relative to the finer step/channel RMS, conditional scaling lowers three-seed mean CRPS on 8/8 datasets by $2.26\%$ to $22.24\%$, improving all 24 dataset-seed combinations. Mean CRPS is also lower than channel RMS on all eight datasets. For example, CRPS falls from fixed RMS's 0.184 to conditional scaling's 0.175 on Electricity, and from 0.246 to 0.211 on Traffic. Both fixed controls already provide training-residual magnitude information. Conditional scaling further uses each window's history and center and consistently improves performance, supporting the added predictive value of input-dependent scale learning and its independent scoring supervision. This and Part 1 address separate components: **Table E.2 tests how to set residual scales; Table E.1 tests how to construct point-ensemble candidates.** Together they support conditional scaling and the generated mean.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.