acceptodds
Under review as a conference paper at ICLR 2027

MercerFlow: Flow Matching in a Kernel-Induced Latent Space for Probabilistic Forecasting

Abstract

Probabilistic flow matching for time series forecasting usually absorbs local correlation with a sequential architecture—an RNN, S4, or a Transformer—rather than a global MLP that maps an entire history to an entire horizon. That sequential backbone costs GPU memory and time per epoch. A cheaper alternative is latent flow matching: embed the time series via an invertible map to a single latent vector and learn the flow there, so a tabular MLP can treat the series as one coordinate vector. However, choosing that space remains challenging: a Fourier basis of the forecast window does not diagonalise a Gaussian-process prior on that window, a truncated signature is difficult to invert, and a learned encoder adds its own training cost. We introduce MercerFlow, which learns conditional flow matching in the orthonormal eigenbasis of a Gaussian-process kernel matrix: the latent covariance is diagonal under that kernel and the map is linearly invertible. Across six datasets, mean CRPS is lower than for sequential-architecture flow matching. On the long-window sets the MLP uses 20–60 less training memory and 3–7 less time per epoch (up to 19 on Solar).

Primary area:
generative models
Keywords:
time series forecasting, probabilistic forecasting, flow matching, Gaussian processes, latent representations, kernel methods

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_HkjiOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

MercerFlow performs conditional flow matching in the eigenbasis of a Gaussian-process prior, using a lightweight MLP for probabilistic forecasting. The paper investigates how orthogonal coordinates interact with adaptive optimization and reports competitive accuracy with lower computational costs than TSFlow. The approach is practical, but its methodological novelty and advantages over simpler lightweight alternatives appear limited.

Critical strengths

1 The prior-aligned basis provides a simple, interpretable way to represent temporal structure without information loss.

2 Shared-MLP codec comparisons and optimizer ablations help isolate representation effects.

3 The reported accuracy–efficiency tradeoff is practically relevant for probabilistic forecasting.

Critical weaknesses

1 The method primarily combines a classical kernel eigendecomposition with standard linear flow matching. It changes the optimization coordinates rather than the underlying transport construction.

2 The main computational comparison replaces a 186k-parameter S4 with a 55k-parameter MLP. Since FM already supports lightweight networks, this does not establish how much efficiency the proposed basis itself contributes.

Questions for authors
  1. How large are Mercer’s absolute accuracy gains over Raw, DCT, and Fourier under individually tuned optimization and matched computational budgets?

  2. How reliably does prior-based Jacobi conditioning explain actual AdamW behavior on conditional interpolants?

  3. Do the conclusions remain convincing when reporting effect sizes, confidence intervals, and rankings under a more conventional significance threshold?

4 comments

Part 1/4: Positioning and synthetic rank comparisons

We thank you for asking us to clarify the contribution of the representation and separate it from the efficiency of the neural model. Your comments have helped us refine the paper's positioning. TSFlow focuses on matching the prior to the temporal structure of the data. We ask whether the prior is a good probabilistic forecaster in its own right. When it is, how much can a very small neural network gain from the predictive information it already contains? We build on TSFlow's use of Gaussian process priors in flow matching to learn a small correction to that forecast. The goal is to make the most of that starting forecast with minimal training cost. For real data, our empirical Gaussian prior follows Lin et al., Empirical Gaussian Processes (ICML 2026; https://arxiv.org/abs/2602.12082).

Our results show that a small learned correction improves the statistical forecast and reaches accuracy competitive with larger task-trained forecasters, at substantially lower training cost in the profiled implementations.

The forecasting comparisons measure how much the correction adds to the starting forecast and how much training it requires. Matched controls separate this benefit from the effect of the coordinates. The original representation studies focused on optimization and stability. New truncation experiments explore whether the prior can also guide a smaller correction.

W1: Contribution beyond a coordinate rotation

We added a synthetic variant that learns a correction in fewer directions. The starting GP forecast y0∼N(μ,Σ)y_0\sim\mathcal{N}(\mu,\Sigma) stays at full resolution. The MLP operates in the top kk eigenvectors Φk∈RLf×k\Phi_k\in\mathbb{R}^{L_f\times k} of the conditional future covariance Σ\boldsymbol{\Sigma}:

y^f=y0+Φk(c^1−c0).\hat{y}_f = y_0 + \Phi_k (\hat{c}_1 - c_0).

At full rank (k=Lfk=L_f), changing the basis preserves the possible corrections. At reduced rank (k≪Lfk\ll L_f), the basis selects which directions the model can correct. This restriction can also regularize learning.

Here μ\mu and Σ\Sigma are the conditional forecast's mean and covariance. The horizon length is LfL_f, and kk is the correction rank. The starting coordinates are c0=Φk⊤y0c_0=\Phi_k^\top y_0, and c^1\hat c_1 is their learned endpoint. The network is a multilayer perceptron (MLP).

We compare the prior's eigenbasis (Mercer) with principal component analysis (PCA), the discrete cosine transform (DCT), a Fourier basis, and time coordinates (Raw). PCA estimates its directions from data. DCT and Fourier use fixed directions.

We report the continuous ranked probability score (CRPS), which measures probabilistic forecast error. Lower is better. Percentages show how much of the prior-to-oracle gap the correction closes: (prior CRPS − model CRPS) / (prior CRPS − oracle CRPS). The oracle score is the synthetic benchmark's reference.

Table 1: Effect of truncation rank kk on non-stationary modulated periodic data (Lf=32L_f=32, GP Prior CRPS = 0.2565, Oracle = 0.0200).

Basisk=4k=4 CRPS ↓\downarrowk=6k=6 CRPS ↓\downarrowk=10k=10 CRPS ↓\downarrow (Gap ↑\uparrow)k=32k=32 (Full) CRPS ↓\downarrow
Mercer (Ours)0.19260.16190.0768 (76.0%)0.0531
PCA0.19590.16600.0884 (71.1%)0.0547
DCT0.23850.23150.2140 (18.0%)0.0528
Fourier0.24210.23040.2167 (16.8%)0.0543
Raw0.24110.23310.2180 (16.3%)0.0546

At full rank, forecast errors are close across bases. With ten correction directions, Mercer reaches CRPS 0.0768, compared with 0.2140–0.2180 for DCT, Fourier, and Raw. The error is nearly three times lower. Here the basis determines which corrections the smaller model can learn.

Part 2/4: Basis estimation, optimizer analysis, and reporting

W1: Basis estimation and regularization

We added two controlled studies of this choice. First, we fix the correction's training data (512 samples, k=8k=8) and vary only the data used to estimate PCA's basis. Mercer gives lower CRPS when PCA's calibration sample is scarce or comes from a shifted historical regime.

Second, we compare rank-6 and full-rank corrections with only N=64N=64 training samples. For the tested radial basis function (RBF) and Matérn-5/2 kernels, six directions capture more than 99.6% of prior variance. The smaller correction gives lower CRPS. This shows a regularization benefit in this small-sample setting.

Table 2: Absolute CRPS effects in the controlled studies (negative differences favor the first method).

SettingComparisonCRPS difference
Periodic, rank 10Mercer − Fourier−0.1399
Periodic, rank 10Mercer − PCA−0.0116
Scarce calibrationMercer − PCA−0.0137
Shifted calibrationMercer − PCA−0.0359
RBF, N=64Rank 6 − full rank−0.0313

The prior can help select useful correction directions at low rank. It can also reduce sensitivity to scarce or shifted PCA calibration data. Once the kernel is chosen, its basis requires no separate calibration sample. Covariance construction and eigendecomposition still incur computation.

Q2: Optimizer analysis and conditional interpolants

We have corrected the conditioning analysis. It examines how the prior's coordinates affect the geometry of learning. For fixed covariance KK and orthogonal basis Φ\Phi, write A=Φ⊤KΦA=\Phi^\top K\Phi and S=diag⁡(A)S=\operatorname{diag}(A). The corrected frozen input-curvature surrogate is S−1/4AS−1/4S^{-1/4}AS^{-1/4}. In the eigenbasis, this matrix is diagonal. Its eigenvalues are the square roots of those of AA, so its condition number is κ(A)\sqrt{\kappa(A)}.

Measurements on the source–target interpolation test this geometry in the synthetic training distributions. Actual Adam and AdamW also depend on residuals, momentum, and evolving second moments. The surrogate explains a mechanism, while forecast scores measure accuracy. The real-data experiments use Adam.

Q3: Effect sizes and statistical reporting

We will replace threshold-based rankings with absolute scores and effect sizes, as in the tables above. The real-data results use one training seed per task and describe the observed benchmark effects. The controlled studies identify settings with large representation effects. The full-rank real-data controls show a smaller effect.

Part 3/4: Real-data benchmark and efficiency attribution

W2 / Q1: Efficiency attribution and real-data comparisons

The 4.7×4.7\times memory and 4×4\times runtime gains over TSFlow come from replacing S4 with an MLP. We will attribute them to that architecture change. To isolate the coordinates' effect, we now compare the same MLP and training recipe in time coordinates and in the prior's eigenbasis. TiDE provides a direct-forecast MLP reference.

A smaller MLP alone leaves the role of the representation unresolved. Your questions motivated matched controls and a broader evaluation across all 54 GIFT-Eval tasks whose datasets contain single-channel series. We use official splits, forecast origins, and scoring. The controls identify where improvements come from. The external comparisons assess accuracy and training cost.

Prior and representation controls

GP-only is the fitted statistical forecast. Raw-GP learns a correction in time coordinates; MercerFlow learns it in the prior's eigenbasis. Raw-noise starts from independent Gaussian noise. All neural models share the correction network and training recipe, isolating the effects of the starting forecast and coordinates.

The matched controls below use the same 54 tasks and equal-weight geometric mean wQL.

MethodGeom. mean wQL ↓
GP-only (statistical forecast)0.163321
Raw-noise (correction from independent noise)0.156343
Raw-GP (prior + correction in time coordinates)0.141639
MercerFlow (prior + correction in its eigenbasis)0.140626
MercerFlow, half input/output ranks (same width)0.142803

Halving both the input and correction output ranks while keeping the same network width increases aggregate wQL by only 1.55%, preserving most of the full-rank model's accuracy.

The main gain comes from learning a correction to a useful forecast. The correction improves on the GP alone, and starting from that forecast also improves on starting from independent noise. Changing to the prior's eigenbasis gives a smaller additional gain. The synthetic studies examine settings where the choice of correction directions matters more.

Broader forecasting comparison

DLinear, TiDE (a direct-forecast MLP), PatchTST, and TFT are trained separately on each dataset. Chronos-2, TiRex, and TimesFM-2.5 are pretrained probabilistic models evaluated zero-shot. Our model learns a correction to an empirical Gaussian forecast.

We report weighted quantile loss (wQL) for probabilistic forecasts and mean absolute scaled error (MASE) for forecast medians. Training cost is measured in millions of floating-point operations (MFLOPs) per sample. Lower is better for all three.

MethodGeom. mean wQL ↓Geom. mean MASE ↓Geom. mean training MFLOPs/sample ↓
MercerFlow (empirical eigenbasis + GP)0.14061.10252.763
TSFlow (locally reproduced)0.15781.1926663.046 (estimated)
DLinear0.20541.34661.456
TiDE0.17511.36770.341
PatchTST0.14491.148811.653
TFT0.14201.152171.051
Chronos-2 (zero-shot)0.11890.9829--
TiRex (zero-shot)0.12050.9936--
TimesFM-2.5 (zero-shot)0.12070.9895--

All columns are equal-weight geometric means over the same 54 tasks. Our results use one training seed, with context lengths and checkpoints selected on validation data. Our local TSFlow reproduction uses one seed and its original training recipe, with early validation-based selection among three kernels followed by 400 total training epochs. Other external accuracy scores come from the published GIFT-Eval results.

Our correction achieves 10.86% lower aggregate wQL and 7.55% lower aggregate MASE than this TSFlow reproduction on the complete shared panel.

Training costs cover forward and backward passes, including our prior conditioning, sampling, and coordinate transforms, at batch size one. TSFlow's estimate includes GP operations and estimated S4 arithmetic; shared costs amortize over larger batches. Baseline costs use reference architectures and context lengths. These profiles exclude optimizer updates, validation, and one-time preparation. The zero-shot models require no task-specific training; their pretraining cost is separate.

A strong statistical forecast lets a small learned correction reach accuracy competitive with PatchTST and TFT at substantially lower profiled training cost. The aggregate accuracy gains are modest, and performance varies across tasks. The component controls give the clearest evidence for the approach: the correction improves the starting forecast itself. The pretrained models achieve stronger accuracy and require no downstream training.

Part 4/4: Scope of the optimization comparison

These comparisons use Adam with learning rate 0.001 and a matched training budget. They measure performance under a shared training recipe. The synthetic rank comparisons examine the larger representation effects that appear when the correction is restricted to fewer directions.

Reviewer_CHREOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MercerFlow, which leverages the Mercer eigenbasis of the prior kernel, for probabilistic forecasting of time series data. The key idea is to first map the joint context and horizon window into the decorrelated Mercer eigenbasis of the GP prior, then use a lightweight MLP to learn conditional flow matching directly in this latent space. A detailed study is provided to compare the use of different codecs for projection. Empirical results are provided to evaluate MercerFlow, showing that it can improve predictive accuracy and reduce computational cost when compared with TSFlow and other baselines.

Critical strengths
  • Overall the paper is well written and is easy to follow.
  • The detailed, controlled study on the use of different codecs for orthonormal projection and their interaction with the optimizer (SGD vs Adam) used during training is insightful.
Critical weaknesses
  • Empirical evaluations are rather limited, focusing on the five standard time series datasets. Evaluations on spatio-temporal datasets such as videos and PDEs are missing. I believe that evaluations on PDE datasets (rather than only on real-wold time series datasets) whose temporal structure is well understood can provide further insights into the advantages of the proposed method.
  • There are no empirical comparisons to other probabilistic forecasters such as LS4 [1] and Probabilistic Transformer [2].
  • There are no comparison/ablation studies for other choices of kernel such as the Matern kernel, signature kernel [5], and other sequential kernels.
  • It is not clear how the performance of the proposed method compares to latent flow matching, where an autoencoder is first pretrained and then flow matching is performed on the latent space (see, e.g., the approach in [3], [4]), to see how the use of Mercer basis compares to learned basis/latent representations.
  • Although a brief discussion on computational efficiency is provided, it is not clear how the computational cost scales with dimension and sequence length.
  • Table 1 is not informative enough as the main result table. Quantitative metrics such as CRPS would be better to clearly demonstrate the differences between the compared baselines.

[1] Zhou, Linqi, et al. "Deep latent state space models for time-series generation." International Conference on Machine Learning. PMLR, 2023.

[2] Tang, Binh, and David S. Matteson. "Probabilistic transformer for time series analysis." Advances in neural information processing systems 34 (2021): 23592-23608.

[3] Lim, S. H., Wang, Y., Yu, A., Hart, E., Mahoney, M. W., Li, X. S., & Erichson, N. B. (2024). Elucidating the design choice of probability paths in flow matching for forecasting. arXiv preprint arXiv:2410.03229.

[4] Carvajal Reyes, C., & Tobar, F. (2026). Time series generation with spectrally aligned latent flow matching. arXiv e-prints, arXiv-2609.

[5] Király, F. J., & Oberhauser, H. (2019). Kernels for sequentially ordered data. Journal of Machine Learning Research, 20(31), 1-45.

Questions for authors
  • The paper focuses on the exponential (OU) kernel as the primary kernel used for implementation. How will this work for non-Gaussian datasets? Will other kernels such as the Matern kernel and signature kernel help improving the empirical results?
  • The channels are treated independently in the method, which is fine for simplifying the method. How would the method fare in situations where we have multivariate time series data that are highly correlated (across the channels)?
  • How would the performance of the method depends on the smoothness of the dataset?
  • How would the computational cost scales with dimension and sequence length?
4 comments

Part 1/4: Positioning and non-Gaussian experiments

We thank you for asking for broader evidence, probabilistic comparisons, and a clearer account of kernel and representation choices. Your comments have helped us refine the paper's positioning. TSFlow focuses on matching the prior to the temporal structure of the data. We ask whether the prior is a good probabilistic forecaster in its own right. When it is, how much can a very small neural network gain from the predictive information it already contains? We build on TSFlow's use of Gaussian process priors in flow matching to learn a small correction to that forecast. The goal is to make the most of that starting forecast with minimal training cost. Our empirical Gaussian prior follows Lin et al., Empirical Gaussian Processes (ICML 2026; https://arxiv.org/abs/2602.12082).

Our results show that a small learned correction improves the statistical forecast and reaches accuracy competitive with larger task-trained forecasters, at substantially lower training cost in the profiled implementations.

The controlled examples test whether the correction can improve a Gaussian forecast when the data are non-Gaussian or nonstationary. The kernel studies examine which correction directions are useful. New truncation experiments extend the original focus on optimization and stability to smaller corrections. The real-data benchmark and matched controls measure forecast quality and training cost.

W1 / Q1: Non-Gaussian dynamics and evaluation scope

We added two controlled examples with strongly non-Gaussian dynamics. The first is a bimodal Double-Well Langevin stochastic differential equation (SDE). Trajectories move toward either of two wells at ±1\pm1, while the Gaussian prior predicts their unstable mean at zero: dyt=3(yt−yt3)dt+σdWtdy_t=3(y_t-y_t^3)dt+\sigma dW_t. The second is a nonstationary periodic system with a quadratic distortion: yt=xt+a(t)(xt2−Ktt)+ϵty_t=x_t+a(t)(x_t^2-K_{tt})+\epsilon_t. These examples test whether the correction can recover forecast behavior that the Gaussian starting point misses.

Mercer uses the conditional prior's covariance eigenvectors. Principal component analysis (PCA) estimates directions from data. The discrete cosine transform (DCT) and Fourier transform use fixed directions. Raw uses time coordinates. The GP draw stays at full resolution. Only the learned correction is restricted to kk directions over a horizon of length LfL_f.

In the equations, WtW_t is Brownian motion and σ\sigma controls its strength. The latent process xtx_t is Gaussian with covariance KK. The function a(t)a(t) controls the quadratic distortion, and ϵt\epsilon_t is observation noise.

We report the continuous ranked probability score (CRPS), which measures probabilistic forecast error. Lower is better. Percentages show how much of the prior-to-oracle gap the correction closes: (prior CRPS − model CRPS) / (prior CRPS − oracle CRPS). The oracle score is the synthetic benchmark's reference.

Table 1: Performance on non-Gaussian synthetic dynamical systems (Lf=32L_f=32; CRPS ↓\downarrow and % of Prior-to-Oracle Gap Closed ↑\uparrow).

System / SettingGP Prior CRPSOracle CRPSMercer (Ours)PCADCTFourierRaw
Bimodal Double-Well SDE (k=12k=12, high learning rate)0.36470.23490.2489 (89.2%)0.2493 (88.9%)0.2496 (88.7%)0.2511 (87.5%)0.3647 (0.0%)
Modulated Nonlinear Periodic (k=10k=10)0.25650.02000.0768 (76.0%)0.0884 (71.1%)0.2140 (18.0%)0.2167 (16.8%)0.2180 (16.3%)

On the bimodal Double-Well SDE, Mercer, PCA, DCT, and Fourier achieve similar CRPS, with Mercer obtaining the lowest numerical value (0.2489). All four representations substantially improve the GP prior, closing approximately 88–89% of the prior-to-oracle gap. Raw shows no improvement under the high learning rate used in this experiment.

In contrast, the nonlinear periodic example shows much larger differences between representations. At k=10, Mercer achieves a CRPS of 0.0768, compared with 0.2140 for DCT and 0.2167 for Fourier.

These results suggest that the benefit of the Mercer basis depends on the structure of the dynamics and the selected correction subspace.

Part 2/4: Kernel choice and learned representations

W3 / Q1 / Q3: Kernel choice and smoothness

We added radial basis function (RBF) and Matérn-5/2 kernels to test smoother priors. We vary training sample size (N∈{64,512}N\in\{64,512\}) and correction rank (k=6k=6 or full rank k=32k=32). We also test whitening, which divides each coordinate by the square root of its covariance eigenvalue. This amplifies directions with small prior variance.

For these smooth kernels, six directions capture almost all conditional future variance: 99.97% for RBF and 99.67% for Matérn-5/2. The remaining 26 directions account for only a small fraction of the total prior variance.

Table 2: Smooth kernel ablation (RBF and Matérn-5/2, Lf=32L_f=32; CRPS ↓\downarrow and % Oracle Gap Closed ↑\uparrow).

Kernel & Sample SizeTop-6 Σ\Sigma EnergyMercer (k=6k=6)PCA (k=6k=6)DCT (k=6k=6)Raw (k=6k=6)Mercer (k=32k=32 Full)Mercer (k=32k=32 Whitened)
RBF (N=64N=64)99.97%0.3932 (+27.4%)0.4078 (+24.3%)0.4167 (+22.4%)0.5173 (+0.9%)0.4245 (+20.7%)0.4852 (+7.7%)
RBF (N=512N=512)99.97%0.3457 (+37.6%)0.3494 (+36.8%)0.3477 (+37.2%)0.5095 (+2.5%)0.3495 (+36.8%)0.3618 (+34.1%)
Matérn-5/2 (N=64N=64)99.67%0.3726 (+2.3%)0.3750 (+1.5%)0.3821 (-0.6%)0.3817 (-0.5%)0.3856 (-1.7%)0.4291 (-15.1%)
Matérn-5/2 (N=512N=512)99.67%0.3303 (+15.2%)0.3321 (+14.7%)0.3321 (+14.7%)0.3758 (+1.3%)0.3339 (+14.1%)0.3331 (+14.4%)

A smaller correction helps most in the small-sample RBF setting. Six directions retain almost all prior variance and give lower forecast error than the full-rank correction. The differences shrink at N=512N=512. The benefit therefore depends on the sample size and covariance spectrum. Forecast scores test whether these directions are useful for correction, beyond how well they reconstruct the covariance.

Together with the Ornstein–Uhlenbeck (OU) kernel and the modulated periodic example, these studies cover stationary, smooth, and nonstationary priors. The correction learns the non-Gaussian target behavior. Signature and sequential kernels offer directions for future work.

W4: Learned latent representations

Our representation describes a correction to a forecast. The model keeps the full-resolution GP draw and adds a correction in selected directions. Variation outside those directions is preserved. Autoencoder-based latent flow matching instead learns an encoder and decoder for generation. We will make this distinction clearer.

Once a kernel and its parameters are specified, the Mercer basis needs no separate calibration sample. Computing the covariance and its eigendecomposition still has a cost. The empirical Gaussian basis in our real-data study is fitted from training data. In the synthetic study, we isolate basis estimation by fixing the multilayer perceptron (MLP) training set (N=512,k=8N=512,k=8). We change only the separate calibration sample used to estimate PCA's directions.

Table 3: Robustness to basis-estimation scarcity and historical regime shift (k=8k=8, CRPS ↓\downarrow and Gap ↑\uparrow).

Basis (k=8k=8)Basis Calibration CostSmall Calibration (Nbasis=10N_{\text{basis}}=10)Matched Calibration (Nbasis=128N_{\text{basis}}=128)Shifted Historical Regime
Mercer (Ours)0 separate calibration samples0.1250 (55.6%)0.1250 (55.6%)0.1250 (55.6%)
PCA (Standard)Empirical Cov(yfy_f)0.1387 (49.8%)0.1304 (53.3%)0.1608 (40.4%)
DCT / Fourier0 samples (Fixed)0.2156 / 0.2184 (17%)0.2156 / 0.2184 (17%)0.2156 / 0.2184 (17%)

PCA's forecast quality depends on its calibration data. Scarce or shifted calibration worsens its results even though the correction's training set stays fixed. The specified-prior basis stays unchanged. This isolates the effect of estimating the basis.

Part 3/4: Real-data benchmark and computation

W2 / W6: Probabilistic baselines and numerical results

The original evaluation covered five datasets and summarized results by rank. Your requests for broader coverage, probabilistic baselines, and absolute scores motivated a larger evaluation. We now cover all 54 GIFT-Eval tasks whose datasets contain single-channel series. These span datasets, frequencies, and forecast horizons. We use official splits and forecast origins, and report probabilistic accuracy, point-forecast accuracy, and training cost.

We compare against two groups of forecasters. DLinear, TiDE, PatchTST, and TFT are trained separately on each dataset. DLinear is linear, TiDE is a direct-forecast MLP, and PatchTST is a patch-based transformer. TFT is the Temporal Fusion Transformer. Chronos-2, TiRex, and TimesFM-2.5 are pretrained probabilistic models used without dataset-specific training (zero-shot). Our model fits an empirical Gaussian forecast to the training data, then learns a correction in its covariance eigenbasis.

We report weighted quantile loss (wQL) for probabilistic forecasts and mean absolute scaled error (MASE) for forecast medians. Training cost is measured in millions of floating-point operations (MFLOPs) per sample. Lower is better for all three.

MethodGeom. mean wQL ↓Geom. mean MASE ↓Geom. mean training MFLOPs/sample ↓
MercerFlow (empirical eigenbasis + GP)0.14061.10252.763
TSFlow (locally reproduced)0.15781.1926663.046 (estimated)
DLinear0.20541.34661.456
TiDE0.17511.36770.341
PatchTST0.14491.148811.653
TFT0.14201.152171.051
Chronos-2 (zero-shot)0.11890.9829--
TiRex (zero-shot)0.12050.9936--
TimesFM-2.5 (zero-shot)0.12070.9895--

All columns are geometric means over the same 54 tasks, with equal weight per task. Our results use one training seed. Context lengths and checkpoints are selected on validation data. Our forecasts use 20 velocity-field evaluations per trajectory and 100 draws. TSFlow is locally reproduced on all 54 tasks using seed 6432: we select among OU, squared-exponential, and periodic kernels by the best validation wQL through epoch 40, then continue the selected kernel to 400 total epochs. Its model, optimizer, EMA, gradient clipping, normalization, and sampling settings follow the original TSFlow recipe; early kernel selection and single-seed evaluation define this campaign's budget. The other external accuracy scores come from the published GIFT-Eval results.

Our correction achieves 10.86% lower aggregate wQL and 7.55% lower aggregate MASE than this TSFlow reproduction on the complete shared panel.

Training profiles cover a forward pass, loss computation, and backward pass for one univariate window at batch size one. They include our prior conditioning, sampling, and coordinate transforms. TSFlow uses its final selected kernels and native training loss, including GP conditioning and factorization; S4 arithmetic uses a dense Cauchy reference and analytical FFT estimates. Shared S4 kernel generation and GP factorization are counted once per window at batch one and amortize over larger batches. Optimizer updates, clipping, EMA, validation, and one-time preparation are excluded. Other task-trained baseline costs use profiled reference architectures and context lengths. Accuracy scores follow the published evaluation recipes. The pretrained models require no task-specific training, so their entries are blank in this column. They rely on substantial computation already invested in pretraining. That cost is separate from the task-specific training measured here.

A strong statistical forecast lets a small learned correction reach accuracy competitive with PatchTST and TFT at substantially lower profiled training cost. The aggregate accuracy gains are modest, and performance varies across tasks. The component controls give the clearest evidence for the approach: the correction improves the starting forecast itself. The pretrained models achieve stronger accuracy and require no downstream training.

Part 4/4: Controls, computational scaling, and channels

Prior and representation controls

We compare the statistical forecast alone with three learned models. GP-only is the fitted forecast without a correction. Raw-GP adds a correction in the original time coordinates. It uses the same Gaussian starting forecast and correction network as our model. Raw-noise uses that network in time coordinates but starts from independent Gaussian noise. Our model starts from the GP forecast and learns the correction in its eigenbasis. All neural models use the same training recipe.

The matched controls below use the same 54 tasks and equal-weight geometric mean wQL.

MethodGeom. mean wQL ↓
GP-only (statistical forecast)0.163321
Raw-noise (correction from independent noise)0.156343
Raw-GP (prior + correction in time coordinates)0.141639
MercerFlow (prior + correction in its eigenbasis)0.140626
MercerFlow, half input/output ranks (same width)0.142803

Halving both the input and correction output ranks while keeping the same network width increases aggregate wQL by only 1.55%, preserving most of the full-rank model's accuracy.

The main gain comes from learning a correction to a useful forecast. The correction improves on the GP alone, and starting from that forecast also improves on starting from independent noise. Changing to the prior's eigenbasis gives a smaller additional gain. The synthetic studies examine settings where the choice of correction directions matters more. Our model wins on 23/54 tasks against PatchTST and 21/54 against TFT.

W5 / Q4: Computational scaling

We will separate the costs of preparing the prior, sampling a forecast, and learning its correction. Let CC be the context length and HH the forecast horizon. The conditional covariance is prepared once for a given prior, context, and horizon. Dense preparation can take O((C+H)3)O((C+H)^3) time and O((C+H)2)O((C+H)^2) storage. A smaller correction reduces the MLP's input and output projection cost. In the synthetic suite, parameters fall from 24.9k to 18.0k. The starting forecast still requires full-rank prior sampling. At fixed hidden width, network and projection costs grow with the retained dimensions. Inference cost also grows with solver steps and forecast draws.

Q2: Correlated channels

The evaluated model forecasts each channel from its own history, with neural parameters shared across channels. It targets marginal probabilistic forecasts. Joint multichannel forecasting would also require modeling dependence between channels. We will state this scope clearly in the method and experiments.

Reviewer_E9EXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies conditional flow matching for probabilistic time-series forecasting with a Gaussian-process (GP) source distribution, following TSFlow. Instead of a sequential (S4) backbone in observation space, it encodes the joint context-plus-horizon window in an orthonormal basis and trains a small MLP on the coefficients. Five bases are compared under a matched setup: Raw, Fourier, DCT, PCA, and the eigenbasis of the prior kernel (called the Mercer basis). The authors show that orthonormal bases are equivalent under Euclidean SGD (Prop. 1) but differ under Adam through first-layer conditioning (Prop. 2). They argue that the Mercer basis is data-free, exactly decorrelates the prior, and adapts to non-OU kernels. On five long-horizon benchmarks, the MLP variants are reported to match or beat TSFlow in CRPS with about 4.7× less peak training memory and about 4× less time per epoch.

Critical strengths
  1. A clear, simple idea with a clean controlled comparison: all bases share the same MLP, prior, loss, and protocol.
  2. Prop. 1 isolates optimizer geometry as the only place where the choice of an orthonormal basis can matter, which is an interesting question for GP-prior flow matching.
  3. The MLP is much cheaper to train than S4, and the paper is upfront about its limitations (dependence on kernel choice, univariate channels).
Critical weaknesses
  1. The central mechanism rests on a mis-specified model of Adam. The analysis treats Adam as Jacobi preconditioning with D=diag(A)−1/2D=\mathrm{diag}(A)^{-1/2} and studies κ(DAD)\kappa(DAD). Adam, however, divides gradients by the square root of the second moment (cf. Balles & Hennig [2]). Under the Kronecker model the paper adopts, this applies DD once rather than twice, so the relevant matrix is diag(A)−1/4A diag(A)−1/4\mathrm{diag}(A)^{-1/4}A\,\mathrm{diag}(A)^{-1/4}. Under this model a diagonalized covariance is not optimally conditioned (for the Mercer basis the condition number is κ(K)\sqrt{\kappa(K)}, not 1), so the claimed conditioning advantage does not follow. Prop. 2 also gives the same bound for Fourier and DCT, so it does not separate the bases it is meant to compare.

  2. Under the OU prior used in all main experiments, the Mercer basis is essentially the DCT basis, so the claimed advantage and adaptivity are weakly supported. The paper itself notes that DCT is an asymptotic Karhunen–Loève basis for OU processes, so Mercer and DCT should behave almost identically in the main benchmark. The benefit of Mercer is shown only in a single periodic-kernel experiment on one dataset, and the claim of adapting to non-stationary priors is not tested.

  3. The gains over TSFlow cannot be attributed to the proposed basis. In Table 1 even the plain Raw MLP clearly outranks TSFlow, and by Prop. 1 the choice of basis does not change the model class. The accuracy and efficiency gains therefore come from replacing S4 with an MLP rather than from the Mercer basis. Moreover, TSFlow [1] was originally evaluated on GluonTS datasets with short horizons, whereas here all models are run on different benchmarks with long horizons, so the comparison is not anchored to TSFlow's own setting. Apart from TSFlow, the baselines are deterministic forecasters, which cannot be judged on calibration; probabilistic baselines are missing.

Questions for authors
  1. Could the conditioning analysis be redone with the Adam-consistent matrix diag(A)−1/4A diag(A)−1/4\mathrm{diag}(A)^{-1/4}A\,\mathrm{diag}(A)^{-1/4}? Which conclusions survive?
  2. Decorrelation is exact only for the centred joint prior. Since the conditional covariance Σ\Sigma of the horizon does not depend on the context, why not use its eigenbasis, which decorrelates the source noise exactly for every window? (Relatedly, for an OU kernel the conditional mean is essentially determined by the last context point, so how much history does the prior actually carry?)
  3. Can the method be compared with TSFlow in its original protocol, and with at least one additional probabilistic baseline?
  4. For the periodic-prior study, can a Fourier baseline matched to the period be included, and can the benefit be shown on more than one dataset?

Little tip: you can position the work relative to Functional Flow Matching [3] and Blurring Diffusion Models [4].

References

4 comments

Part 1/4: Positioning and optimizer analysis

We thank you for identifying the error in our Adam analysis and asking us to separate the role of the representation from the efficiency of the neural model. Your comments have helped us refine the paper's positioning. TSFlow focuses on matching the prior to the temporal structure of the data. We ask whether the prior is a good probabilistic forecaster in its own right. When it is, how much can a very small neural network gain from the predictive information it already contains? We build on TSFlow's use of Gaussian process priors in flow matching to learn a small correction to that forecast. The goal is to make the most of that starting forecast with minimal training cost. For real data, we fit an empirical Gaussian prior following Lin et al., Empirical Gaussian Processes (ICML 2026; https://arxiv.org/abs/2602.12082).

Our results show that a small learned correction improves the statistical forecast and reaches accuracy competitive with larger task-trained forecasters, at substantially lower training cost in the profiled implementations.

We first correct the optimizer analysis, then examine when the choice of correction directions matters. The original paper studied optimization and stability. New truncation experiments explore whether the prior can also guide a smaller correction. Finally, the forecasting comparisons and matched controls show what the prior, correction, and neural architecture each contribute.

W1 / Q1: Adam conditioning analysis

Let KK be a fixed positive-definite covariance and Φ\Phi an orthogonal basis. Then A=Φ⊤KΦA=\Phi^\top K\Phi is the covariance in that basis. We use κ\kappa for the condition number and diag⁡(A)\operatorname{diag}(A) for the diagonal matrix formed from AA. The retained eigenvalues are λ1≥⋯≥λk\lambda_1\geq\cdots\geq\lambda_k.

The original analysis applied the second-moment scaling incorrectly. Thank you for catching this error. We have corrected the analysis and will replace Proposition 2 and its interpretation. Adam divides gradients by the square root of the second moment. Under the Kronecker model (Balles & Hennig, 2018), the resulting matrix is MAdam=D1/2AD1/2M_{\text{Adam}} = D^{1/2} A D^{1/2}, where D=diag⁡(A)−1/2D = \operatorname{diag}(A)^{-1/2}. The original analysis used DADDAD. We use this matrix as a frozen input-curvature surrogate. Actual Adam also depends on residuals, momentum, and evolving second moments. In the eigenbasis, the corrected surrogate has condition number κ(A)\sqrt{\kappa(A)}.

In time coordinates, the covariance can retain strong off-diagonal dependencies. In the Mercer basis, the prior covariance is diagonal. However, this does not imply that AdamW achieves optimal conditioning, especially on the conditional training interpolants.

In the Mercer basis, Φ⊤KΦ=Λ\Phi^\top K\Phi=\Lambda is diagonal. The corrected scaling gives condition number κ(A)\sqrt{\kappa(A)} and zero off-diagonal prior covariance. If only kk directions are retained, the condition number is λ1/λk\sqrt{\lambda_1/\lambda_k}. The benefit therefore depends on the retained spectrum. Whitening also rescales the directions. It changes their loss weights unless the loss is adjusted.

We compare the prior's eigenbasis (Mercer) with a basis estimated from data (PCA), two fixed bases (DCT and Fourier), and the original time coordinates (Raw). The representation acts on the forecast horizon of length LfL_f. Here c0c_0 is the starting sample in the chosen coordinates, and ctc_t is its flow-matching interpolation with the target. OffDiag measures off-diagonal covariance coupling; zero means the covariance is diagonal.

Table 1: First-layer interpolant geometry (ctc_t) and prior decorrelation (c0c_0) on Horizon-Only Matérn-3/2 (Lf=32L_f=32).

BasisPrior OffDiag (c0c_0) ↓\downarrowInterpolant OffDiag (ctc_t) ↓\downarrowκ(DAD)\kappa(DAD) (Train / Test) ↓\downarrowκ(D1/2AD1/2)\kappa(D^{1/2}AD^{1/2}) (Train / Test) ↓\downarrow
Mercer (Ours)0.0000.0873.1 / 4.4250 / 320
PCA0.1810.10148.0 / 49.0330 / 450
DCT0.5240.538640 / 870900 / 1,100
Fourier0.5450.5614,500 / 5,5002,900 / 3,800
Raw0.9660.97058,000 / 68,00046,000 / 56,000

With the corrected scaling, Mercer reduces the surrogate's condition number by two orders of magnitude relative to Raw in this experiment. It also removes off-diagonal covariance coupling in the prior.

Part 2/4: Synthetic representation experiments

Q2: Conditional-horizon basis and low-rank correction

Let pp denote the observed past and ff the future. Their covariance blocks are KppK_{pp} and KffK_{ff}, with cross-covariance Kfp=Kpf⊤K_{fp}=K_{pf}^\top. The conditional forecast has mean μ\mu and covariance Σ\Sigma.

Following your suggestion, we use the eigenbasis of the conditional future covariance in the synthetic low-rank experiments: Σ=Kff−KfpKpp−1Kpf=ΦΛΦ⊤∈RLf×Lf\boldsymbol{\Sigma} = K_{ff} - K_{fp}K_{pp}^{-1}K_{pf} = \Phi \Lambda \Phi^\top \in \mathbb{R}^{L_f \times L_f}. The starting forecast remains a full-rank draw y0∼N(μ,Σ)y_0\sim\mathcal{N}(\mu,\Sigma). The multilayer perceptron (MLP) receives the history and prior mean [yp,μ][y_p,\mu] through a separate conditioning branch. It learns a correction in only the top kk directions:

y^f=y0+Φk(c^1−c0),Φk∈RLf×k.\hat{y}_f = y_0 + \Phi_k(\hat{c}_1 - c_0), \quad \Phi_k \in \mathbb{R}^{L_f \times k}.

Here ypy_p is the observed history, and Φk\Phi_k contains the kk retained eigenvectors. The starting coordinates are c0=Φk⊤y0c_0=\Phi_k^\top y_0. The network learns their endpoint c^1\hat c_1.

At full rank (k=Lfk=L_f), the orthonormal bases allow the same corrections and perform similarly. With fewer directions (k≪Lfk\ll L_f), the choice of basis determines what the model can correct. The differences become large in the tested example that departs from stationary Ornstein–Uhlenbeck (OU) structure.

We report the continuous ranked probability score (CRPS), which measures probabilistic forecast error. Lower is better. Percentages show how much of the prior-to-oracle gap the correction closes: (prior CRPS − model CRPS) / (prior CRPS − oracle CRPS). The oracle score is the synthetic benchmark's reference.

Table 2: Controlled truncation study on non-stationary modulated periodic data (Lf=32,p=5.5L_f=32, p=5.5, GP Prior CRPS = 0.2565, Oracle CRPS = 0.0200).

Basisk=6k=6 CRPS ↓\downarrow (Gap ↑\uparrow)k=10k=10 CRPS ↓\downarrow (Gap ↑\uparrow)k=16k=16 CRPS ↓\downarrow (Gap ↑\uparrow)k=32k=32 (Full) CRPS ↓\downarrow (Gap ↑\uparrow)
Mercer (Ours)0.1619 (40.0%)0.0768 (76.0%)0.0597 (83.2%)0.0531 (86.0%)
PCA0.1660 (38.3%)0.0884 (71.1%)0.0613 (82.5%)0.0547 (85.3%)
DCT0.2315 (10.6%)0.2140 (18.0%)0.1834 (30.9%)0.0528 (86.1%)
Fourier0.2304 (11.0%)0.2167 (16.8%)0.1839 (30.7%)0.0543 (85.5%)
Raw0.2331 (9.9%)0.2180 (16.3%)0.1819 (31.6%)0.0546 (85.4%)

At full rank (k=32k=32), forecast errors are close across bases. With only ten correction directions, Mercer closes 76% of the prior-to-oracle gap. DCT closes 18% and Fourier 17%. The basis matters because it determines which corrections the smaller model can learn.

W2 / Q4: OU, periodicity, and nonstationarity

OU is a weak setting for separating Mercer from DCT. We therefore added the nonstationary periodic experiment in Table 2. Its kernel is K(s,t)=(0.5+cmin⁡(s,t))KPer(s,t)K(s,t)=(0.5+c\min(s,t))K_{\text{Per}}(s,t), with period p=5.5p=5.5 and horizon length 32. This tests whether the prior's directions remain useful for correction when the covariance departs from stationary OU structure.

We will revise the periodic-basis discussion to match this evidence. Fourier truncation depends on the chosen period and frequencies. This experiment shows the value of adapting to the specified covariance. The real-data benchmark below tests the broader value of correcting a statistical forecast.

Part 3/4: Real-data benchmark and computation

W3 / Q3: Backbone attribution and forecasting comparisons

The 4.7×4.7\times memory and 4×4\times speed gains over TSFlow come from replacing S4 with an MLP. We will attribute them to that architecture change. To measure the effect of coordinates separately, we use the same MLP and training recipe in time coordinates and in the prior's eigenbasis. Table 2 also compares corrections restricted to fewer directions. These controls distinguish a small full-rank coordinate effect from the larger effect of choosing which directions to correct.

The original evaluation mixed architecture and basis effects and offered limited probabilistic comparisons. We now cover all 54 GIFT-Eval tasks with single-channel series, spanning datasets, frequencies, and short, medium, and long horizons. All comparisons use official splits, forecast origins, and scoring.

Prior and representation controls

GP-only is the fitted statistical forecast. Raw-GP learns a correction in time coordinates; MercerFlow learns it in the prior's eigenbasis. Raw-noise starts from independent Gaussian noise. All neural models share the correction network and training recipe, isolating the effects of the starting forecast and coordinates.

The controls use the same 54 tasks and equal-weight geometric mean wQL.

MethodGeom. mean wQL ↓
GP-only (statistical forecast)0.163321
Raw-noise (correction from independent noise)0.156343
Raw-GP (prior + correction in time coordinates)0.141639
MercerFlow (prior + correction in its eigenbasis)0.140626
MercerFlow, half input/output ranks (same width)0.142803

Halving the input and correction output ranks while keeping the same network width increases aggregate wQL by only 1.55%, preserving most of the full-rank accuracy.

The main gain comes from learning a correction to a useful forecast. The correction improves on the GP alone, and starting from that forecast also improves on starting from independent noise. Changing to the prior's eigenbasis gives a smaller additional gain. The synthetic studies examine settings where the choice of correction directions matters more.

Broader forecasting comparison

DLinear, TiDE (a direct-forecast MLP), PatchTST, and TFT are trained separately on each dataset. Chronos-2, TiRex, and TimesFM-2.5 are pretrained probabilistic models evaluated zero-shot. Our model learns a correction to an empirical Gaussian forecast.

We report weighted quantile loss (wQL) for probabilistic forecasts and mean absolute scaled error (MASE) for forecast medians. Training cost is measured in millions of floating-point operations (MFLOPs) per sample. Lower is better for all three.

MethodGeom. mean wQL ↓Geom. mean MASE ↓Geom. mean training MFLOPs/sample ↓
MercerFlow (empirical eigenbasis + GP)0.14061.10252.763
TSFlow (locally reproduced)0.15781.1926663.046 (estimated)
DLinear0.20541.34661.456
TiDE0.17511.36770.341
PatchTST0.14491.148811.653
TFT0.14201.152171.051
Chronos-2 (zero-shot)0.11890.9829--
TiRex (zero-shot)0.12050.9936--
TimesFM-2.5 (zero-shot)0.12070.9895--

All columns are equal-weight geometric means over the same 54 tasks. Our results use one training seed, with context lengths and checkpoints selected on validation data. Our local TSFlow reproduction uses one seed and its original training recipe, with early validation-based selection among three kernels followed by 400 total training epochs. Other external accuracy scores come from the published GIFT-Eval results.

Our correction achieves 10.86% lower aggregate wQL and 7.55% lower aggregate MASE than this TSFlow reproduction on the complete shared panel.

Training costs cover forward and backward passes, including our prior conditioning, sampling, and coordinate transforms, at batch size one. TSFlow's estimate includes GP operations and estimated S4 arithmetic; shared costs amortize over larger batches. Baseline costs use reference architectures and context lengths. These profiles exclude optimizer updates, validation, and one-time preparation. The zero-shot models require no task-specific training; their pretraining cost is separate.

A strong statistical forecast lets a small learned correction reach accuracy competitive with PatchTST and TFT at substantially lower profiled training cost. The aggregate accuracy gains are modest, and performance varies across tasks. The component controls give the clearest evidence for the approach: the correction improves the starting forecast itself. The pretrained models achieve stronger accuracy and require no downstream training.

Part 4/4: History conditioning and related work

Q2: Historical information under the prior

The correction has direct access to history. In the synthetic horizon-only model, the MLP receives both the observed history ypy_p and the prior mean μ\mu. It can therefore learn dependencies that the prior misses.

For an exactly observed latent OU process, the last state is sufficient for the prior's conditional forecast. The correction receives the full history and can learn dependencies beyond this summary. The real-data controls show the value of adding a correction: it improves aggregate wQL by 13.90% over the fitted empirical Gaussian forecast.

Related work

Thank you for these connections. We will add both references and clarify how our work builds on them. Functional Flow Matching (Kerrigan et al., 2024) provides a function-space framework for learning transport from a Gaussian prior. We use this perspective for conditional forecasting, with a small learned correction to a statistical forecast. Blurring Diffusion Models (Hoogeboom & Salimans, 2023) shows how structured noise can encode useful assumptions for images. TSFlow applies the corresponding idea to time series through Gaussian process priors. We build on this foundation by studying how to make the most of the starting forecast with a small correction, and how its coordinates affect learning.

Comments on the paper

1 comment

We thank the reviewers for their constructive feedback, which has helped us refine the positioning of our paper. TSFlow focuses on matching the prior to the temporal structure of the data. We ask whether the prior is a good probabilistic forecaster in its own right. When it is, how much can a very small neural network gain from the predictive information it already contains? Building on TSFlow, we learn a small correction to that forecast. The challenge is to build on that strong starting point as effectively as possible, with minimal training cost.

Our results support this approach. A small learned correction improves the statistical forecast and reaches accuracy competitive with larger task-trained forecasters, at substantially lower training cost in the profiled implementations. Expressing the correction in the prior's eigenbasis gives a smaller additional improvement. These results show the value of combining a strong statistical forecast with a small learned correction.

The reviews also prompted us to clarify what the choice of coordinates contributes. The original paper studied full-rank rotations and their effect on optimization and stability. In the rebuttal, we also test smaller corrections restricted to selected directions in synthetic experiments. The basis then determines which corrections the model can express. Nonstationary and smooth-kernel examples show when this choice improves accuracy or regularization. We have also corrected the Adam analysis. The measured efficiency gains from replacing the S4 sequence model with an MLP come from that architecture change.

The individual responses below develop these points and provide the supporting experiments and numerical comparisons.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.