acceptodds
Under review as a conference paper at ICLR 2027

Pairwise Consistency Models for Bayesian Transport and Variational Data Assimilation

Abstract

Bayesian inference in high-dimensional dynamical systems is traditionally approached through either score-based diffusion models or variational data assimilation. Although developed independently, both rely on the numerical integration of continuous Bayesian transport dynamics, making inference computationally expensive. We introduce a unified pairwise consistency framework that learns finite-time Bayesian transport operators between arbitrary solver times instead of infinitesimal transport fields. By enforcing pairwise consistency, the proposed approach preserves the semigroup structure of the underlying flow while avoiding explicit integration of the transport dynamics. This perspective naturally generalizes consistency models beyond diffusion processes and extends to variational inference without requiring an explicit forward diffusion model. The resulting framework encompasses supervised diffusion learning, self-consistent variational optimization, and hybrid diffusion–physical transport within a common mathematical formulation. Experiments on synthetic and geophysical inverse problems demonstrate efficient few-step reconstruction, sequential forecasting, and uncertainty-aware posterior sampling, showing that finite-time transport learning provides an effective alternative to conventional score-based integration.

Primary area:
generative models
Keywords:
Consistency Models, Variational Data Assimilation, Bayesian Inference, Generative Modeling, Spatiotemporal Dynamics, Uncertainty Quantification

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_aaynOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes learning finite-time transport operators rather than infinitesimal transport fields for posterior sampling in dynamical systems. The authors introduce four variants to further discuss the idea.

Critical strengths

1.The finite-time transport perspective offers an interesting alternative to modeling infinitesimal flows. 2. The four variants systematically address different scenarios.

Critical weaknesses

1.While it is appreciated that the authors admitted about VarCM convergence and identifiability being open problems, the paper provides very little discussion of what happens if neither the variational objective J nor the transport operator g converge properly. If both components fail to converge, how can we trust that the final learned sampler produces valid posterior samples? This concern deserves more attention than a single sentence acknowledgment. 2.The font sizes in the main figures (especially Figure 2) are far too small to read. Axis labels and legends appear to be generated directly by plotting code without manual adjustment for publication. 3. Table 1 shows Pairwise CM has CRPS 6.8 times worse than Flow Matching on the GP task (0.109 vs 0.016), even though RMSE is similar. Since GP has an exact posterior, this suggests the method severely miscalibrates uncertainty. The paper does not analyze why consistency constraints fail to preserve posterior variance. 4. The paper builds on Consistency Trajectory Models but does not directly compare with CTM. Given that Pairwise CM appears to be a reformulation of CTM's pairwise consistency, it is hard to assess the actual improvement. 5. NFE versus accuracy curves are relegated to the appendix (line 922), but they directly support the main claim about computational efficiency. These should be in the main paper.

Questions for authors

1.Can you discuss what theoretical or empirical evidence suggests VarCM training converges to a valid posterior sampler? 2. Why does CRPS degrade so much in the GP experiment? Does the consistency constraint inherently distort the learned posterior variance, and if so, is this a fundamental limitation? 3. How does Pairwise CM compare directly to CTM on the same tasks? What is the actual novelty beyond reformulation? 4. The paper enforces semigroup consistency only on sampled triplets. Can you discuss whether this guarantees consistency for unsampled time combinations?

1 comment

We thank the reviewer for the constructive review. All five weaknesses led to changes.

W1 / Q1. What if Jϕ\mathcal J_\phi or gθg_\theta do not converge? The revision makes this question checkable. (i) VarCM is now trained only through a strictly proper score (fair ensemble CRPS of fields and increments) of the ensemble it generates, with no L2L^2 anchor: at its optimum, members follow the conditional law of these quantities. (ii) ∇Jϕ\nabla\mathcal J_\phi is a conditioning input of gθg_\theta, not the transport itself: a poorly learned prior makes the conditioning less informative, but the score still calibrates the output given that conditioning, and the observation term of Jϕ\mathcal J_\phi has no parameters. (iii) Convergence is monitored by quantities that do not require it: the checkpoint is selected on validation CRPS, and we report SSR, rank histograms and a composition test. On GP, whose posterior is known, VarCM reaches SSR 0.80 with a nearly flat rank histogram and the lowest RMSE (0.182 vs 0.198 for FM). The remaining open points are stated in the conclusion: calibration holds for marginals and increments, not for the full joint law, and DynVarCM is consistent in distribution, not sample by sample.

W2. Figure fonts. Fig. 2 was redrawn with readable fonts; the new figures (rank histograms, NFE study) use the same font sizes.

W3 / Q2. CRPS of CM vs FM on GP. Two causes, both fixed. The submitted CRPS values were not computed with the same normalisation for all models; all CRPS are now computed by one code, over all grid points. And the submitted CM targets built from a shared noise draw do shrink the variance (new Gaussian appendix: one-step std 0.0125 instead of 1). CM is now trained with targets from an ODE teacher, as CTM. Result on GP: CRPS 0.110 for CM vs 0.111 for FM, SSR 0.82 vs 0.70. Consistency itself does not distort the variance; the training targets did.

W4 / Q3. Comparison with CTM. In the revision, CM is a CTM-style model (two-time map, teacher ODE targets) conditioned on the observations, and we do not claim the pairwise primitive; CM is the CTM reference of our framework, with the same architecture and teacher as the other models. The novelty lies elsewhere: VarCM and DynVarCM learn the transport without teacher or reference trajectories, conditioned on a learned variational gradient and trained by a proper score; DynCM/DynVarCM put spin-up and physical time on one timeline, giving a smoothed trajectory with one evaluation per frame. On GP, DynVarCM improves on DynCM (CRPS 0.107 vs 0.127).

W5. NFE in the main text. Done (Fig. 4, Sec. 6.1), for all four models, with SSR and a composition test.

Q4. Consistency on unsampled time pairs. Training draws random pairs, so consistency is enforced in expectation, not for all triplets. We therefore test it on grids not used in training: the composition gap (samples of a grid vs the finest grid, same noises) decreases with the number of steps for CM, DynCM and VarCM. DynVarCM keeps its RMSE and SSR on refined grids, but its samples move by more than one spread: it is consistent in distribution only, which we state as a limitation.

Reviewer_XETnOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper learns two-time transport maps g(x,t,t′) instead of integrating transport fields, unifying diffusion sampling and learned 4DVar. Beyond CTM's pairwise primitive, it adds VarCM (teacher-free self-consistency conditioned on ∇Jϕ), DynCM (spin-up, then physical time, in one operator) and DynVarCM (both), evaluated on GP, SSH and SIC.

Critical strengths
  1. A clear unifying view of diffusion samplers, consistency models and learned 4DVar as transport integrators (Sec. 2, App. A.3), candid about open identifiability (L308–310).
  2. Realistic testbeds (public SSH OSSE, real SIC) with a timeline-matched control (DynFM).
  3. Strong results in places: SSH RMSE (0.055−0.062)/0.062 = −11.3% for CM vs FM, approximately matching FM's best at 2 NFE (Fig. 7a); SIC RMSE (0.108−0.126)/0.126 = −14.3% for DynVarCM.
Critical weaknesses

1 Semigroup preservation (L017–018) rests on Eq. 20, treating shared-noise states (L776) as lying on "the same reverse trajectory" (L764–765); they lie on the forward interpolant. In my toy (N(0,1) data, VE, σ 10→1→0), the exact PF-ODE map has loss (1/√101−1/√2)² + (10/√101−1/√2)² ≈ 0.45, while a constant map scores 0. Shared noise is justified only for adjacent levels as Δt→0 (Song et al., ICML 2023, Thm. 2), and CTM's targets use a teacher ODE solver (Kim et al., 2024, Eq. 3). DynCM (Alg. 5) is plain L2 regression, minimized by a conditional mean; with shared-noise references (step 6 is unspecified) this is a DDIM step, which is no semigroup (toy, σ 80→0: 1.6×10⁻⁴ directly vs 2.4×10⁻³ via σ=5) and has one-step output std 1/√6401 = 0.0125, not 1. CM's step-count dependence (approximately 11% RMSE change, Fig. 7a; L1524–1525) fits this. Adjacent-step EMA or teacher-trajectory targets would address this.

2 Under the linear schedule α=t′/t (E.3, L1607), g = m + (t′/t)(x−m) has zero pairwise loss (Eq. 24, Alg. 3) and minimizes both L2 anchors at m = E[x0|·]. It need not follow Jϕ's gradient flow (curved for anisotropic J), contrary to L177–178, and targets the mean, not the mode (L911–912). Alg. 3 never evaluates ∇JΦ or uses y (cf. Eq. 27), and L_IC and L_phys (Alg. 7) are undefined. A tangent condition tying ∂_{t′}g at t′=t to ∇Jϕ would make the claim well-posed.

3 The pairwise primitive is not claimed (L255–257), but VarCM's direct-vs-composed loss has the form of progressive self-distillation (Boffi et al., NeurIPS 2025) and shortcut-model self-consistency (Frans et al., ICLR 2025), both anchored by the flow-matching term W2 lacks. MeanFlow (NeurIPS 2025), GCTM (ICLR 2025), SDA (NeurIPS 2023; short-segment scores, cf. L320–323) and FlowDAS (NeurIPS 2025) also need discussion.

4 For "uncertainty-aware posterior sampling" (L025, L483–484): x(1)=σmax z is independent of x0, so the L2-optimal VarCM output ignores z (zero spread), and App. D's stochastic variants are never evaluated. The GP posterior is known (L417–419), yet no exact-posterior RMSE, CRPS or spread is reported. On GP, CRPS/RMSE is 0.016/0.198 = 0.08 (FM) and 0.018/0.229 = 0.08 (DynVarCM), vs 0.47 (CM) and 0.47–0.73 on SSH/SIC. Heavy-tailed ensembles can score low, but none can beat the exact posterior's expected CRPS (propriety); reporting it would resolve this.

5 Forecasting (L024, L483) is never tested; all tasks are reconstructions (L366–367). NFE evidence covers only CM vs FM on SSH; NFE per entry, ensemble size, hyperparameters, seeds and splits are unreported. The SSH training window is unstated; the challenge allows full-field learning only on 2013-01-02→2013-09-30. λx lacks units; if in degrees, the 4DVarNet baseline (1.35) trails every leaderboard 4DVarNet entry (0.62°–1.19°). Ablations are missing; VarCM never beats CM in RMSE, and DynVarCM beats DynCM only on SIC.

6 App. F is incorrect . Eqs. 59–60 (RTS-gain backward conditional) are right, but Eqs. 61 (G = M − QP⁻¹) and 62 (Σ_back = Q) are false; L1989 itself gives M⁻¹(I − QP⁻¹). My scalar check (m=0.9, q=0.1, P=1): G = 0.9/0.91 = 0.989, not 0.790, and Σ_back = 1 − 0.81/0.91 = 0.110, not 0.1. Eq. 62 holds only at stationarity (L2018–2020 is wrong), and App. F has no observations, Jϕ or gθ, so L407–410's check is absent; deriving DynVarCM's actual linear-Gaussian limit would resolve this.

Questions for authors

Questions are resolved.

3 comments

We thank the reviewer for a very precise review; the toy computations were decisive for the revision.

W1. Shared-noise targets, semigroup. Agreed: shared-noise states lie on the forward interpolant, not on a reverse trajectory. CM and DynCM are now trained as CTM, with targets from an ODE teacher (one Heun step of a flow-matching model, then the EMA model) (Sec. 3, Alg. 1). A new appendix reproduces the reviewer's toy: shared-noise targets give a one-step std of 0.0125 and a semigroup gap of 2.3×10−32.3\times10^{-3}; teacher targets give std 1.000 and a gap below 10−1710^{-17}. On GP, CM with one evaluation matches FM with 40 (RMSE 0.198 vs 0.200), with SSR 0.75–0.91 from 1 to 15 evaluations, and a composition test (samples of a grid vs the finest grid, same noises) decreases with the number of steps (Fig. 4).

W2. Collapse of VarCM to m=E[x0∣⋅]m=\mathbb E[\mathbf x_0\mid\cdot]; gradient flow; Alg. 3. Agreed. The L2L^2 anchors came from learned 4DVar, whose natural target is the conditional mean; self-consistency alone does not regularise away from it. The formulation is unchanged, but the training criterion now contains no L2L^2 anchor: data enter only through the fair ensemble CRPS (strictly proper) of M=8M=8 members, including the student's direct jump from the noise (Sec. 4, Alg. 3). Alg. 3 now evaluates ∇~Jϕ(⋅,y)\widetilde\nabla\mathcal J_\phi(\cdot,\mathbf y) at every call; LICL_{\rm IC}, LphysL_{\rm phys} are removed. We no longer claim that gθg_\theta follows the gradient flow of Jϕ\mathcal J_\phi or targets the mode: ∇Jϕ\nabla\mathcal J_\phi is a conditioning input, and the tangent condition the reviewer suggests is noted as a way to make that link exact.

W3. Related work. A Related Work section now discusses flow maps / progressive self-distillation, shortcut models, MeanFlow, GCTM, SDA and FlowDAS. The difference with self-distillation and shortcut models is the anchor: VarCM and DynVarCM have no flow-matching or score term, the anchor is a proper score of the generated ensemble.

W4. Uncertainty. The spread is now trained (proper score on the direct jump from the noise) and evaluated: SSR and rank histograms everywhere. GP SSR: FM 0.70, CM 0.82, VarCM 0.80, DynFM 0.99, DynCM 0.96, DynVarCM 0.82. The CRPS is now computed over all grid points; CRPS/RMSE is 0.54–0.56 for every model on GP, close to 1/π≈0.561/\sqrt\pi\approx0.56, the value for a calibrated Gaussian ensemble (the submitted ratios came from a different normalisation). The stochastic variants (former App. D) were removed and are mentioned only as future work.

W5. Claims, protocol, ablations. Forecasting claims were removed. NFE, ensemble size (20) and protocol are stated in Sec. 6; NFE studies cover all four models on GP (App. C) [TBD: SSH]. SSH splits: training 2013-02-24 → 2013-09-30 (within the challenge's full-field period), test 2012-10-01 → 2012-12-20; λx\lambda_x in degrees for SSH, pixels for GP and SIC. Our 4DVarNet is a controlled baseline trained on the same region and budget as our models, not the leaderboard configuration. With the revised training, VarCM beats CM on GP (RMSE 0.182 vs 0.198, CRPS 0.101 vs 0.110), and DynVarCM beats DynCM on GP (0.197 vs 0.229) and SSH [TBD].

W6. App. F. Thank you: Eqs. 61–62 are corrected (the reviewer's scalar example, G=0.989G=0.989, Σback=0.110\Sigma_{\rm back}=0.110, is now given), the stationarity claim is fixed, and the appendix states that it concerns the unconditional chain only, without observations, Jϕ\mathcal J_\phi or gθg_\theta.

Questions. The four questions appear to refer to another submission (hard gates, Plate/AMG, ETH, two-pulse test); It would be nice to not copy paste questions of another paper before sending the comments to the authors. I know that we are in the LLM era but a bit more attention would be appreciated

Most of my other points are addressed. Appreciated the work that authors put to improve the paper. For example, you dropped the anchors and the gradient-flow, mode and forecasting claims, corrected App. F, and added the related work.

Some things still hold me back. The training objective for VarCM and DynVarCM has changed quite a lot, so the method is now fairly different from the one submitted. Some key results are also still marked TBD, including the SSH NFE study and DynVarCM vs DynCM on SSH. On GP, the CRPS/RMSE ratios went from about 0.08 to about 0.55 after the normalisation change, and there’s still no comparison against the exact GP posterior. That posterior is known, so comparing against it would be a good calibration check.

I’m raising my score accordingly.

Thank you for raising your score and for the precise follow-up. I would like to answer as well on your last points, because they were most of them alreadfy available on my side, or fast to produce, and I believe they can again help in understanding the main key messages of the paper, and strengthen the results.

Objective. The change is real, but confined to the training loss of VarCM/DynVarCM. The model, clock, architecture and samplers are unchanged. The L2L^2 anchors targeted E[x0∣y]\mathbb E[\mathbf x_0\mid\mathbf y] and collapsed the ensembles, as you noted. The fair CRPS (strictly proper) targets the conditional law instead.

TBD entries, now filled.

  • DynVarCM vs DynCM on SSH (Table 3): RMSE 0.055 vs 0.079, CRPS 0.029 vs 0.042, SSR 0.58 vs 0.98.
  • SSH NFE study (App. C, new figure):
    • DynCM is flat from 9 to 72 evaluations (RMSE 0.079–0.083, SSR ≈ 1), i.e. one per frame, against 34–90 for DynFM.
    • DynVarCM is best with one jump per frame.
    • CM needs ~9 evaluations on SSH (RMSE 0.083 at 2, 0.064 at 9); FM converges from 10.

CRPS/RMSE. For a calibrated Gaussian ensemble the ratio is 1/π≈0.561/\sqrt\pi\approx0.56. The submitted 0.08 was not attainable and was retracted.

Exact GP posterior (new row of Table 2, Sec. 6.1).

  • Computation. We use the exact prior of the generating scheme (checked on the data: residual variance 0.9985 vs 1) and condition exactly on the noise-free observations, with 20 exact samples.
Reviewer_qyTjOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper extends the existing consistency trajectory models for Bayesian inverse problems with 4 formulations. Pairwise CM is a CTM conditioned on the observation. VarCM introduces variational-gradient conditioning. DynCM concats the diffusion time and physical time in one timeline. DynaVarCM combines the variational-gradient conditioning with DynCM.

Critical strengths

N/A

Critical weaknesses
  • Algorithm 3 is flawed. The current form leads to a collapsed data mean.
    • Let μ=E[X0]\mu=\mathbb{E}[X_0] and Gθ(x,t,s)=μG_{\theta}(x,t,s)=\mu , so gθ(x,t,s)=stx+(1−st)μg_{\theta}(x, t, s) = \frac{s}{t}x+(1- \frac{s}{t})\mu.
      • This gives zero pairwise loss in algorithm 3. And the mapping to data mean gθ(x,t,0)=μg_{\theta}(x, t, 0)=\mu for any t.
    • The training algorithm of VarCM (algorithm 3) does not mention variational gradient at all while its sampling algorithm (algorithm 4) requires variational gradient as input to the network.
  • Limited empirical evaluation. The paper missed many relevant baselines and ablation study.
    • The author should evaluate the proposed methods on the settings in [1] and [2] and compare against the baselines there.
  • Claims about uncertainty calibration do not have supporting evidence:
    • Abstract claims "uncertainty-aware posterior sampling.”
    • Fig 7. claims "stable ensemble calibration". But the reported metrics RMSE and CRPS do not measure uncertainty calibration.

[1] : Rozet, François, and Gilles Louppe. "Score-based data assimilation." Advances in Neural Information Processing Systems 36 (2023): 40521-40541.

[2] : Huang, Langwen, et al. "Diffda: a diffusion model for weather-scale data assimilation." arXiv preprint arXiv:2401.05932 (2024).

Questions for authors
  • Can the authors evaluate their methods in the setting of [1] and [2] and compare against the baselines?
  • Can the authors report the Spread-Skill-Ratio to evaluate the uncertainty calibration?

[1] : Rozet, François, and Gilles Louppe. "Score-based data assimilation." Advances in Neural Information Processing Systems 36 (2023): 40521-40541.

[2] : Huang, Langwen, et al. "Diffda: a diffusion model for weather-scale data assimilation." arXiv preprint arXiv:2401.05932 (2024).

1 comment

We thank the reviewer. The collapse argument and the missing calibration evidence were both correct; the revision addresses them directly.

  1. Collapse to the data mean (Alg. 3). Agreed. The submitted data term was the L2L^2 reconstruction loss of learned variational solvers such as 4DVarNet, whose minimiser is the conditional mean, the right target for a deterministic analysis. Self-consistency was meant to make the transport stochastic, but a deterministic map satisfies it, so it did not regularise enough (closed-form illustration in the new Gaussian appendix). The formulation is unchanged; the training criterion is completed: no term regresses on x0\mathbf x_0 any more, and the data enter only through the fair ensemble CRPS (strictly proper) of M=8M=8 members, including the student's direct jump from the noise, which makes the spread a learned quantity (Sec. 4, Alg. 3). On the GP benchmark VarCM now has SSR 0.80, a nearly flat rank histogram, and the lowest RMSE (0.182 vs 0.198 for FM).

  2. Variational gradient missing from training. Fixed: in Alg. 3 every call to gθg_\theta recomputes d=∇~xJϕ\mathbf d=\widetilde\nabla_{\mathbf x}\mathcal J_\phi at its input, as in sampling.

  3. Calibration evidence, SSR. All results now report SSR, CRPS over all grid points and rank histograms. GP (known posterior), SSR: FM 0.70, CM 0.82, VarCM 0.80, DynFM 0.99, DynCM 0.96, DynVarCM 0.82. SSH: CM 0.83 [TBD others]. The step-count study (Fig. 4) now includes SSR (CM: 0.75–0.91 from 1 to 15 evaluations) and a composition test; "uncertainty-aware" was removed from the abstract.

  4. SDA and DiffDA. Both are now discussed in a Related Work section; they rely on tens to hundreds of network evaluations per sample, while our models use one per frame. A comparison in their settings was not feasible within the rebuttal period. We added instead a benchmark with known posterior, many-step flow-matching samplers as generative references, and ablations of training targets, step count and time grid.

Presentation. Method sections rewritten, with a notation paragraph, a summary table of the four models and an appendix notation table.

Comments on the paper

2 comments
Area_Chair_zNkoOpenReview ↗

Please engage in the discussion

Authors: please respond to reviewers' questions and engage in the discussion.

Dear Area Chair,

We would like to share a few points about the revision and the reviews.

A real issue, now corrected. Two reviewers (R1 and R2) correctly identified that the objectives of the variational models (VarCM, DynVarCM) admitted a collapsed solution: inherited from learned variational assimilation, their L2L^2 data terms are minimised by the conditional mean, which self-consistency alone does not prevent. We acknowledge this without reservation. The formulation is unchanged, but the training criterion is now a strictly proper score of the generated ensemble, with no regression on the data, and the distilled models use ODE-teacher targets as in CTM.

All experiments redone. We retrained and re-evaluated every model on the three benchmarks, with calibration metrics throughout (spread–skill ratio, rank histograms, CRPS over all grid points, full test sets, 20 members). The results are now satisfactory in all cases: on the Gaussian-process benchmark, whose posterior is known, the four models are calibrated (SSR 0.80–0.96) and the two teacher-free models are the most accurate of their families. Open questions remain (calibration of marginals and increments rather than of the full joint law; pathwise consistency of DynVarCM) and are stated explicitly in the paper.

A much lighter paper. Following the comments of R1 and R3 on readability, the presentation was rewritten and the paper reduced from 40 to 30 pages, appendices included (notation table, summary table of the four models, unevaluated stochastic variants removed, related work added).

On the reviews. We found the reviews useful: they led us to strengthen the existing framework rather than change it. We note, however, that all four questions of R2 (hard gates, Plate/AMG, ETH, two-pulse test) concern another submission. In the era of LLMs, producing critiques of papers has become easy; a little more attention to whether the questions concern our paper would have been appreciated.

Thank you for your work on this submission. The new version of the paper, once finalized, will be updated on Nov. 19.

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.