acceptodds
Under review as a conference paper at ICLR 2027

Chromatic Gibbs Sampling for Long-Range Data Assimilation

Abstract

Estimating the state of an evolving system from intermittent, noisy observations is a central scientific problem, known as data assimilation (DA). Training generative models directly for long-range DA, such as atmosphere or ocean state estimation, can be prohibitively expensive. A common alternative is to compose models with short time horizons at inference. However, existing composition methods fail to sample from the target assimilation distribution, either due to approximations or intrinsic flaws. Drawing from the Gibbs sampling and graphical model literature, we introduce GiBBS, an iterative method for composing Markov-local generative models that provably converges to the target distribution under reasonable assumptions. We validate and ablate our method on the classical Ornstein-Uhlenbeck and Lorenz-63 systems, before scaling it to high-dimensional Kuramoto-Sivashinsky and turbulent Navier-Stokes systems. Across these experiments, GiBBS consistently and substantially improves upon its initial guess in few (1-16) cycles, highlighting its potential for operational assimilation.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Diffusion models, Bayesian Inference, Dynamical Systems, Data Assimilation, Probabilistic Modeling

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_1RogOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper alternates block updates from short window generative models and shifts the block boundaries to propagate information from sparse observations along a long trajectory. It compares this approach with autoregressive and all at once sampling on several dynamical systems.

Critical strengths

The analytical Gaussian process experiment uses exact local conditionals and permits direct comparison with the true posterior. Its analyses of initialization and block shifts also examine burn in and sample mixing separately. This validates the central sampling mechanism more convincingly than final prediction errors alone would.

Critical weaknesses

The target posterior guarantee in “Block & Blanket” depends on neighboring states forming a sufficient Markov blanket for each block. Yet the appendix section “STOCHASTICITY” says that the turbulent flow is simulated on a finer grid and stored after coarsening, so that “the unresolved scales act as model error on the coarse dynamics.” Such a coarse process may retain temporal memory beyond the conditioning strip, while the experimental setup uses the narrowest strip because its temporal windows are short. The paper does not check whether a coarse state is conditionally independent of more distant states given this strip. Neither the exact posterior comparison on the analytical Gaussian process nor predictive scores on learned systems rule this out, so the turbulent flow improvement does not establish that the chain targets that system's true assimilation posterior.

The “Parallelism” section describes a cycle as sequential updates of alternating block sets and derives a cycle time independent of trajectory length given sufficient processors. Each block update still requires a full reverse diffusion, and the appendix “SAMPLING” configurations show repeated updates after initialization on the learned systems. “COMPUTE RESOURCES” reports training cost only, without inference latency, parallel resource use, or quality comparisons at equal compute budgets. The current results therefore show improved quality after extra cycles but do not establish its practical time and resource cost relative to autoregressive and all at once sampling. The abstract's claim of potential for operational assimilation consequently lacks an efficiency assessment.

Questions for authors

Could the authors test whether more distant history still informs a target block of the coarsened flow after conditioning on the current strip, and explain what this implies for the scope of the target posterior guarantee in that experiment?

Could the authors report inference time or computation against sample quality under the same hardware and stated parallel resources, comparing the complete procedure including initialization with the baselines at equal budgets?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 1Rog

We would first like to thank the reviewer for their relevant review and valuable feedback. We hope to address your concerns both in this response and in the revised manuscript (changes in blue).

W1 & Q1. Markov order approximation

Following your suggestion, we retrained a denoiser on Kolmogorov flow with a larger window size, W=7W = 7, to allow an ablation on the size of the conditioning strip. The new Table 12 (Appendix D.3, referenced l. 327–329) shows that a wider conditioning strip (C=2C = 2 instead of C=1C = 1) does not improve the quantitative ensemble metrics on Kolmogorov. Skill and CRPS are unchanged, and the only notable difference is a slightly better calibrated SSR for C=2C = 2. If more distant history still informed a block beyond the current strip, conditioning on an additional frame on each side would improve the reconstruction. Since it does not, this experiment indicates that the memory introduced by the coarse-graining is weak beyond one frame at this resolution, which validates the first-order Markov assumption, and thus the size of the Markov blanket used, as reasonable in practice. Regarding the scope of the guarantee, the manuscript now states that on Kolmogorov, GiBBS targets the posterior under this first-order Markov approximation of the coarse dynamics (l. 327–328).

W2 & Q2. Computational efficiency

All samplers are now evaluated on the same hardware, a single NVIDIA RTX A5000 GPU, and every table reports both the wall-clock time and the number of function evaluations (NFEs) of the complete procedure, initialization included (Tables 1, 3 and 4). The compute and time complexity of the three samplers is now analyzed in Section 2 (l. 122–137) and Table 2. To relate quality to cost, Figure 15 (Appendix D.2) reports the CRPS of GiBBS on Kuramoto-Sivashinsky as a function of the number of cycles, with AR and AAO as references. From a blind initialization, which comes at no cost, GiBBS matches AR after 8 cycles, that is 16 sequential reverse diffusions against the 36 of the AR rollout (l. 349–351).

That said, we do not claim that GiBBS is faster than other composition methods. The perspective we find most relevant is that of quality: a sampler that runs fast but does not target the right distribution returns quick but incorrect trajectories, which is of little use for data assimilation, where the posterior itself is what we seek. This is exactly where existing composition methods fall short. AR cannot anticipate observations it has not reached yet, so it drifts and is then pulled back onto them abruptly, while AAO hardly carries information away from the observation sites. On Ornstein-Uhlenbeck, where the exact posterior is available in closed form, neither of them reaches it (Figures 2B and 4B). GiBBS, on the other hand, targets the true posterior, and every additional cycle moves the chain closer to it, at the price of more compute. The number of cycles thus becomes a trade-off for practitioners, depending on how much they are willing to spend to obtain good posterior samples. Updating all blocks of a color in parallel keeps this cost reasonable, and for long trajectories (N>2IN > 2I), GiBBS even needs fewer sequential denoiser calls than AR (l. 136–137).

We believe we have addressed your concerns and would be thankful if you could raise your score accordingly. We remain available for any further clarification.

Thank you for your response. It addressed my concerns, and I have decided to increase the score.

Reviewer_tkfcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper adapts chromatic Gibbs sampling to use diffusion models trained on short windows for data assimilation over longer trajectories. It samples alternating blocks in parallel, conditioned on neighboring states and local observations, and shifts the block boundaries between cycles to improve mixing. The evaluation uses an Ornstein–Uhlenbeck process to study convergence and block shifting, then extends to three nonlinear systems where the method achieves lower CRPS than autoregressive and all-at-once baselines.

Critical strengths
  1. I find the Gibbs formulation useful here. It lets short-window models incorporate observations from both sides of a trajectory without assuming that the noisy states remain Markov-local. The argument is straightforward under exact conditional sampling, and the authors acknowledge where the learned implementation introduces approximations.

  2. The OU experiments are very helpful. The posterior is known, so readers can check convergence and see what shifting the blocks changes. The gains also carry over to the nonlinear systems.

Critical weaknesses

None

Questions for authors

How much additional inference time does GiBBS need for the gains in Table 2, including initialization? It would be helpful if the authors could show CRPS against runtime as the number of cycles increases, with AR and AAO included for reference. This would help readers judge whether the improvement is worth the added cost.

3 comments
AuthorsOpenReview ↗

Response to Reviewer tkfc

We would first like to thank the reviewer for their interest in our work and for their very relevant question.

All tables reporting quantitative results now include the runtime on a single NVIDIA RTX A5000 GPU and the number of function evaluations (NFEs), initialization included (now Tables 1, 3 and 4; changes are highlighted in blue in the revised manuscript). We also added an analysis of the compute and time complexity of the three samplers (Section 2, l. 122–137, and Table 2).

We understand your concern about the computational cost, but we believe speed is only meaningful if the resulting trajectories are correct. A fast sampler that does not target the right distribution produces quick but wrong answers, which is of little use for data assimilation, where the posterior itself is the object of interest. This is precisely the limitation of existing composition methods. AR is blind to future observations by construction and drifts before being abruptly pulled back onto them, while AAO fails to propagate information away from the observation sites. We therefore do not claim that GiBBS is faster than other composition methods. The perspective we find most relevant is that of quality: unlike the other schemes, GiBBS targets the true posterior, and each additional cycle brings the chain closer to it, at the cost of more compute. How many cycles to run is a trade-off left to practitioners, depending on the budget they are willing to spend to produce good posterior samples. Making the algorithm maximally parallel keeps this cost reasonable, and for long trajectories (N>2IN > 2I), GiBBS even requires fewer sequential denoiser calls than AR (l. 136–137).

Regarding your question, Figure 15 (Appendix D.2) now shows the CRPS as a function of the number of cycles on Kuramoto-Sivashinsky, with AR and AAO as references. Even from an uninformative initialization, a random trajectory from the dataset that comes at no cost, GiBBS overtakes AAO after 4 cycles and matches the CRPS of AR after only 8 cycles (l. 349–351). Since the trajectory spans N=36N = 36 blocks, 8 cycles amount to 16 sequential reverse diffusions, fewer than the 36 required by the AR rollout. With more cycles, the chain reaches the quality of the AR-initialized chain.

We hope that the revised manuscript and these answers address your concerns, and if so, we would be grateful if you would consider raising your score. We remain available for any further clarification.

Thank you for the additional experiments and the runtime analysis. I remain supportive of acceptance.

AuthorsOpenReview ↗

Response to Reviewer tkfc

Thank you for taking the time to respond to our rebuttal. Since no further concerns were raised, we would be grateful to know whether our responses have changed your assessment and whether you would consider updating your score. If there are remaining points that keep you from doing so, we would greatly appreciate any additional feedback so we can address them before the discussion period ends.

Thank you again for your engagement.

Reviewer_hKCmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GiBBS (Gibbs Block Blanket Sampling), a method for composing short-horizon conditional generative models to perform long-range data assimilation. The key idea is to partition a trajectory into blocks and iteratively resample each block conditioned on its Markov blanket and local observations. By adopting an odd-even chromatic Gibbs schedule, the method allows conditionally independent blocks to be updated in parallel. A periodic shift of block boundaries is further introduced to improve mixing.

Under exact conditional sampling and the assumed Markov structure, the method inherits the convergence properties of blocked Gibbs sampling. The authors instantiate GiBBS using diffusion models and evaluate it against autoregressive (AR) and all-at-once (AAO) sampling on Ornstein-Uhlenbeck, Lorenz-63, Kuramoto-Sivashinsky, and Kolmogorov flow systems.

The experiments demonstrate improved reconstruction accuracy and probabilistic forecasting quality, particularly under sparse observations. Additional ablations investigate initialization, block shifting, observation sparsity, and noise levels.

Critical strengths
  1. A principled formulation for long-range generative model composition.

The paper provides a conceptually clean solution to an important limitation of existing composition methods. Instead of committing to sequential predictions or approximating the global diffusion score, GiBBS formulates trajectory generation as conditional block resampling. The use of Markov blankets allows the method to incorporate information from both past and future observations, while chromatic updates preserve parallelism.

  1. Convincing results across several dynamical systems.

The experimental evaluation covers systems of increasing complexity, ranging from an analytically tractable Gaussian process to chaotic and high-dimensional dynamical systems. The Ornstein-Uhlenbeck experiments are particularly valuable because an exact posterior reference is available, enabling a direct examination of convergence.

The improvements on Kuramoto-Sivashinsky are substantial: GiBBS reduces CRPS from 0.11 (AR) and 0.16 (AAO) to 0.05. The ablations on initialization, block shifting, observation sparsity, and noise further help establish when and why the method is effective.

Critical weaknesses
  1. The computational cost and practical efficiency of GiBBS are insufficiently evaluated.

The main practical limitation is the absence of a systematic comparison of computational cost. GiBBS performs repeated full reverse-diffusion sampling for each block, with two sequential color updates per cycle. The learned-system experiments use 16 cycles, often following an additional AR or AAO initialization.

This can introduce a substantial computational overhead relative to the baselines. However, the paper does not report wall-clock inference time, sampling throughput, or quality-versus-compute comparisons. The asymptotic parallelism argument does not fully address this issue because the number of cycles required for convergence, available hardware parallelism, and diffusion sampling cost all affect practical performance.

This omission is particularly relevant because some improvements are relatively modest. For example, on Kolmogorov flow, CRPS improves from 0.93 for AAO to 0.84 for GiBBS. Without accounting for the additional inference cost, it is difficult to assess whether this improvement represents a favorable practical trade-off.

Since computational feasibility is an important motivation for using local models in long-range assimilation, this limitation significantly weakens the practical significance of the results.

  1. The posterior convergence guarantee does not directly establish correctness with learned conditional samplers.

The theoretical guarantee relies on exact conditional sampling and the Markov structure of the target distribution. The underlying convergence argument follows established results for blocked and chromatic Gibbs samplers, including Gonzalez et al. (2011).

In the practical experiments, however, block conditionals are approximated using learned diffusion models and approximate observation-conditioning procedures. These approximations need not correspond to mutually consistent conditionals of the true joint posterior. Consequently, the Gibbs convergence guarantee does not directly imply convergence to the desired assimilation posterior in these settings.

The Markov assumption also deserves additional justification for the spatially coarse-grained Kolmogorov system, where unresolved dynamics may introduce temporal memory.

Moreover, the evaluation primarily relies on skill, spread-skill ratio, and coordinate-wise CRPS. While informative, these metrics do not fully characterize the joint posterior distribution or its temporal dependencies. Thus, the strong empirical improvements do not necessarily establish that the learned sampler recovers the correct joint posterior.

A more careful separation between theoretical correctness under ideal conditions and empirical posterior approximation with learned models would strengthen the paper.

Questions for authors

Q1. Computational efficiency and fair comparison.

Could the authors provide wall-clock inference times and model evaluation counts for AR, AAO, and GiBBS, including initialization and all Gibbs cycles? In particular, how do these methods compare under a matched inference-time or computational budget? A quality-versus-runtime curve would substantially affect my assessment of the practical contribution.

Q2. Block partitioning and boundary handling.

Algorithm 1 defines the number of blocks as N=ceil (L/B), whereas Table 3 uses configurations satisfying L=N*B+2C. For example, Kuramoto-Sivashinsky has L=254, B=7, C=1, and N=36, although ceil(254/7)=37.

Could the authors clarify how the endpoint conditioning strips are handled, particularly when boundary states are not fully observed? Are all unobserved states updated under the shifted partitions? This is relevant to the claimed invariance of the target posterior.

Q3. Accuracy of the joint posterior with learned models.

Can the authors provide additional evidence that GiBBS preserves important temporal dependencies of the posterior, rather than only improving marginal reconstruction metrics? For example, a quantitative comparison against the bootstrap particle filter reference already used for Lorenz-63, or an analysis of temporal correlations, would help establish the practical validity of the Gibbs formulation.

Relatedly, how sensitive are the results to deviations from the assumed Markov structure, particularly for the coarse-grained Kolmogorov system?

3 comments
AuthorsOpenReview ↗

Response to Reviewer hKCm

We thank the reviewer for their thorough review and interest in our work. We believe the concerns raised are legitimate, and we have addressed them here and in the revised manuscript (changes in blue).

W1 & Q1. Computational efficiency and fair comparison

Following your suggestion, all results now report the runtime on a single NVIDIA RTX A5000 GPU and the number of function evaluations (NFEs), initialization included (Tables 1, 3 and 4). We also added a compute and time complexity analysis (Section 2, l. 122–137, and Table 2), and the CRPS as a function of the number of cycles on Kuramoto-Sivashinsky (Figure 15, Appendix D.2). From an uninformative initialization, a random trajectory from the dataset that comes at no cost, GiBBS matches the CRPS of AR after only 8 cycles (l. 345–351).

That said, we do not claim that GiBBS is faster than other composition methods. The perspective we find most relevant is that of quality, as a fast sampler that does not target the right distribution produces quick but wrong answers, which is of little use for data assimilation, where the posterior itself is the object of interest. This is precisely the limitation of existing composition methods. AR is blind to future observations by construction and drifts before being abruptly pulled back onto them, while AAO fails to propagate information away from the observation sites. GiBBS, in contrast, targets the true posterior, and each additional cycle brings the chain closer to it, at the cost of more compute. How many cycles to run is a trade-off left to practitioners, depending on the budget they are willing to spend to produce good posterior samples. Making the algorithm maximally parallel keeps this cost reasonable, and for long trajectories (N>2IN > 2I), GiBBS even requires fewer sequential denoiser calls than AR (l. 136–137).

We acknowledge that Kolmogorov flow is the least favorable case: the improvement over AAO (CRPS from 0.93 to 0.84) costs about 18 times more NFEs (Table 4), and lies mainly in the fine structure of the vortices, which AAO oversmooths (Figure 8).

Q2. Block partitioning and boundary handling

We thank the reviewer for spotting this inconsistency. Algorithm 1 oversimplified our implementation, and Table 5 (Table 3 in the submission) reflects what we ran. The trajectory is covered by N=⌈(L−2C)/B⌉N = \lceil (L-2C)/B \rceil windows of size WW, starting BB frames apart (for KS, N=252/7=36N = 252/7 = 36). Interior blocks are conditioned on CC states of each neighbor, and the two edge blocks, having a single neighbor, on that neighbor only. The shifted grid pads the trajectory to fit: padding frames carry no observations, are drawn jointly with the edge blocks and discarded, which marginalizes them out. On both grids, the free blocks tile x1:Lx_{1:L}, so every state is redrawn exactly once per cycle from its exact conditional, which leaves p(x1:L∣y)p(x_{1:L} \mid y) invariant. We corrected Algorithm 1 accordingly (l. 154).

W2 & Q3. Accuracy of the joint posterior with learned models

We addressed your concern about convergence with learned rather than analytical denoisers in Figure 11 (Appendix C.1, l. 879). We do not claim that GiBBS removes the approximation errors of the neural networks used to learn the denoisers: these networks are imperfect, and GiBBS inherits their errors like any other sampler. However, among the composition schemes we consider, GiBBS is the only one that converges to the true posterior when its conditionals are exact. All methods share the same denoiser, so that no strategy is favored over another, and they therefore inherit the same approximation errors. To quantify these errors, we trained a masked denoiser on Ornstein-Uhlenbeck (a small residual MLP) and compared analytical and learned denoisers for all strategies. Observations are incorporated architecturally, so that the error of the denoiser is isolated from that of posterior sampling. GiBBS still converges to the exact posterior with the learned denoiser (Figure 11B).

Regarding temporal dependencies, on Lorenz-63, we compare all methods against the bootstrap particle filter reference (Table 10, Appendix D.1). GiBBS reduces the W1W_1 distance to 13.7, against 20.6 for AAO and 29.6 for AR, the remaining gap combining model and composition errors since the BPF uses the true equations.

On Kolmogorov flow, we trained an additional denoiser on windows of W=7W = 7 frames, allowing C=2C = 2 (Table 12, Appendix D.3). A wider strip does not improve skill or CRPS, which indicates that the memory introduced by the coarse-graining is weak beyond one frame. This validates that the first order Markov assumption or the used Markov blanket size is a reasonable assumption in this case.

We believe our responses address your main concerns, and would be grateful if you could reconsider your score in light of our revisions. We remain available for any further clarification.

I thank the authors for their detailed response and the additional runtime and NFE results. However, without a matched-budget comparison, it remains unclear whether the accuracy gains justify the substantial additional computational cost. For now, I will maintain my score of 2 (weak rejection), but remain open to reconsidering it in light of further clarification.

AuthorsOpenReview ↗

Response to Reviewer hKCm (Part 2)

We thank the reviewer for engaging with our response. To address the matched-budget question directly, we added a quality-versus-compute comparison on Kuramoto-Sivashinsky (Figure 16, Appendix D.2, l. 1089–1096). The CRPS and spread-skill ratio of each method are plotted against the number of NFEs per member. AAO spends a larger budget on more Langevin corrections per diffusion step, while GiBBS starts from the AR pass and increases its budget on more cycles.

At every budget, GiBBS reaches a lower CRPS than AAO and also improves on AR from its very first cycles. At ≈0.5\approx 0.5M NFEs, GiBBS reaches ≈0.051\approx 0.051 against ≈0.067\approx 0.067 for AAO. Even with four times more compute (≈2\approx 2M NFEs) AAO still remains above GiBBS. The spread-skill ratio tells the same story since GiBBS reaches a calibrated ensemble (SSR →1\rightarrow 1) while AAO remains overdispersed whatever its budget.

This is expected from theory. In this work, we argue that two existing assimilation approaches, AR and AAO, are not capable of generating from the true posterior distribution, regardless of the computational budget allocated to them. Furthermore, these approaches, especially AR, do not have a tunable parameter which balances quality and cost. Matching compute budgets is therefore an unfair criterion for comparison in this setting.

GiBBS, in contrast, converges to the true posterior given enough cycles. The number of cycles lets practitioners convert additional compute into better posterior samples. You could think of this as similar to the reasoning steps of a large language model, which spend an adjustable inference budget for better answers, while the same model without them cannot make use of extra compute.

We believe this addresses your remaining concern, and would be grateful if you would reconsider your score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.