Sequential Data Assimilation using Rolling Diffusion Models with Editable History
Abstract
Data assimilation (DA) combines dynamical models with observations to estimate the hidden states of complex systems. Classical methods such as the ensemble Kalman filter struggle under highly non-Gaussian dynamics, while existing diffusion-based approaches are either fixed-window smoothers or bias-prone online filters. We introduce Rolling Diffusion Data Assimilation (RDA), which uses rolling diffusion models for forecasting, and assimilating data sequentially by guiding the denoising process with observations. Additionally, we propose a new rolling diffusion architecture with editable history blocks that can revise past frames as new observations arrive. For an idealized version with exact conditional denoising, our analysis identifies the limiting window posterior and bounds its discrepancy from the full filtering distribution under strong mixing. Across a range of nonlinear dynamical systems, from low-dimensional chaotic dynamics to high-dimensional geophysical settings, RDA matches or outperforms both classical and ML-based filtering methods, especially under sparse and nonlinear observations.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.