A Novel Sequential Data Assimilation by Conditional Denoising Score Matching

This study introduces a novel sequential data assimilation method that uses conditional denoising score matching (CDSM). The CDSM leverages iterative refinement of noisy samples guided by conditional score functions to achieve real-time state estimation by incorporating observational constraints at each time step. Unlike traditional methods, such as variational assimilation and Kalman ffltering, which rely on Gaussian assumptions and can be computationally expensive because of iterations or ensembles, CDSM is based on stochastic differential equations (SDEs). It does not require explicit noise addition or manipulation of probability density functions, thus simplifying the assimilation process and enhancing the computational efficiency. Here, error growth and reduction were modeled using noise addition and denoising processes based on SDEs. This transforms the data assimilation problem into a denoising problem based on conditional score matching. Our approach integrates dynamic models, performs data assimilation through Langevin dynamics at the observation times, and uses the analyzed states for subsequent integration. The noise addition process is embedded in the score model training using neural networks and is not explicitly used in the assimilation process. The results from twin experiments using the Lorenz ‘63 model demonstrate that the CDSM achieves a performance comparable to that of traditional methods in nonlinear systems. This method is robust and flexible with low requirements for training data quality. This is particularly suitable for scenarios in which the observation intervals are much larger than the model integration steps. The CDSM shows great potential for application inlarge-scale numerical and data-driven models.