NP5.2

The idea of using machine learning (ML) methods to reconstruct the dynamics of a system is the topic of recent studies in the geosciences in which the key output is a surrogate model meant to emulate the dynamical model. In order to treat sparse and noisy observations in a rigorous way, ML can be combined with data assimilation (DA). This yields a class of iterative methods in which, at each iteration a DA step estimates the system's state, and alternates with a ML step to learn the system's dynamics from the DA analysis.

This framework can be used to correct the error of an existent, physical model. The resulting surrogate model is hybrid, with a physical and a statistical part. In practice, the correction can be added as an integrated term (i.e. in the model resolvent) or directly inside the tendencies of the physical model. The resolvent correction is easy to implement but is not suited for short-term predictions. The tendency correction is more technical since it requires the adjoint of the physical model, but also more flexible and can be used for any forecast lead time.

In this presentation, we start by a proof of concept for the use of joint DA and ML tools to correct model error. We use the resolvent correction with simple neural networks to correct the error of a two-dimensional, two layer quasi-geostrophic layer. The difference between the resolvent and the tendency correction is then illustrated with the two- scale Lorenz model. Finally, we show that the tendency correction opens the possibility to make online model error correction, i.e. improving the model progressively as new observations become available. We compare online and offline learning using the same twin experiment with the two-scale Lorenz model.

How to cite: Farchi, A., Bocquet, M., Laloyaux, P., Bonavita, M., Chrust, M., and Malartic, Q.: Model error correction with data assimilation and machine learning, EGU General Assembly 2022, Vienna, Austria, 23–27 May 2022, EGU22-5692, https://doi.org/10.5194/egusphere-egu22-5692, 2022.

Geoenergies such as underground gas storage and energy storage, geothermal energy and geologic carbon storage are key technologies on the way to the foreseeable energy transition. The reservoir characterization in these projects remains challenging since predictive modeling approaches face limitations in identifying the spatial distribution of distinct lithologies and their hydro-mechanical properties from downwell testing procedures. Pumping tests are usually carried out to infer permeability, but offer only few observation points in space and require large extrapolation through inversion. The application of Physics Informed Neural Networks (PINN) offers a promising solution which can seamlessly incorporate field data, while enforcing the accordance with physical laws in the domain of study. This concept is implemented via two distinct loss terms for both the physical constraints and for the observational data in the loss function of an Artificial Neural Network (ANN). The physics-informed loss term contains a mass balance equation consisting of a storage and a diffusion component. The process is considered to be purely hydraulic and Darcy flow is assumed. The observational loss term compares the output of the ANN to a set of training data. This set consists of  the system’s initial fluid pressure, as well as of a fluid pressure time series at the domain boundaries and at the borehole location. Preliminary results suggest that our PINN model is able to forecast the spatiotemporal fluid pressure distribution in a 2D domain for a variety of pumping test schemes. In this way, we give a first impression of the opportunities that PINN applications offer in the field of reservoir modeling.

How to cite: Walter, L., Parisio, F., and Vilarrasa, V.: Prediction of Subsurface Fluid Flow via Physics Informed Neural Networks, EGU General Assembly 2022, Vienna, Austria, 23–27 May 2022, EGU22-12361, https://doi.org/10.5194/egusphere-egu22-12361, 2022.

Ensemble-variational methods form the basis of the state-of-the-art for nonlinear, scalable data assimilation, yet current designs may not be cost-effective for reducing prediction error in online, short-range forecast systems. We propose a novel, outer-loop optimization of the Bayesian maximum a posteriori formalism for ensemble-variational smoothing in applications for which the forecast error dynamics are weakly nonlinear, such as synoptic meteorology. In addition to providing a rigorous mathematical derivation our technique, we systematically develop and inter-compare a variety of ensemble-variational schemes in the Lorenz-96 model using the open-source Julia package DataAssimilationBenchmarks.jl. This high-performance numerical framework, supporting our mathematical results, produces extensive benchmarks that demonstrate the significant performance advantages of our proposed technique versus several similar estimator designs. In particular, our single-iteration ensemble Kalman smoother (SIEnKS) is shown both to improve prediction / posterior accuracy and to simultaneously reduce the leading order cost of iterative, sequential smoothers in a variety of relevant test cases for operational short-range forecasts.  These results are currently in open review in Geoscientific Model Development (Preprint gmd-2021-306) and the Journal of Open Source Software (Preprint #3976).

How to cite: Grudzien, C. and Bocquet, M.: A fast, single-iteration ensemble Kalman smoother for sequential data assimilation, EGU General Assembly 2022, Vienna, Austria, 23–27 May 2022, EGU22-1218, https://doi.org/10.5194/egusphere-egu22-1218, 2022.

This study aims at proposing a new framework to perform ensemble-based estimations of dynamical trajectories of a geophysical fluid flow system. To perform efficient estimations, the ensemble members are embedded in a set of evolving reproducing kernel Hilbert spaces (RKHS) defining a manifold of spaces, we nicknamed Wonderland, due to its analytical properties.

The method proposed here is designed to deal with very large scale systems such as oceanic or meteorological flows, where it is out of the question to explore the whole attractor, neither to run very long time simulations. Instead, we propose to learn the system locally, in phase space, from an ensemble of trajectories.

The novelty of the present work relies on the fact that the feature maps between the native space and the RKHS manifold are transported by the dynamical system. This creates, at any time, an isometry between the tangent RKHS at time t and the initial conditions. This has several important consequences. First, the kernel evaluations are constant along trajectories, instead to be attached to a system state. By doing so, a new ensemble member embedded in the RKHS manifold at the initial time can be very simply estimated at a further time. This framework displays striking properties. The Koopman and Perron-Frobenius operators on such RKHS manifold are unitary, even though the system might be non invertible. They are furthermore uniformly continuous (with bounded generators) and diagonalizable. As such they can be rigourously expended in exponential forms. 

This set of analytical properties enables us to provide a practical estimation of the Koopman eigenfunctions. In the proposed strategy, evaluations of these Koopman eigenfunctions at the ensemble members are exact. To perform robust estimations, the finite-time Lyapunov exponents associated with each Koopman eigenfunction (which are easily accessible on the RKHS manifold as well) are determined. On this basis, we are able to filter the kernel by removing contributions of the Koopman modes that exceed the predictability time. We show that it leads to robust estimations of new unknown trajectories. This framework allows us to write an ensemble-based data assimilation problem, where constant-in-time linear combinations coefficients between ensemble members are sought in order to estimate the QG flow based on noisy swath observations.

The methodology is demonstrated on a barotropic quasi-geostrophic model of a double gyres. After comparing various kernels and provided guidelines to adapt the kernel with the spread of the ensemble, we show isometry and Koopman-filtered reconstructions. Finally, the data assimilation is presented.

How to cite: Tissot, G., Mémin, E., and Hug, B.: Koopman eigenfunctions estimation from reproducing kernel Hilbert space manifold, and ensemble data assimilation, EGU General Assembly 2022, Vienna, Austria, 23–27 May 2022, EGU22-2604, https://doi.org/10.5194/egusphere-egu22-2604, 2022.

In ensemble variational (EnVar) data assimilation systems, background error covariances are sampled from an ensemble of forecasts evolving with time. One possible way of generating this ensemble is by running an Ensemble of Data Assimilations (EDA) that samples all possible error sources (initial condition errors, boundary condition errors, model errors). Large ensemble sizes are desirable to minimize sampling errors, but generating a single ensemble member is usually expensive due to the cost of integrating the physical model. In practice, ensembles with coarser spatial resolutions are sometimes used, allowing for cheaper generation of individual members, and thus larger ensemble sizes.

Multilevel Monte Carlo (MLMC) methods propose to go beyond this usual trade-off between grid resolution and ensemble size, by expressing a fine-grid estimator as an astute combination of estimators computed on a hierarchy of spatial grids. Starting from a Monte Carlo covariance estimator on a coarse grid but with a large ensemble size, correction terms are added to form a quasi-telescopic sum. The correction terms come from EDAs of increasing spatial resolutions and decreasing ensemble sizes, with a pairwise stochastic coupling between EDAs of two successive resolutions. The expectation of this MLMC estimator is equal to the expectation of the Monte Carlo estimator on the finest grid, so that no bias is introduced by the coarse resolution forecasts. Without increasing the computational cost, MLMC effectively reduces the variance of the covariance estimator, i.e. reduces the sampling noise on covariances.

We first present the theoretical basis of MLMC and how it can apply to the estimation of covariance matrices. An illustration with a quasi-geostrophic model is then presented. For a given computational budget, we compare three equal-cost methods to estimate background error covariances: (1) the usual single-resolution ensemble estimate, (2) a combination of estimates of various resolutions based on Bayesian Model Averaging and (3) the MLMC estimate. The methods are compared in terms of mean square error of the covariance estimators, and in terms of quality of the resulting analyses for one assimilation cycle. The role of covariance localization in each case is also briefly discussed.

This work is partially supported by 3IA Artificial and Natural Intelligence Toulouse Institute, French "Investing for the Future- PIA3" program under the Grant agreement ANR-19-PI3A-0004.

This project has received financial support from the CNRS through the 80Prime program.

How to cite: Destouches, M., Mycek, P., Briant, J., Gürol, S., Weaver, A., Gratton, S., and Simon, E.: Multilevel Monte Carlo estimation of background error covariances in ensemble variational data assimilation, EGU General Assembly 2022, Vienna, Austria, 23–27 May 2022, EGU22-336, https://doi.org/10.5194/egusphere-egu22-336, 2022.

This note examines the relationship between what at first sight looks like two unrelated methods for estimating second order statistics of relevance to data assimilation. The first method is due to Desroziers et al. (2005) and relies on residual statistics readily available from data assimilation applications. The second method, due to Gray and Allan (1974), only recently making its appearance in atmospheric sciences, is generally formulated to use three data sets and seems in principle capable of deriving estimates of observation, background and analysis just as well. The usefulness of either method lies in them not requiring knowledge of the true value of the quantities at play. Desroziers derives its results by relying explicitly on the constraints associated with the data assimilation minimization problem; the 3CH method is general and its estimates hold as long as random errors in the three data sets of choice are independent. Establishing the relationship between the methods amounts to identifying the data sets of 3CH with be the observation, background, and analysis associated with Desroziers. The choice of observation and background for two of the data sets of 3CH is acceptable under the typical assumption of independence in their errors. Specifying the third data set of 3CH as the analysis seems unreasonable for analysis errors are by construction dependent on errors in both observations and background. This note finds that when the assumption of optimality required of Desroziers is applied to 3CH the latter method recovers the Desroziers error estimates for observation and background. More interestingly, in contrast with Desroziers estimate of errors in the analysis, the remaining corner of 3CH obtains the negative of the analysis error variance. An illustration of this finding is provided by deriving various uncertainties in bending angle.

How to cite: Todling, R., Semane, N., Anthes, R., and Healy, S.: The Relationship between Desroziers and Three-Cornered Hat Methods, EGU General Assembly 2022, Vienna, Austria, 23–27 May 2022, EGU22-8918, https://doi.org/10.5194/egusphere-egu22-8918, 2022.

How to cite: Pein, A. and Van Leeuwen, P. J.: Model error covariance estimation in observation space weak-constraint 4DVar, EGU General Assembly 2022, Vienna, Austria, 23–27 May 2022, EGU22-7448, https://doi.org/10.5194/egusphere-egu22-7448, 2022.

In this work we show how it is possible to derive a new set of nudging equations, a tool still used in many data assimilation problems, starting from statistical physics considerations and availing ourselves of stochastic parameterizations that take into account unresolved interactions. The fluctuations used are thought of as Gaussian white noise with zero mean. The derivation is based on the conditioned Langevin dynamics technique. Exploiting the relation between the Fokker–Planck and the Langevin equations, the nudging equations are derived for a maximally observed system that converges towards the observations in finite time. The new nudging term found is the analog of the so called quantum potential of the Bohmian mechanics. In order to make the new nudging equations feasible for practical computations, two approximations are developed and used as bases from which extending this tool to non-perfectly observed systems. By means of a physical framework, in the zero noise limit, all the physical nudging parameters are fixed by the model under study and there is no need to tune other free ad-hoc variables. The limit of zero noise shows that also for the classical nudging equations it is necessary to use dynamical information to correct the typical relaxation term. A comparison of these approximations with a 3DVar scheme, that use a conjugate gradient minimization, is then shown in a series of four twin experiments that exploit low order chaotic models.

How to cite: Conti, G., Aydoğdu, A., Gualdi, S., Navarra, A., and Tribbia, J.: On the physical nudging equations, EGU General Assembly 2022, Vienna, Austria, 23–27 May 2022, EGU22-1164, https://doi.org/10.5194/egusphere-egu22-1164, 2022.

Extensive numerical evidence for real and/or simulated data shows that the assimilation of observations has a stabilizing effect on unstable dynamics in numerical weather prediction and elsewhere.  In this talk, I will discuss mathematically rigorous considerations showing why this is so. In particular we prove that the expected value of the Wasserstein distance between the forecast-assimilation (FA) process starting from the true initial conditions and FA process wrongly initialized can be controlled uniformly in time. Under suitable circumstances, the number of observations required to achieve this stabilization can be much smaller than the number of model variables. In particular, it suffices to observe the model's unstable degrees of freedom. 

How to cite: Crisan, D. and Ghil, M.: Asymptotic behavior of the forecast-assimilation process with unstable dynamics, EGU General Assembly 2022, Vienna, Austria, 23–27 May 2022, EGU22-8659, https://doi.org/10.5194/egusphere-egu22-8659, 2022.

Data assimilation (DA) aims at optimally merging observational data and model outputs to create a coherent statistical and dynamical picture of the system under investigation. Indeed, DA aims at minimizing the effect of observational and model error and at distilling the correct ingredients of its dynamics. DA is of critical importance for the analysis of systems featuring sensitive dependence on the initial conditions, as chaos wins over any finitely accurate knowledge of the state of the system, even in absence of model error. Clearly, the skill of DA is guided by the properties of dynamical system under investigation, as merging optimally observational data and model outputs is harder when strong instabilities are present. In this paper we reverse the usual angle on the problem and show that it is indeed possible to use the skill of DA to infer some basic properties of the tangent space of the system, which may be hard to compute in very high-dimensional systems. Here, we focus our attention on the first Lyapunov exponent and the Kolmogorov–Sinai entropy and perform numerical experiments on the Vissio–Lucarini 2020 model, a recently proposed generalization of the Lorenz 1996 model that is able to describe in a simple yet meaningful way the interplay between dynamical and thermodynamical variables.

How to cite: Chen, Y., Carrassi, A., and Lucarini, V.: Inferring the instability of a dynamical system from the skill of data assimilation exercises, EGU General Assembly 2022, Vienna, Austria, 23–27 May 2022, EGU22-1664, https://doi.org/10.5194/egusphere-egu22-1664, 2022.

The purpose of employing data assimilation methods in operational ocean forecasting systems is to provide good initialization to the models and is dependent on good quality ocean observations being assimilated. Using or accepting erroneous data can result in an inaccurate analysis and alternatively, rejecting extreme or valid data can result in missing important events.

In this study two ocean-sea ice coupled systems are considered: HYCOM-CICE4 and MOM6-CICE6 at ¼-deg horizontal resolution and 41 vertical layers. The two coupled models are initialized from the World Ocean Atlas 2018 (WOA) temperature and salinity climatology for a period of 20 years. Both models are forced with GEFS (Global Ensemble Forecast System created by the National Centers for Environmental Prediction: NCEP). The data assimilation is performed on a 24-hr cycle using RTOFS-DA (Real-time Ocean Forecasting system-DA; 3DVAR) for HYCOM-CICE4 and SOCA (Sea ice Ocean Coupled Assimilation; 3DVAR) for MOM6-CICE6 to compare the data Quality Control (QC) methods. The ocean data being assimilated include satellite sea surface temperature (SST) and sea surface salinity (SSS), in-situ temperature & salinity, absolute dynamic topography (ADT), sea ice concentration.

The QC in RTOFS-DA and SOCA are fully automated and are performed through various filters applied (e.g., land-sea area fraction to eliminate satellite data near the coast, temperature inversion elimination in in-situ profile data, etc). The various QC methods in both DA systems are described. The results of the analysis and 24-forecast are compared against independent observations and statistics of the data accepted and rejected between the two DA systems are presented and discussed.

How to cite: Paturi, S., Bozec, A., Chassignet, E., Garraffo, Z., Mehra, A., and Kleist, D.: Quality Control Methods in Ocean-Sea ice Coupled Data Assimilation, EGU General Assembly 2022, Vienna, Austria, 23–27 May 2022, EGU22-2698, https://doi.org/10.5194/egusphere-egu22-2698, 2022.

The multi-model ensemble approach is applied in geosciences to provide better predictions or projections, by weighting the outputs from different dynamical models. Basically, the weighting procedure relies on the choice of a performance metric to measure the closeness of individual model outputs to actual observations. The highest weight is then given to the model that best matches the observations, and so forth. Model weights can be used to constrain both the mean and the uncertainty in future projections of climate models.

In this study, we seek to combine different parameterizations of an idealized three-dimensional chaotic model of the Atlantic Meridional Overturning Circulation. One of the parameterizations plays the role of the observations. Each parameterization is evaluated online in a data assimilation framework using the EnKF by comparing the forecasts with the observations.

Traditional data assimilation procedures require access to the model equations, resulting in significant computational costs to run multiple model simulations to obtain forecasts at each time step. Here, a machine learning approach is implemented to provide the forecasts (i.e., analog forecasting). For each parameterization, the classical way of producing the forecasts is, in our case, replaced by an already existing catalog of trajectory time evolutions (e.g., long-term simulations), allowing to statistically emulate the model dynamics. This data-driven methodology retains the benefits given by the classical EnKF (i.e., optimal initial conditions, uncertainties consideration), at low computational costs. For each model-parameterization, a local performance metric (namely, the contextual model evidence) is computed at each time step in order to compare observations and model forecasts. This metric, based on the innovation likelihood, is sensitive to differences in the model dynamics and takes into account both the uncertainties of the forecasts and of the observations. To validate the methodology, different case studies are performed with various sensitivity tests (e.g., changing the parameterization used for the observations).

The results of the proposed weighting scheme on projections are discussed considering different quality metrics compared to benchmark methodologies. These include the equally weighting approach (also called the “model democracy”) and the direct comparison between the climatological probability distributions of simulations and observations.

How to cite: Le Bras, P., Ailliot, P., Le Carrer, N., Ruiz, J., Sévellec, F., and Tandeo, P.: Data-driven data assimilation to better characterize both accuracy and uncertainty of climate projections: a case study with an idealized chaotic AMOC model, EGU General Assembly 2022, Vienna, Austria, 23–27 May 2022, EGU22-8578, https://doi.org/10.5194/egusphere-egu22-8578, 2022.

Assessing pesticide transfers and fate in agricultural catchments is a major challenge to protect water ressources and aquatic organisms. To do so, physically-based, spatialized hydrological models are useful tools as they can be used to set up relevant mitigation strategies. The PESHMELBA model (Rouzies et al. 2019) is one such model that focuses on accurately simulating water and pesticide transfers both in the surface and the subsurface compartments of the soil. This model also aims at explicitely integrating and assessing the impact of landscape structures such as hedges, vegetative filter strips or ditches on transfers. To do so, the PESHMELBA model is characterized by a highly modular structure that relies on various code units standing for different physical processes in the different soil compartments. Such code units are thus coupled in a dedicated framework to reach a complete representation of the catchment with interacting processes. The resulting structure is quite complex and leads to significant difficulties to quantify and reduce the uncertainties associated to the simulation outputs.

In this study, we aim at setting a relevant data assimilation framework to reduce the uncertainty into the PESHMELBA coupled surface / subsurface water flow and reactive solute transport model. To do so, we test several data assimilation methods on hydrological and pesticide variables describing the catchment behavior. At first, these methods are implemented by combining the PESHMELBA model and surface moisture satellite images, at the small catchment scale. Different filtering and smoothing stochastic assimilation methods are explored: the Ensemble Kalman Filter, the Ensemble Smoother with Multiple Data Assimilation and the iterative Ensemble Kalman Smoother. Their abilities to retrieve moisture and pesticide concentration in the observed surface compartment but also in the deeper soil, that is not observed, are assessed. Furthermore, the conducted experiments also aim at retrieving some input parameters that characterize such different soil compartments.

Preliminary results on this part show that all tested methods only succeed in retrieving surface moisture. The Ensemble Smoother is shown to particulary outperform the other methods as it fully integrates the system dynamics. However, its performances are much more limited to retrieve moisture and input parameters in the deeper compartment due to poor correlations between the surface and the subsurface compartments. To overcome such limitation, other sources of data are gradually integrated in the DA framework. The process is proven successfull and we explore how the corrections from the DA process can propagate to other compartments such as the river streamflow and pesticide related variables .

How to cite: Rouzies, E., Lauvernet, C., and Vidard, A.: Which data assimilation method and data source for a multi-compartment hydrology/water quality model? Application on the PESHMELBA model in a small agricultural catchment, EGU General Assembly 2022, Vienna, Austria, 23–27 May 2022, EGU22-10384, https://doi.org/10.5194/egusphere-egu22-10384, 2022.

The Cosmic-Ray Neutron Sensing (CRNS) technology determines soil moisture for a few tens of hectares in a non-invasive way. These measurements, however, can be used to extend soil moisture characterization at regional scales using data assimilation. In the present study, we deployed the Ensemble Adjustment Kalman Filter (EAKF) to assimilate the CRNS neutron counts in order to update the spatial soil moisture, soil infiltration, and evapotranspiration parameters of the Noah-MP land surface model witch is also part of the WRF-Hydro modelling system. The study was conducted in the southern part of Germany, which includes the Rott and Ammer catchments within the TERENO Pre-Alpine observatory. The assimilation was carried out for both, a Noah-MP standalone scenario with observed rainfall as input and a coupled WRF-Hydro scenario with simulated rainfall to fully evaluate the added value of the assimilation. The assimilation performance was analysed at local and regional scale using independent soil moisture observations across the modelling domain. During the assimilation period, the Noah-MP standalone findings demonstrate a significant improvement in field scale soil moisture characterisation. The RMSE of simulated soil moisture was decreased by up to 66 % at field scale and up to 23 % at catchment scale. Additionally, the spatial patterns in the field scale soil moisture have showed improvement with reduction in spatial Bias by 0.025 cm³/cm³. The initial results from coupled WRF-Hydro scenario demonstrate that the soil moisture and parameter estimation experiment had a significant impact on estimated soil moisture and, humidity and evapotranspiration at regional scale. These findings support the use of the CRNS technique to improve the land surface and coupled hydro-atmospheric modelling.

How to cite: Patil, A., Fersch, B., Hendricks Franssen, H.-J., and Kunstmann, H.: Improved soil moisture-atmospheric boundary layer interactions by assimilation of Cosmic-Ray Neutron counts, EGU General Assembly 2022, Vienna, Austria, 23–27 May 2022, EGU22-10717, https://doi.org/10.5194/egusphere-egu22-10717, 2022.