NP5.1

Hybrid error covariance models construct the covariance matrix to be used in variational data assimilation methods through a linear combination P^h= α_cP^c + α_eP^l of the climatological error covariance matrix P^c and the localized ensemble covariance matrix P^l = C⊗P with scalar weights α_c and α_e.

This work aims to provide a theoretical justication for current hybrid error covariance models and identify a critical issue in them in order to improve them in future research. In the framework of Bayes' theorem, a theory is developed by modelling the climatological distribution of true forecast error covariance matrix P^f as an inverse matrix gamma distribution (prior distribution) and the distribution of the localized ensemble covariance matrix P^l given a true forecast error covariance matrix P^f as a Wishart or matrix gamma distribution (likelihood distribution). The following formulas for the expected values of the prior and likelihood distributions are assumed: E [P^f ] = P^c and E [P^l P^f ]= P^f , respectively. The posterior distribution for the true forecast error covariance matrix P^f given the localised covariance matrix P^l is derived: it turns out to be an inverse matrix gamma distribution. Within this theory, a formula for the expected value E [P^fP ] of the true forecast error covariance matrix P^f given the ensemble covariance matrix P is derived: E [P^fP]= β_cP^c+ β_eP^l (where β_c  and β_e are scalar weights). This provides a theoretical justication for hybrid error covariance models. Moreover, expressions (and thus an interpretation) for the scalar weights β_c and β_e in terms of the relative variances of the diagonal elements of the prior and likelihood distributions are obtained.

Hence, the consistency of current hybrid covariance models with the assumption E [P^l P^f ]= P^f is showed. This assumption is, in turn, inconsistent with E [PP^f]= P^f , which ensemble DA schemes are meant to satisfy, and it is falsiable.

To illustrate the above theory, an experiment is run to simulate 3200 replicate Earth's all having the same true state trajectory, weather prediction system and observational network, but different realizations of the observations. Each replicate Earth is simulated through a 10-variable Lorenz '96 model with an ETKF data assimilation system. From the set of the true forecast errors of all replicate Earth's, the (otherwise hidden) true forecast error covariance matrix P^f is computed at each time step and the (dis)similarity of its climatological distribution from the best-fit inverse matrix gamma distribution is considered. It is found that (i) the inverse-matrix gamma distribution overestimates the probability of signicant error correlations between widely separated model variables; (ii) it is the un-localized ETKF ensemble covariance matrix that equals the mean climatological covariance matrix, not the localized ensemble covariance matrix. These findings motivate research to discover more accurate approximations to the climatological distribution of the true forecast error covariance matrix and more accurate hybrid covariance models.

How to cite: Sardelli, F. and Bishop, C.: Insights about the hybrid error covariance models, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-13764, https://doi.org/10.5194/egusphere-egu21-13764, 2021.

The second-order exact particle filter NETF (nonlinear ensemble transform filter) is combined with local ensemble transform Kalman filter (LETKF) to build a hybrid filter scheme (LKNETF). The filter combines the stability of the LETKF with the nonlinear properties of the NETF to obtain improved assimilation results for smaller ensembles. Both filter components are localized in a consistent way so that the filter can be applied with high-dimensional models. The degree of filter nonlinearity is defined by a hybrid weight, which shifts the analysis between the LETKF and NETF. Since the NETF is more sensitive to sampling errors than the LETKF, the latter filter should be preferred in linear cases. It is discussed how an adaptive hybrid weight can be defined based on the nonlinearity of the system so that the adaptivity yields a good filter performance in linear and nonlinear situations. The filter behavior is exemplified based on experiments with the chaotic Lorenz-63 and Lorenz-96 models, in which the nonlinearity can be controlled by the length of the forecast phase.

How to cite: Nerger, L.: Ensemble data assimilation for systems with different degrees of nonlinearity with a hybrid nonlinear-Kalman ensemble transform filter, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-2463, https://doi.org/10.5194/egusphere-egu21-2463, 2021.

Satellite observations have been a growing source for data assimilation in the operational numerical weather prediction. Remotely sensed observations require a nonlinear observation operator.  Most ensemble-based data assimilation methods are formulated for tangent linear observation operators, which are often substituted by nonlinear observation operators. By contrast, the Maximum Likelihood Ensemble Filter (MLEF), which has features of both variational and ensemble approaches, is formulated for linear and nonlinear operators in an identical form and can use non-differentiable observation operators. 

In this study, we investigate the performance of MLEF and Ensemble Transform Kalman Filter (ETKF) with the tangent linear and nonlinear observation operators in assimilation experiments of nonlinear observations with a one-dimensional Burgers model.

The ETKF analysis with the nonlinear operator diverges when the observation error is small due to unrealistically large increments associated with the high order observation terms. The filter divergence can be avoided by localization of the extent of observation influence, but the analysis error is still larger than that of MLEF. In contrast, MLEF is found to be more stable and accurate without localization owing to the minimization of the cost function. Notably, MLEF can make an accurate analysis solution even without covariance inflation, eliminating the labor of parameter adjustment. In addition, the smaller observation error is, or the stronger observation nonlinearity is, MLEF with the nonlinear operators can assimilate observations more effectively than MLEF with the tangent linear operators. This result indicates that MLEF can incorporate nonlinear effects and evaluate the observation term in the cost function appropriately. These encouraging results imply that MLEF is suitable for assimilation of satellite observations with high nonlinearity.

How to cite: Nakashita, S. and Enomoto, T.: Assimilation of Nonlinear Observations with the Maximum Likelihood Ensemble Filter, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-10591, https://doi.org/10.5194/egusphere-egu21-10591, 2021.

A homotopy schedule is proposed, wherein from a known probability distribution the normalization constant for an improper probability density function can be found. An improper distribution is one for which the normalization is not known, but its functional form is. In the statistical mechanics constant this amounts to finding the canonical ensemble for the improper distribution. Along the way, the method will generate samples from the target distribution.

This homotopy schedule can be adopted to particle filters used for Bayesian estimation with the aim of improving estimates of the mean path and the uncertainty of a noisy dynamical system, for which noisy observations are available. The method is useful when the dynamics are highly nonlinear, especially if the observations that inform the likelihood have low uncertainty. In the context of data assimilation we require that the stochastic dynamics of the system have an asymptotic stationary distribution, which we use as a the known distribution in the homotopy procedure.

In this talk we present the methodology, apply it to the estimation of canonical ensembles and present numerical comparisons of the standard particle filter estimates with those of the homotopy data assimilation. 

How to cite: Restrepo, J. and Ramirez, J.: Homotopy Particle Filter and Data Assimilation, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-129, https://doi.org/10.5194/egusphere-egu21-129, 2021.

There is growing awareness that errors in the model equations cannot be ignored in data assimilation methods such as four-dimensional variational assimilation (4D-Var). If allowed for, more information can be extracted from observations, longer time windows are possible, and the minimization process is easier, at least in principle. Weak constraint 4D-Var estimates the model error and minimizes a series of linear least-squares cost functions using the conjugate gradient (CG) method; minimising each cost function is called an inner loop. CG needs preconditioning to improve its performance. In previous work, limited memory preconditioners (LMPs) have been constructed using approximations of the eigenvalues and eigenvectors of the Hessian in the previous inner loop. If the Hessian changes signicantly in consecutive inner loops, the LMP may be of limited usefulness. To circumvent this, we propose using randomised methods for low rank eigenvalue decomposition and use these approximations to cheaply construct LMPs using information from the current inner loop. Three randomised methods are compared. Numerical experiments in idealized systems show that the resulting LMPs perform better than the existing LMPs. Using these methods may allow more efficient and robust implementations of incremental weak constraint 4D-Var.

How to cite: Dauzickaite, I., Lawless, A., Scott, J., and van Leeuwen, P. J.: Randomised preconditioning for the forcing formulation of weak constraint 4D-Var, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-4414, https://doi.org/10.5194/egusphere-egu21-4414, 2021.

Air quality monitoring systems vary in temporal and spatial coverage, the composition of the observed chemicals, and the data's accuracy. The developed inverse modeling approach [1] is based on sensitivity operators and ensembles of adjoint equations solutions. An inverse problem is transformed to a quasi-linear operator equation with the sensitivity operator. The sensitivity operator is composed of the sensitivity functions, which are evaluated on the adjoint ensemble members. The members correspond to the measurement data elements. 

This ensemble construction allows working in a unified way with heterogeneous measurement data in a single operator equation. The quasi-linear structure of the resulting operator equation allows both solving and analyzing the inverse problem. More specifically, by analyzing the sensitivity operator's singular structure, we can estimate the informational content in the measurement data with respect to the considered process model. This type of analysis can estimate the inverse problem solution before its actual solution and evaluate the monitoring system efficiency with respect to the considered inverse modeling task [1,2]. 

Numerical experiments with the emission source identification problem for air pollution transport and transformation model were carried out to illustrate the developed framework. In the numerical experiments, we considered in-situ, image-type, and integral-type measurement data.

The work was supported by the grant №075-15-2020-787 in the form of a subsidy for a Major scientific project from Ministry of Science and Higher Education of Russia (project "Fundamentals, methods and technologies for digital monitoring and forecasting of the environmental situation on the Baikal natural territory").

References

[1] Penenko, A. Convergence analysis of the adjoint ensemble method in inverse source problems for advection-diffusion-reaction models with image-type measurements // Inverse Problems & Imaging, American Institute of Mathematical Sciences (AIMS), 2020, 14, 757-782 doi: 10.3934/ipi.2020035

[2] Penenko, A.; Gochakov, A. & Penenko, V. Algorithms based on sensitivity operators for analyzing and solving inverse modeling problems of transport and transformation of atmospheric pollutants // IOP Conference Series: Earth and Environmental Science, IOP Publishing, 2020, 611, 012032 doi: 10.1088/1755-1315/611/1/012032

How to cite: Penenko, A., Penenko, V., Tsvetova, E., Gochakov, A., Pyanova, E., and Konopleva, V.: Sensitivity Operator Inverse Modeling Framework for Advection-Diffusion-Reaction Models with Heterogeneous Measurement Data, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-7510, https://doi.org/10.5194/egusphere-egu21-7510, 2021.

A Geophysical model (subsurface imaging) is built up by a combination of many units that reflect the distribution of a certain physical property in the earth subsurface. The physical property can be any type like density, magnetic susceptibility, velocity, resistivity, or other properties. All the quantities which describe a geophysical model are termed as 'model parameters.' A geophysical model should explain the set of measurements recorded on the earth's surface to understand the subsurface structures. The set of all measurements is termed as 'data vector.' The present work deals with the inversion procedure to obtain a reliable model from the measured data sets. Regular grid discretization is an obstacle to define complex geological models and topography as well. In this context complex geological model can be generated through a triangular grid. Also, any type of complex geological model can be represented using triangular grids, which are difficult using a common discretization approach. In the present work, we have used Delaunay triangulation to discretize the subsurface to overcomes the problems encountered by the regular grid discretization. We have coded our forward formulation in such a way that multiple geophysical datasets can be generated on the same setup. Further, we have developed a common inversion framework to handle many geophysical datasets like Gravity, Magnetic, and VLF EM methods. This framework is utilizing the optimization scheme of the Conjugate Gradient Method. Since potential field anomalies decay with increasing depth of source, we have provided preconditioning to our kernel matrix to counteract the decay effect. We also noted that the preconditioned conjugate gradient method effectively deals with large matrices as it reduces the storage space and computation time. We demonstrated the developed approach using synthetic and real field data sets.

Keywords: Gravity, Magnetic, VLF EM, Geophysical inversion; Subsurface discretization, Delaunay triangulation, Preconditioned Conjugate Gradient method

How to cite: Verma, V. K. and Singh, A.: Triangular grid-based common inversion framework for different geophysical data to improve subsurface imaging, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-13994, https://doi.org/10.5194/egusphere-egu21-13994, 2021.

Numerical solvers using adaptive meshes can focus computational power on important regions of a model domain capturing important or unresolved physics. The adaptation can be informed by the model state, external information, or made to depend on the model physics. 
 In this latter case, one can think of the mesh configuration  as part of the model state. If observational data is to be assimilated into the model, the question of updating the mesh configuration with the physical values arises. Adaptive meshes present significant challenges when using popular ensemble Data Assimilation (DA) methods. We develop a novel strategy for ensemble-based DA for which the adaptive mesh is updated along with the physical values. This involves including the node locations as a part of the model state itself allowing them to be updated automatically at the analysis step. This poses a number of challenges which we resolve to produce an effective approach that promises to apply with some generality. We evaluate our strategy with two testbed models in 1-d comparing to a strategy that we previously developed that does not update the mesh configuration. We find updating the mesh improves the fidelity and convergence of the filter. An extensive analysis on the performance of our scheme beyond just the RMSE error is also presented.

How to cite: Sampson, C., Carrassi, A., Aydogdu, A., and Jones, C.: Ensemble Kalman Filter for non-conservative moving mesh solvers with a joint physics and mesh location update, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-16403, https://doi.org/10.5194/egusphere-egu21-16403, 2021.

Quantification of evolving uncertainties is required for both probabilistic forecasting and data assimilation in weather prediction. In current practice, the ensemble of model simulations is often used as primary tool to describe the required uncertainties. In this work, we explore an alternative approach, so called stochastic Galerkin method which integrates uncertainties forward in time using a spectral approximation in the stochastic space. 

In an idealized two-dimensional model that couples compressible non-hydrostatic Navier-Stokes equations to cloud dynamics, we investigate the propagation of initial uncertainty. The propagation of initial perturbations is followed through time for all model variables during two types of forecasts: the ensemble forecast and stochastic Galerkin forecast. Since model simulations are very expensive in weather forecasting, our hypothesis is that the stochastic Galerkin would provide more accurate and cheaper forecast statistics than the ensemble simulations. Results indicate that uncertainty as represented with mean, standard deviation and evolution of trace through time provides almost identical results if a 10000-member ensemble is used and truncation of stochastic Galerkin is made at ten spectral modes.  However, for coarser approximations,  for example if 50 ensemble members are used or the stochastic Galerkin is truncated at two modes, differences in standard deviations become significant in both approaches.  A series of experiments indicates that differences in performance of the two methods depend on the system state. For example, for stable flows, the stochastic Galerkin outperforms the ensemble of simulations for every truncation and every variable. In very unstable,  turbulent flows the estimate of the mean between the two methods still remains similar. However,  the ensemble of simulations needs more than 100 members (depending on the model variable) and the stochastic Galerkin a truncation with more than five spectral modes, to produce accurate results.

How to cite: Janjic, T., Lukacova, M., Ruckstuhl, Y., Spichtinger, P., and Wiebe, B.: A test of an alternative approach for uncertainty representation in weather forecasting, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-13077, https://doi.org/10.5194/egusphere-egu21-13077, 2021.

Parameterizations of turbulent mixing in the ocean surface boundary layer (OSBL) are key Earth System Model (ESM) components that modulate the communication of heat and carbon between the atmosphere and ocean interior. OSBL turbulence parameterizations are formulated in terms of unknown free parameters estimated from observational or synthetic data. In this work we describe the development and use of a synthetic dataset called the “LESbrary” generated by a large number of idealized, high-fidelity, limited-area large eddy simulations (LES) of OSBL turbulent mixing. We describe how the LESbrary design leverages a detailed understanding of OSBL conditions derived from observations and large scale models to span the range of realistically diverse physical scenarios. The result is a diverse library of well-characterized “synthetic observations” that can be readily assimilated for the calibration of realistic OSBL parameterizations in isolation from other ESM model components. We apply LESbrary data to calibrate free parameters, develop prior estimates of parameter uncertainty, and evaluate model errors in two OSBL parameterizations for use in predictive ESMs.

How to cite: Wagner, G., Souza, A., Hillier, A., Ramadhan, A., and Ferrari, R.: LESbrary: A library of large eddy simulation data for the calibration and uncertainty quantification of ocean surface boundary layer turbulence parameterizations, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-16199, https://doi.org/10.5194/egusphere-egu21-16199, 2021.

Quantifying forecast uncertainty is a key aspect of state-of-the-art data assimilation systems which has a large impact on the quality of the analysis and then the following forecast. In recent years, most operational data assimilation systems incorporate state-dependent uncertainty quantification approaches based on 4-dimensional variational approaches, ensemble-based approaches, or their combination. However, these quantifications of state-dependent uncertainties have a large computational cost. Machine learning techniques consist of trainable statistical models that can represent complex functional dependencies among different groups of variables. In this work, we use a fully connected two hidden layer neural network for the state-dependent quantification of forecast uncertainty in the context of data assimilation. The input to the network is a set of three consecutive forecasted states centered at the desired lead time and the network’s output is a corrected forecasted state and an estimation of its uncertainty. We train the network using a loss function based on the observation likelihood and a large database of forecasts and their corresponding analysis. We perform observing system simulation experiments using the Lorenz 96 model as a proof-of-concept and for an evaluation of the technique in comparison with classic ensemble-based approaches.

 Results show that our approach can produce state-dependent estimations of the forecast uncertainty without the need for an ensemble of states (at a much lower computational cost),  particularly in the presence of model errors. This opens opportunities for the development of a new type of hybrid data assimilation system combining the capabilities of machine learning and ensembles.

How to cite: Ruiz, J., Sacco, M., Zhen, Y., Tandeo, P., and Pulido, M.: Machine learning-based uncertainty quantification for data assimilation: a simple model experiment, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-9254, https://doi.org/10.5194/egusphere-egu21-9254, 2021.

This talk focuses on the application of projection-based reduced-order models (pROMs) to seismic elastic shear waves. Specifically, we present a method to efficiently propagate parametric uncertainties through the system using a novel formulation of the Galerkin ROM that exploits modern many-core computing nodes.

Seismic modeling and simulation is an active field of research because of its importance in understanding the generation, propagation and effects of earthquakes as well as artificial explosions. We stress two main challenges involved: (a) physical models contain a large number of parameters (e.g., anisotropic material properties, signal forms and parametrizations); and (b) simulating these systems at global scale with high-accuracy requires a large computational cost, often requiring days or weeks on a supercomputer. Advancements in computing platforms have enabled researchers to exploit high-fidelity computational models, such as highly-resolved seismic simulations, for certain types of analyses. Unfortunately, for analyses requiring many evaluations of the forward model (e.g., uncertainty quantification, engineering design), the use of high-fidelity models often remains impractical due to their high computational cost. Consequently, analysts often rely on lower-cost, lower-fidelity surrogate models for such problems.

Broadly speaking, surrogate models fall under three categories, namely (a) data fits, which construct an explicit mapping (e.g., using polynomials, Gaussian processes) from the system's parameters to the system response of interest, (b) lower-fidelity models, which simplify the high-fidelity model (e.g., by coarsening the mesh, employing a lower finite-element order, or neglecting physics), and (c) pROMs which reduce the number of degrees of freedom in the high-fidelity model by a projection process of the full-order model onto a subspace identified from high-fidelity data. The main advantage of pROMs is that they apply a projection process directly to the equations governing the high-fidelity model, thus enabling stronger guarantees (e.g., of structure preservation or of accuracy) and more accurate a posteriori error bounds.

State-of-the-art Galerkin ROM formulations express the state as a rank-1 tensor (i.e., a vector), leading to computational kernels that are memory bandwidth bound and, therefore, ill-suited for scalable performance on modern many-core and hybrid computing nodes. In this work, we introduce a reformulation, called rank-2 Galerkin, of the Galerkin ROM for linear time-invariant (LTI) dynamical systems which converts the nature of the ROM problem from memory bandwidth to compute bound, and apply it to elastic seismic shear waves in an axisymmetric domain. Specifically, we present an end-to-end demonstration of using the rank-2 Galerkin ROM in a Monte Carlo sampling study, showing that the rank-2 Galerkin ROM is 970 times more efficient than the full order model, while maintaining excellent accuracy in both the mean and statistics of the field.

How to cite: Rizzi, F., Parish, E., Blonigan, P., and Tencer, J.: Enabling efficient uncertainty quantification for seismic modeling via projection-based model reduction, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-4807, https://doi.org/10.5194/egusphere-egu21-4807, 2021.

Accurate forecast of ocean circulation is important in many aspects. A lack of direct ocean velocity observations has been one of the overarching issues in nowadays operational ocean data assimilation (DA) system. Satellite-tracked surface drifters, providing measurement of near-surface ocean currents, have been of increasing importance in global ocean observation system. In this work, the impact of an augmented-state Lagrangian data assimilation (LaDA) method using Local Ensemble Transform Filter (LETKF) is investigated within a realistic ocean DA system. We use direct location data from 300 surface drifters released in the Gulf of Mexico (GoM) by the Consortium for Advanced Research on Transport of Hydrocarbon in the Environment (CARTHE) during the summer 2012 Grand Lagrangian Deployment (GLAD) experiment. These drifter observations are directly assimilated into a realistic eddy-resolving GoM configuration of the Modular Ocean Model version 6 (MOM6) of the Geophysical Fluid Dynamics Laboratory (GFDL). Ocean states (T/S/U/V) are updated at both the surface and at depth by utilizing dynamic forecast error covariance statistics. Four experiments are conducted: (1) a free run generated by MOM6; 2) a DA experiment assimilating temperature and salinity profile observations from World Ocean Database 2018 (WOD18); and 3) a DA experiment assimilating both drifter and the profile observations. The LaDA results are then compared with the traditional assimilation using the drifter-derived velocity field from the same GLAD database. In addition, we evaluate the impact of the LaDA algorithm on different eddy-permitting and eddy-resolving model resolutions to determine the most effective horizontal resolutions for assimilating drifter position data using LaDA.

How to cite: Sun, L., Penny, S., and Harrison, M.: Improving Ocean Circulations Using Lagrangian Data Assimilation of Surface Drifters During Grand Lagrangian Deployments, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-10424, https://doi.org/10.5194/egusphere-egu21-10424, 2021.

Recent geoscience and palaeoclimatic modelling advances in have seen an increasing demand for spatio-temporal reconstructions of climatic variables. Satisfactory reconstructions should consider all sources of information: both numerical model ensembles and measured data. The difficulty in modelling climatic variables often gives rise to a multiplicity of models due to large uncertainty in the inputs. Climate proxy-based measurements are similarly uncertain due to both measurement noise and reconstruction error. It is thus vital to provide a reconstruction methodology in which these uncertainties are appropriately quantified. Instead of utilising probability based approaches that can be very computationally demanding for geospatio-temporal problems, we have developed a new approach to do this utilising a second-order framework; namely, Bayes linear analysis. This framework avoids the explicit specification of probability distributions and allows reconstructions to be described simply by means and variances. Methodological advances are made to the traditional Bayes linear mechanics to allow for non-linearity. To demonstrate the methodology, average monthly spatial reconstructions of sea-surface temperature and sea-ice concentration are estimated for the Last Glacial Maximum (21 ka), combining PMIP3 and PMIP4 outputs and available palaeodata syntheses. The methodology presented is generalisable to many spatio-temporal quantities and is highly germane to the geoscience community. 

How to cite: Astfalck, L., Williamson, D., Gandy, N., Gregoire, L., and Ivanovic, R.: A Statistical Reconstruction of Sea-Surface Temperature and Sea-Ice Concentration for the Last Glacial Maximum, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-7057, https://doi.org/10.5194/egusphere-egu21-7057, 2021.

A multiscale alignment (MSA) method was proposed by Ying (2019) for ensemble data assimilation to reduce the errors caused by displacement of coherent features. The MSA method decomposes a model state into components ranging from large to small spatial scales, then applies ensemble filters to update each scale component sequentially. After a larger scale component analysis increment is derived from the observations, displacement vectors are computed from the analysis increments through an optical flow algorithm. These displacement vectors are then used to warp the model mesh, which reduces position errors in the smaller scale components before the ensemble filter is applied again.

The MSA method is now applied to a sea ice prediction problem at NERSC to assimilate satellite-derived sea ice deformation observations into the next generation Sea Ice Model (neXtSIM) simulations. Preliminary results show that the MSA can more effectively reduce the position errors of the linear kinematic features of sea ice than the traditional ensemble Kalman filter. The alignment step is shown to be a big contributor for error reduction in our test case. We will also discuss the remaining challenges of tuning parameters in the MSA method and dealing with model deficiencies.

How to cite: Ying, Y. and Bertino, L.: Assimilating sea ice deformation observations using a multiscale alignment ensemble data assimilation method, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-3880, https://doi.org/10.5194/egusphere-egu21-3880, 2021.

Indirect inversion approaches are widely used in Geosciences, and in particular also for the identification of the hydraulic properties of aquifers. Nevertheless, their application requires a substantial number of model evaluation (forward problem) runs, a task that for complex problems can be computationally intensive. Reducing this computational burden is an active research topic, and many solutions, including the use of hybrid optimization methods, the use of physical proxies or again machine-learning tools allow to avoid considering the full physics of the problem when running a numerical implementation of the forward problem.

Direct inversion approaches represent computationally frugal alternatives to indirect approaches, because in general they require a smaller number of runs of the forward problem. The classical drawbacks of these methods can be alleviated by some implementation approaches and in particular by using multiple sets of data, when available.

This work is an effort to improve the robustness of the Comparison Model Method (CMM), a direct inversion approach aimed at the identification of the hydraulic transmissivity of a confined aquifer. The robustness of the CMM is here ameliorated by (i) improving the parameterization required to handle small hydraulic gradients; (ii) investigating the role of different criteria aimed at merging multiple data-sets corresponding to different flow conditions.

On a synthetic case study, it is demonstrated that correcting a small percentage of the small hydraulic gradients (about 10%) allows to obtain reliable results, and that a criteria based on the geometric mean is adequate to merge the results coming from multiple data-sets. In addition, the use of multiple-data sets allows to noticeably improve the robustness of the CMM when the input data are affected by noise.

All the tests are performed by using open source and widely used tools like the USGS Modflow6 and its Python interface flopy to foster the application of the CMM. The scripts and corresponding package, named cmmpy, is available on the Python Package Index (PyPI) and on bitbucket at the following address: https://bitbucket.org/alecomunian/cmmpy.

How to cite: Comunian, A. and Giudici, M.: Approaches to improve the robustness of the Comparison Model Method for the inverse problem of groundwater hydrology, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-14539, https://doi.org/10.5194/egusphere-egu21-14539, 2021.

In variational data assimilation, background-error covariance structures have the ability to spread information from an observed part of the system to unobserved parts.  Hence an accurate specification of these structures is crucially important for the success of assimilation systems and therefore of forecasts that their outputs initiate.  For oceanic models, background-error covariances have traditionally been modelled by parametrisations which mainly depend on macroscopic properties of the ocean and have limited dependence on local conditions.  This can be problematic during passage of tropical cyclones, when the spatial and temporal variability of the ocean state depart from their characteristic structures.  Furthermore, the traditional method of estimating oceanic background-error covariances could amplify imbalances across the air-sea interface when weakly coupled data assimilation is applied, thereby bringing a detrimental impact to forecasts of cyclones.  Using the case study of Cyclone Titli, which affected the Bay of Bengal in 2018, we explore hybrid methods that combine the traditional modelling strategy with flow-dependent estimates of the ocean's error covariance structures based on the latest-available short-range ensemble forecast.  This hybrid approach is investigated in the idealised context of a single-column model as well as in the UK Met Office’s state-of-the-art system.  The idealised model helps inform how the inclusion of ensemble information can improve coupled forecasts.  Different methods for producing the ensemble are explored, with the goal of generating a limited-sized ensemble that best represents the uncertainty in the ocean fields.  We then demonstrate the power of this hybrid approach in changing the analysed structure of oceanic fields in the Met Office system, and explain the difference between the traditional and hybrid approaches in light of the ways the assimilation systems respond to single synthetic observations.  Finally, we discuss the benefits that the hybrid approach in ocean data assimilation can bring to atmospheric forecasts of the cyclone.

How to cite: Leung, T. Y., Smith, P. J., Lawless, A. S., Nichols, N. K., and Martin, M. J.: The role of flow-dependent oceanic background-error covariance information in air-sea coupled data assimilation during tropical cyclones: a case study, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-3170, https://doi.org/10.5194/egusphere-egu21-3170, 2021.

We compare the results of strongly coupled data assimilation and weakly coupled data assimilation by analyzing the assimilation effect on the prediction of the ocean as well as the atmosphere variables. The AWI climate model (AWI-CM), which couples the ocean model FESOM and the atmospheric model ECHAM, is coupled with the parallel data assimilation framework (PDAF, http://pdaf.awi.de). The satellite sea surface temperature is assimilated. For the weakly coupled data assimilation, only the ocean variables are directly updated by the assimilation while the atmospheric variables are influenced through the model. For the strongly coupled data assimilation, both the ocean and the atmospheric variables are directly updated by the assimilation algorithm. The results are evaluated by comparing the estimated ocean variables with the dependent/independent observational data, and the estimated atmospheric variables with the ERA-interim data. In the ocean, both the WCDA and the SCDA improve the prediction of the temperature and SCDA and WCDA give the same RMS error of SST. In the atmosphere, WCDA gives slightly better results for the 2m temperature and 10m wind velocity than the SCDA. In the free atmosphere, SCDA yields smaller errors for the temperature, wind velocity and specific humidity than the WCDA in the Arctic region, while in the tropical region, the error are larger in general.

How to cite: Tang, Q., Mu, L., Goessling, H., Semmler, T., and Nerger, L.: Strongly coupled data assimilation with the coupled ocean-atmosphere model AWI-CM: comparison with the weakly coupled data assimilation, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-14181, https://doi.org/10.5194/egusphere-egu21-14181, 2021.

Coupled general circulation models (GCM) of the atmosphere, ocean, land and sea-ice have many parameters. Some of which govern the numerics of the dynamical core, whilst others represent the influence of unresolved subgrid process based on our current fundamental physical understanding. The spatio-temporal structure of many of these parameters are known with little precision, which contributes to the inherent model biases in the underlying GCM. To address this problem we use the CSIRO Climate re-Analysis and Forecast Ensemble (CAFE) system to estimate both the climate state (atmosphere, ocean, sea-ice) and also spatio-temporally varying parameter maps of the ocean surface albedo and shortwave radiation e-folding length scale in a coupled climate GCM of CMIP resolution and complexity. The CAFE system adopts a 96 member ensemble transform Kalman filter within a strongly coupled data assimilation (DA) framework. The parameters (and states) are determined by minimising the error between short term DA cycle forecasts of the climate model and a network of real world atmospheric, oceanic, and sea-ice observations.  Several DA cycle lengths are tested between 3 to 28 days. The DA system has an improved fit to observations over the period from 2010 to 2012, when estimating both of the ocean optical parameters either individually or simultaneously. However, only individually estimated maps of shortwave e-folding length scale attain systematically reduced bias in multi-year climate forecasts during the out-of-sample period from 2012 to 2020. Parameter maps determined from longer DA cycle lengths also have further reduced multi-year forecast bias. Such improved climate forecasts would potentially enable policy makers to make better informed decisions on water, energy and agricultural infrastructure and planning.

How to cite: Kitsios, V., Sandery, P., O'Kane, T., and Fiedler, R.: Strongly coupled ensemble transform Kalman filter estimation of ocean optical parameters in a coupled GCM, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-9306, https://doi.org/10.5194/egusphere-egu21-9306, 2021.

The forecast of tropical cyclone (TC) intensity is a significant challenge.  In this study, we showcase the impact of strongly coupled data assimilation with hypothetical ocean currents on analyses and forecasts of Typhoon Hato (2017). 

Several observation simulation system experiments were undertaken with a regional coupled ocean-atmosphere model. We assimilated combinations of (or individually) a hypothetical coastal current HF radar network, a dense array of drifter floats and minimum sea-level pressure. During the assimilation, instant updates of many important atmospheric variables (winds and pressure) are achieved from the assimilation of ocean current observations using the cross-domain error covariance, significantly improving the track and intensity analysis of Typhoon Hato. As compared to a control experiment (with no assimilation), the error of minimum pressure decreased by up to 13 hPa (4 hPa / 57 % on average). The maximum wind speed error decreased by up to 18 knots (5 knots / 41 % on average). 

By contrast, weakly coupled implementations cannot match these reductions (10% on average). Although traditional atmospheric observations were not assimilated, such improvements indicate there is considerable potential in assimilating ocean currents from coastal HF radar, and surface drifters within a strongly coupled framework for intense landfalling TCs.

How to cite: Phillipson, L., Li, Y., and Toumi, R.: Strongly Coupled Assimilation of a Hypothetical Ocean Current Observing Network within a Regional Ocean-Atmosphere Coupled Model: An OSSE Case Study of Typhoon Hato, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-8360, https://doi.org/10.5194/egusphere-egu21-8360, 2021.

The ‘observation operator’ is essential in data assimilation (DA) to derive the model equivalent of the observations from the model variables. For satellite radiance observations, it is usually based on complex radiative transfer model (RTM) with a bias correction procedure. Therefore, it usually takes time to start using new satellite data after launching the satellites. Here we take advantage of the recent fast development of machine learning (ML) which is good at finding the complex relationships within data. ML can potentially be used as the ‘observation operator’ to reveal the relationships between the model variables and the observations without knowing their physical relationships. In this study, we test with the numerical weather prediction system composed of the Nonhydrostatic Icosahedral Atmospheric Model (NICAM) and the Local Ensemble Transform Kalman Filter (LETKF). We focus on the satellite microwave brightness temperature (BT) from the Advanced Microwave Sounding Unit-A (AMSU-A). Conventional observations and AMSU-A data were assimilated every 6 hours. The reference DA system employed the observation operator based on the RTTOV and an online bias correction method.

We used this reference system to generate 1-month data to train the machine learning model. Since the reference system includes running a physically-based RTM, we implicitly used the information from RTM for training the ML model in this study, although in our future research we will explore methods without the use of RTM. The machine learning model is artificial neural networks with 5 fully connected layers. The input of the ML model includes the NICAM model variables and predictors for bias correction, and the output of the ML model is the corresponding satellite BT in 3 channels from 5 satellites. Next, we ran the DA cycle for the same month the following year to test the performance of the ML model. Two experiments were conducted. The control experiment (CTRL) was performed with the reference system. In the test experiment (TEST), the ML model was used as the observation operator and there is no separate bias correction procedure since the training includes biased differences between the model and observation. The results showed no significant bias of the simulated BT by the ML model. Using the ECMWF global atmospheric reanalysis (ERA-interim) as a benchmark to evaluate the analysis accuracy, the global-mean RMSE, bias, and ensemble spread for temperature in TEST are 2% higher, 4% higher, and 1% lower respectively than those in CTRL. The result is encouraging since our ML can emulate the RTM. The limitation of our study is that we rely on the physically-based RTM in the reference DA system, which is used for training the ML model. This is the first result and still preliminary. We are currently considering other methods to train the ML model without using the RTM at all.

How to cite: Liang, J., Terasaki, K., and Miyoshi, T.: A machine learning approach to the observation operator for satellite radiance data assimilation, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-10475, https://doi.org/10.5194/egusphere-egu21-10475, 2021.

This paper addresses representation learning for the resolution of inverse problems  with geophysical dynamics. Among others, examples of inverse problems of interest include space-time interpolation, short-term forecasting, conditional simulation w.r.t. available observations, downscaling problems… From a methodological point of view, we rely on a variational data assimilation framework. Data assimilation (DA) aims to reconstruct the time evolution of some state given a series of  observations, possibly noisy and irregularly-sampled. Here, we investigate DA from a machine learning point of view backed by an underlying variational representation.  Using automatic differentiation tools embedded in deep learning frameworks, we introduce end-to-end neural network architectures for variational data assimilation. It comprises two key components: a variational model and a gradient-based solver both implemented as neural networks. A key feature of the proposed end-to-end learning architecture is that we may train the neural networks models using both supervised and unsupervised strategies. We first illustrate applications to the reconstruction of Lorenz-63 and Lorenz-96 systems from partial and noisy observations. Whereas the gain issued from the supervised learning setting emphasizes the relevance of groundtruthed observation dataset for real-world case-studies, these results also suggest new means to design data assimilation models from data. Especially, they suggest that learning task-oriented representations of the underlying dynamics may be beneficial. We further discuss applications to short-term forecasting and sampling design along with preliminary results for the reconstruction of sea surface currents from satellite altimetry data. 

This abstract is supported by a preprint available online: https://arxiv.org/abs/2007.12941

How to cite: Fablet, R., Chapron, B., Drumetz, L., Memin, E., Pannekoucke, O., and Rousseau, F.: Jointly learning variational data assimilation models and solvers for geophysical dynamics, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-15678, https://doi.org/10.5194/egusphere-egu21-15678, 2021.

Going from low- to high-resolution models is an efficient way to improve the data assimilation process in three ways: it makes better use of high-resolution observations, it represents more accurately the small scale features of the dynamics and it provides a high-resolution field that can further be used as an initial condition of a forecast. Of course, the pitfall of such an approach is the cost of computing a forecast with a high-resolution numerical model. This drawback is even more acute when using an ensemble data assimilation approach, such as the ensemble Kalman filter, for which an ensemble of forecasts is to be issued by the numerical model.

In our approach, we propose to use a cheap low-resolution model to provide the forecast while still performing the assimilation step in a high-resolution space. The principle of the algorithm is based on a machine learning approach: from a low-resolution forecast, a neural network (NN) emulates a high-resolution field that can then be used to assimilate high-resolution observations. This NN super-resolution operator is trained on one high-resolution simulation. This new data assimilation approach denoted "Super-resolution data assimilation" (SRDA), is built on an ensemble Kalman filter (EnKF) algorithm.

We applied SRDA to a quasi-geostrophic model representing simplified ocean dynamics of the surface layer, with a low-resolution up to four times smaller than the reference high-resolution (so the cost of the model is divided by 64). We show that this approach outperforms the standard low-resolution data assimilation approach and the SRDA method using standard interpolation instead of a neural network as a super-resolution operator. For the reduced cost of a low-resolution model, SRDA provides a high-resolution field with an error close to that of the field that would be obtained using a high-resolution model.

How to cite: Barthélémy, S., Brajard, J., and Bertino, L.: High-resolution Ensemble Kalman Fiter with a low-resolution model using a machine learning super-resolution approach., EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-2585, https://doi.org/10.5194/egusphere-egu21-2585, 2021.

There are some parameters that affect the resistivity values in the electrical resistivity method which is one of the most fundamental methods in near surface geophysics. One of these parameters is electrical anisotropy which is defined as the change in resistivity depending on the direction. The anisotropy coefficient is calculated by square root of the vertical resistivity to the horizontal resistivity of the layer. Average resistivity in anisotropic media is the geometric mean of the vertical resistivity and the horizontal resistivity of the layer. Artificial Neural Networks (ANN) is a method uses in many different areas for learning, classification, generalization and optimization etc. ANN available to estimate the thickness, vertical and horizontal resistivity values of layers. In this study, a MATLAB code was developed for the inversion of one-dimensional electrical resistivity data in anisotropic medium by using artificial neural networks. Neural Network Toolbox of MATLAB was utilized in the developed program. The code was tested on both noisy-free and five percent noisy synthetic data. Thicknesses, vertical and horizontal resistivity of the layers are estimated by using the code. The mean resistivity values and anisotropy coefficients of each layer were calculated via the estimated parameters. The estimated parameters and the parameters of the subsurface model were similar with acceptable error rates.

How to cite: Durdağ, D. and Pekşen, E.: Inversion of One Dimensional Electrical Resistivity Data in Anisotropic Media via Artificial Neural Networks, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-14158, https://doi.org/10.5194/egusphere-egu21-14158, 2021.

In previous work, it was shown that preservation of physical properties  in the data assimilation framework can significantly reduce forecast errors. Proposed data assimilation methods, such as the quadratic programming ensemble (QPEns) that can impose such constraints on the calculation of the analysis, are computationally more expensive, severely limiting their application to high dimensional prediction systems as found in earth sciences. We therefore propose to use a convolutional neural network (CNN) trained on the difference between the analysis produced by a standard ensemble Kalman Filter (EnKF) and the QPEns to correct any violations of imposed constraints. On this poster, we focus on conservation of mass and show in an idealized setup that the hybrid of a CNN and the EnKF is capable of reducing analysis and background errors to the same level as the QPEns. 

How to cite: Ruckstuhl, Y., Janjic, T., and Rasp, S.: Training a convolutional neural network to conserve mass in data assimilation, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-4350, https://doi.org/10.5194/egusphere-egu21-4350, 2021.

Filtering is an uncertainty quantification technique that refers to the inference of the states of dynamical systems from noisy observations. This work proposes a machine learning-based filtering method for tracking the high-dimensional non-Gaussian state-space models with non-linear dynamics and sparse observations. Our filter method is based on the conditional expectation mean filter and uses machine-learning techniques to approximate the conditional mean (CM). The contribution of this work is twofolds: (i) we demonstrate theoretically that the assimilated ensembles obtained using the ensemble conditional mean filter (EnCMF) provide a correct prediction of the posterior mean and have the optimal variance, and (ii) we implement the EnCMF using artificial neural networks, which has a significant advantage in representing non-linear functions that map between high-dimensionality domains, such as the CM. We implement the machine learning-based EnCMF for tracking the states of the Lorenz-63 and 96 systems under the chaotic regime. Numerical results show that the EnCMF outperforms the ensemble Kalman filter.

How to cite: Hoang, T.-V., Krumscheid, S., and Tempone, R.: Machine learning based conditional mean filter: a non-linear extension of the ensemble Kalman filter, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-9566, https://doi.org/10.5194/egusphere-egu21-9566, 2021.

We propose to use analogs of the forecast mean to generate an ensemble of perturbations for use in ensemble optimal interpolation (EnOI) or ensemble variational (EnVar) methods.  In addition to finding analogs from a library, we propose a new method of constructing analogs using autoencoders (a machine learning method).  To extend the scalability of constructed analogs for use in data assimilation on geophysical models, we propose using patching schemes to divide the global spatial domain into digestable chunks.  Using patches makes training the generative models possible and has the added benefit of being able to exploit parallel computing powers.  The resulting analog methods using analogs from a catalog (AnEnOI), constructed analogs (cAnEnOI), and patched constructed analogs (p-cAnEnOI) are tested in the context of a multiscale Lorenz-`96 model, with standard EnOI and an ensemble square root filter for comparison.  The use of analogs from a modestly-sized catalog is shown to improve the performance of EnOI, with limited marginal improvements resulting from increases in the catalog size.  The method using constructed analogs is found to perform as well as a full ensemble square root filter, and to be robust over a wide range of tuning parameters.  Lastly, we find that p-cAnENOI with larger patches produces the best data assimilation performance despite having larger reconstruction errors.  All patch variants except for the variant that uses the smallest patch size outperform cAnEnOI as well as some traditional data assimilation methods such as the ensemble square root filter.

How to cite: Yang, L. and Grooms, I.: Using machine learning techniques to generate analog ensembles for data assimilation, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-6036, https://doi.org/10.5194/egusphere-egu21-6036, 2021.