The medium-range forecasting of global weather plays a pivotal role in decision-making processes across various societal and economic sectors. Recent years have witnessed a rapid evolution in machine learning (ML) models applications in weather prediction, demonstrating notably superior performance compared to traditional numerical weather prediction (NWP) models. These cutting-edge models leverage diverse ML architectures, such as Graph Neural Networks (GNNs), Convolutional Neural Networks (CNNs), Fourier Neural Operators (FNOs), and Transformers. Notably, Google DeepMind has pioneered a novel ML-based approach known as GraphCast, enabling direct training from reanalysis data and facilitating global predictions for numerous weather variables in less than a minute. Impressively, GraphCast forecasts show improved accuracy in predicting severe weather events, including phenomena like tropical cyclones, atmospheric rivers, and extreme heat. However, the efficiency of GraphCast relies on high-quality historical weather data for training, typically sourced from ECMWF's ERA5 reanalysis. 

Concurrently, the National Centers for Environmental Prediction (NCEP) has initiated efforts in collaboration with the research community, focusing on developing Machine Learning Weather Prediction (MLWP). This study assesses the pre-trained GraphCast model, leveraging Global Data Assimilation System (GDAS) data from the operational GFSv16 model as initial states. Additionally, we explore the potential use of GDAS data as an alternative training source for GraphCast. Notably, GDAS data is available at a 0.25-degree latitude-longitude resolution and with a temporal resolution of 6 hours. Our investigation involves a comparative analysis of GraphCast's performance when initiated on GDAS data versus ERA5 and HRES data. Alongside this comparative analysis, we investigate the advantages and limitations of utilizing GDAS data for GraphCast while proposing potential approaches for enhancing future iterations of this study.

How to cite: Tabas, S., Wang, J., Yang, F., Levit, J., Stajner, I., Montuoro, R., Tallapragada, V., and Gross, B.: GDAS-Powered Machine Learning Weather Prediction: A Comparative Study on GraphCast initialized with GDAS States for Global Weather Forecasting, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-13889, https://doi.org/10.5194/egusphere-egu24-13889, 2024.

Our very recent artificial intelligence – machine learning (AI/ML) augmented wind energy production forecast has successfully demonstrated a consistent >30% wind speed and power generation forecast improvement over the NOAA operational High-Resolution Rapid Refresh (HRRR) standalone capability.

So far, we have used a suite of AI/ML algorithms, including 1) artificial neural networks, 2) ridge regression, 3) lasso regression, 4) support vector machines, 5) gradient boosting, 6) elastic networks, 7) nearest neighboring clustering, and 8) random forest (RF) models for wind energy forecasting applications. These AI/ML augmented forecasts significantly improve the management of the power grid distribution, energy trading strategy, and plant operations with training and testing corresponding to 253 sites in Texas and validated on a year of independent testing data. It has shown that each AI/ML model offers significant forecast improvement (+20% mean squared error) skill over the current official HRRR forecasts. Furthermore, an AI/ML model ensemble of different machine learning models is deployed and demonstrated to significantly improve wind speed accuracy during all seasons, times of day, sites tested, and forecast horizon times.

This fully matured AI/ML augmented framework has shown to be comprehensive and robust, demonstrating that AI/ML is a natural complement to the existing NWP infrastructure and can be expanded to enhance local forecasts.

How to cite: Huang, H.-L. A.: AI/ML Augmented Hyper Local Weather Forecast, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-2320, https://doi.org/10.5194/egusphere-egu24-2320, 2024.

Near-surface wind conditions, specifically at 10 meters above ground, play a crucial role in areas with complex topography like the Austral Chilean Territory, characterized by small islands, channels, and fiords. The impact of topography and land cover on wind patterns is particularly significant. In the other hand, wind impacts local transport and, consequently, the economy and social activities. Accurate forecasting of these winds is essential for optimal planning and heightened maritime safety.

While dynamic models, such as the WRF model, have proven valuable for stakeholders, their operational use is limited by the high computational cost, restricting spatial resolutions to a few kilometers. For example, the Chilean Navy Weather Service employs the WRF model with a resolution of up to 3 km in specific areas, and the Chilean Weather Office uses a 4 km resolution across the entire continental territory.

This study addresses this limitation by developing an emulator for dynamic downscaling of surface wind, aiming for hyper resolutions (~300 m) over Austral Chile. Utilizing cluster analysis of ERA 5 10m-wind fields, eight wind patterns were identified. Multi-day simulations were conducted with telescopic domains reaching 100 m resolution, incorporating NASA's ASTER DEM into WRF and updating the coastline in the default 500 m land-use dataset. The consistency analysis of these results will be presented.

To achieve hyper-resolution forecasting, various deep learning models, including Convolutional Neural Networks (CNN) and Generative Adversarial Networks (GAN), were trained to downscale the 3 km domain to the 100 m one. The presentation will focus on the evaluation and comparison of these models, showcasing the first key results of this research. This study is part of a larger research project that aims to produce a very high-resolution wind forecasting system, based on the downscaling of WRF simulations by using Deep learning techniques (SiVAR-Austral, funded by ANID ID22I10206). Results will be valuable to stakeholders by enhancing both planning capabilities and maritime safety in the Austral Chilean Territory.

How to cite: Arevalo, J., Ávila, A., Gomez, W., Andrade, P., Pozo, D., Bozkurt, D., Lagos, R., Alvial, F., and Cordova, A. M.: Hyper-Resolution Wind Forecasting in Austral Chile combining WRF forecasting and Deep Learning techniques., EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-6633, https://doi.org/10.5194/egusphere-egu24-6633, 2024.

Marine low clouds have a pronounced cooling effect on the climate system because of their large cloud fraction (CF) and high albedo. However, predicting marine low clouds with satellite data remains challenging due to the non-linear response of marine low clouds to cloud-controlling factors (CCFs) and the ignorance of cloud droplet number concentration (N_d). Here, we developed a unified convolutional neural network (CNN) incorporating meteorology and N_d as CCFs to predict critical properties of marine low clouds, such as CF, albedo, and cloud radiative effects (CRE). Our CNN model excels in capturing the variability of these cloud properties, achieving over 70% variance explanation for daily 1x1 degree areas, surpassing previous studies. It also effectively replicates geographical patterns of CF, albedo, and CRE, including climatology and long-term trends from 2003 to 2022. This research underscores the significant potential of deep learning in deep exploitation of the information content of the data and, thus, advancing our understanding of aerosol-cloud interactions, a pioneering effort in the field.

How to cite: Cao, Y., Zhu, Y., Wang, M., Rosenfeld, D., and Zhou, C.: Improving prediction of marine low clouds with cloud droplet number concentration and a deep learning method, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-4951, https://doi.org/10.5194/egusphere-egu24-4951, 2024.

How to cite: Wilson, J., Sampson, J., Girard, M., Hammami, K., Jervis, D., McKeever, J., Ramier, A., and Qudsi, Z.: Automatic cloud detection in GHGSat satellite imagery, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-20404, https://doi.org/10.5194/egusphere-egu24-20404, 2024.

Tropical cyclones are intense weather phenomena that originate over tropical oceans, posing significant threats to human life and property safety. This paper introduces methods for extracting and forecasting tropical cyclone information based on deep learning and satellite infrared images. It includes tropical cyclone wind radii estimation (global), tropical cyclone center location (Northwest Pacific), and tropical cyclone intensity forecast (Northwest Pacific). Utilizing infrared images and ERA5 reanalysis data, datasets for tropical cyclone wind radii estimation from 2004 to 2016, tropical cyclone center location from 2015 to 2018, and 24-hour tropical cyclone intensity forecasts from 1979 to 2021 have been constructed.

Firstly, the DL-TCR model with an asymmetric branch is designed to infer the asymmetric tropical cyclone wind radii (R34, R50 and R64) of global tropical cyclones. A modified MAE-weighted loss function is introduced to enhance the model's underestimation of large-sized tropical cyclone wind radii. The results indicate that the DL-TCR model achieves MAEs for R34 wind radii of 18.8, 19.5, 18.6, and 18.8 n mi in the NE, SE, SW, and NW quadrants, respectively. For R50 wind radii, the MAEs are 11.3, 11.3, 11.1, and 10.8 n mi, and for R64 wind radii, the MAEs are 8.9, 9.9, 9.2, and 8.7 n mi. These values represent an improvement of 12.1-35.5% compared to existing methods.

Then, employing transfer learning by transferring pre-trained models based on the ImageNet natural image dataset significantly improved the precision of tropical cyclone center location models. The results demonstrate that the transfer-learning-based model enhances the location accuracy by 14.1% compared to models without transfer learning. The location error for the tropical cyclone centers in the test data is 29.3 km, and for H2-H5 category, the tropical cyclone center location error is less than 20 km.

Finally, a deep learning model, named the TCIF-fusion model, was developed with two distinct branches engineered to learn multi-factor information and forecast the intensity of TCs over a 24-hour period. Ultimately, heatmaps were generated to capture the model's insights, which were then utilized to augment the original input data, leading to an improved dataset that significantly enhanced the accuracy of the TC intensity forecasting. Utilizing the refined input, the heatmaps (referred to as model knowledge, MK) were employed to direct the modeling process of the TCIF-fusion model. Consequently, the model guided by MK achieved a 24-hour forecast error of 3.56 m/s for Northwest Pacific TCs during the period from 2020 to 2021. The MK-based TCIF-fusion model has improved the forecasting performance by 12.1-35.5% compared to existing methods.

In summary, deep learning exhibits significant potential in the extraction and forecasting of tropical cyclone information, positioning it as a crucial tool for future tropical cyclone monitoring and forecasting.

How to cite: Wang, C. and Li, X.: Tropical Cyclone Information Extraction and Forecast Based on Satellite Infrared Images and Deep Learning Technology, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-7056, https://doi.org/10.5194/egusphere-egu24-7056, 2024.

Quality control (QC) on data has historically been a tedious and time-consuming task. With the currently available computer processing power and machine learning algorithms, it is possible to make QC far faster and more efficient, providing high-quality data in near real time to end users. Many organisations are already using such systems with great success, although the rapid expansion of machine learning continues to open new avenues for improvement. The aim of this project is to create a QC system for Met Eireann, the Irish Meteorological Office, which incorporates the latest machine learning techniques, combined with expert human supervision, to produce the highest possible quality meteorological data.

Presented here are the ideas and concepts we intend to implement to create the QC system, showing the results of the initial trials on air temperature data. The project is still in the earliest stages of development and will benefit from input and feedback from others with experience working on similar projects.

How to cite: Spohn, T., O’Donoghue, J., Horan, K., Charnecki, T., Lally, C., and Haslam, M.: Performing Quality Control on Meteorological Data Using Machine Learning Techniques, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-3905, https://doi.org/10.5194/egusphere-egu24-3905, 2024.

A plethora of studies have used machine learning for quantitative precipitation forecasting. However, only a fraction of those studies have focused on the explainability of the utilized machine learning models. Consequently, to the best of the authors' knowledge, the variability in explainability concerning predictor clusters (i.e., grouped predictor categories based on shared attributes such as climate categories) has not received attention in the literature.

This study aims to address this gap by analyzing variability in explanations at the model level regarding different Köppen Climate Zones (i.e., arid, temperate, and continental climates). To this end, Türkiye is selected as the study area, which has a complex topography and omnigenous in climate types. The utilized dataset covers 687 stations spanning 10 different climate zones (clustered into B, C, and D Köppen climate zones) and more than one million rows covering four years as temporal coverage. While the ground truth is defined as the daily observed precipitation amount, the predictors consist of daily total precipitation forecasts of numerical weather prediction models (ECMWF, GFS, ALARO, and WRF) with a 24-hour lead time, geographical parameters (elevation, roughness, slope, aspect, distance to the sea, latitude and longitude), and seasonality (day of the year, and month). The study uses a multi-layer perceptron (Root Mean Squared Error = 3.6 mm/day),  as the machine learning method with two hidden layers (with Gaussian Error Linear Unit non-linearity). It utilizes Huber-Loss (delta = 1.5) as the loss function to mitigate the adverse effects of the long-tailed dataset. A Linear Interpretable Mogel Agnostic (LIME) approach is utilized to explain the predictions by MLP. Topographical, coordinate-based, and seasonality predictors are grouped except for the distance to the sea.

The importance assessments of predictors are compared with drop-out loss, which quantifies the decline in model performance that occurs when a predictor is removed, showing the relevance of the predictors to the predictions of models. Analysis results indicate that the ECMWF forecasts are the most important predictor for the model for all three climate types, with a drop-out loss value of 0.531 for arid (B) climate zones, 1.617 for temperate (C) climate zones, and 0.901 for continental (D) climate zones. Seasonality is more utilized for generating the predictions for continental climate zones (0.05 vs 0.02 for both arid and temperate zones). Another noteworthy result is that the distance to the sea predictor negatively affects the model over arid zones (-0.03) while positively contributing to both continental (0.013) and temperate zones (0.102). Moreover, the drop-out loss for distance to the sea (0.102) exceeds the WRF forecast's (0.076) over temperate climate zones. This might be related to the average distance to the sea (0.99 degrees over temperate, 1.66 over arid, and 1.72 over continental zones). Similarly, topographical parameters have a positive effect over arid (0.003) and continental zones (0.014) while having a negative effect over temperate (-0.012) zones. These results indicate that both multi-model machine learning designs can be beneficial for complex datasets, and the influence of parameters can vary over different input clusters.

How to cite: Senocak, A. U. G., Kalkan, S., Yilmaz, M. T., Yucel, I., and Amjad, M.: Variability among Machine Learning Explanations for Precipitation Forecasting in Köppen Climate Zones, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-17782, https://doi.org/10.5194/egusphere-egu24-17782, 2024.

Changes to low-level cloud properties and their associated feedback in a warming climate are a significant source of uncertainty in global climate models (GCMs). “Local’’ processes at the droplet scales, such as drizzle growth by collision-coalescence, are not well represented in GCMs and constitute a significant uncertainty in model predictions. Parameterization schemes often derived from empirical fits to spatially averaged cloud size distributions have been used to represent clouds and hence do not fully account for the subgrid-scale variabilities. We hypothesize that inhomogeneities in cloud microphysical properties may be captured by a small number of distinct droplet size distributions called “characteristic distributions” and developed an algorithm capable of retrieving them.

To do this, we developed an algorithm by combining hypothesis testing with a machine-learning clustering algorithm. The test does not presume any specific distribution shape, is parameter-free, and avoids biases from binning. Importantly, for the clustering algorithm, the number of clusters is not an input parameter but is independently determined in an unsupervised fashion. As implemented, it works on an abstract space from the hypothesis test results, and hence spatial correlation is not fundamental for members classified to a characteristic distribution. To validate the algorithm's robustness, we test it on a synthetic dataset that mimics cloud drop distributions. The algorithm successfully identifies the predefined distributions at plausible noise levels.

When implemented on cm-scale cloud samples taken using Holographic Detector for Clouds (HOLODEC) deployed during Aerosol and Cloud Experiments in the Eastern North Atlantic (ACE-ENA), the algorithm reveals that local characteristic distribution types are ubiquitous in stratocumulus clouds. These distribution types are generally narrow with distinct modes and do not resemble the averaged size distribution shape. Each characteristic distribution represents identical-looking local cloud volumes which tend to occur in spatial blocks of varying extent, usually of order 1s to 10s of km. These observations have implications for understanding small-scale cloud properties and can guide the development of novel parameterizations of sub-grid-scale variability for coarse-resolution models. Subsequently, we show the first results from an investigation of characteristic distributions for LESs. The algorithm is general and helps in finding similarities in data representable as CDFs and is expected to have broader applicability in earth sciences.

How to cite: Allwayin, N., Larsen, M., Shaw, A., Chandrakar, K. K., Glienke, S., and Shaw, R.: Understanding cloud structures with machine learning- An algorithm to represent sub-grid scale variability in stratocumulus clouds , EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-14260, https://doi.org/10.5194/egusphere-egu24-14260, 2024.

FORUM (Far-infrared Outgoing Radiation Understanding and Monitoring) represents the ninth Earth Explorer mission chosen by the European Space Agency (ESA) in 2019. This satellite mission focuses on delivering interferometric measurements within the Far-InfraRed (FIR) spectrum, constituting approximately 50% of the Earth's longwave flux emitted into space. Enhanced accuracy in measuring the Top Of the Atmosphere (TOA) spectrum in the FIR is crucial for minimizing uncertainties in climate models. However, current instruments fall short, necessitating the incorporation of innovative computational techniques. The mission aims to refine understanding across various atmospheric variables, including tropospheric water vapor, ice cloud properties, and notably, surface emissivity in the FIR.
During the mission's early development, an End-to-End Simulator (E2ES) was devised to showcase proof-of-concept and assess the impact of instrument characteristics and scene conditions on the accuracy of reconstructed atmospheric properties. This simulator comprises a sequence of modules simulating the entire measurement acquisition process, accounting for all major sources of discrepancies in operational conditions leading to the retrieval of geophysical quantities.
From a mathematical perspective, two challenges arise: the radiative transfer equation, known as the direct problem, and its inversion, referred to as the retrieval problem. Both problems can be addressed through a full physics method, particularly applying the Optimal Estimation (OE) approach—a specialized Tikhonov regularization scheme based on Bayesian formulation. However, the computational demands of a full physics method hinder Near Real-Time (NRT) data analysis. Faster models become imperative for next-generation satellites measuring hundreds of spectra per minute and climatology models simulating years of global-scale radiative transfer.
To expedite solutions for both problems, a hybrid approach is employed, combining an a priori regularized data-driven method utilizing the Moore-Penrose pseudoinverse and a neural network approach.

How to cite: Sgattoni, C., Chung, M., and Sgheri, L.: Advancing Atmospheric Retrieval: A Rapid Physics-Informed Data-Driven Approach using FORUM Simulated Measurements, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-9820, https://doi.org/10.5194/egusphere-egu24-9820, 2024.

This research proposes an optimal design framework for a mesoscale atmospheric greenhouse gas network dedicated to inverse flux monitoring at urban, regional, or national scales.

The framework’s design is based on data processing of atmospheric concentrations using multiple machine learning techniques such as image processing and pattern recognition, among others, all of them powered by optimization algorithms, giving the solution process explorative and exploitative features over the problem search space. 

Besides, the data processing uses graph representation as it considers a discrete search space, which in turn allows for speeding up the information access in each stage, especially during the inverse analysis procedure.

All of the above is framed with a learning system whose purpose is automatizing the processing when combining diverse data sources by mixing the supervised and unsupervised learning types in pre- and post-processing, respectively. 

On the one hand, the problem is related to the design of a monitoring network of greenhouse gases, in which it is required to decide the locations of a specific number of towers according to their measurement influence region, hence minimizing the number of towers while guaranteeing the appropriate parameter estimation.

On the other hand, the solution strategy conducts a data analysis, where observed and fitted data are treated as spatial-temporal images. During the batch processing, these images are filtered, contrasted, binarized, classified, and clustered, among other operations to maximize the data analysis.

Performance tests were based on reference datasets from the Weather Research and Forecasting model (here hourly simulated concentrations at 3 kilometers resolution over eastern France) as well as other synthetically and randomly generated concentration fields, which allowed for comparison of the proposed algorithm processing.

According to the parametric and non-parametric tests used to evaluate the scheme, our framework is competitively capable of designing optimal monitoring networks by using data processing and high-performance computing.

How to cite: Matajira-Rueda, D., Maiwald, R., Abdallah, C., Vardag, S., Butz, A., and Lauvaux, T.: A novel automated framework to design optimal networks of atmospheric greenhouse gas stations, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-18552, https://doi.org/10.5194/egusphere-egu24-18552, 2024.

The chemical transport models face challenges in simulating the concentrations of surface ozone accurately in all conditions when meteorology and chemical environment are changing. The capability of capturing the principle physical and chemical processes is clearly limited. We propose a unified framework based on deep learning to provide a more accurate prediction of surface ozone. The model is tailored to individual observation sites in China, forming a specific graph that would reflect the interaction between spatial and temporal connection in physics and chemistry. This mitigates the uncertainty associated with model resolution and emissions. We show that the model achieves the State-of-the-Art (SOTA) performance in simulating MDA8 ozone among current process-based and other deep learning models. The model structure is also flexible to be applied to other places where observations are available such as Europe and North America. This work underscores great benefits that can be gained through implementing more measurement sites to enhance the density of the model graph.

How to cite: Liu, Z., Li, K., Wild, O., Doherty, R., O'Connor, F., and Turnock, S.: Unified Model of Forecasting Ozone, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-4501, https://doi.org/10.5194/egusphere-egu24-4501, 2024.

Mexico City due to its specific topography and strong ozone precursors emissions often faces high surface ozone concentrations which negatively impact the dwellers and the environment of Mexico City. This necessitates developing models with the capacity to rank meteorological and air quality variables contributing to the build-up of ozone during an ozone episode in Mexico City. Such ranking is crucial for regulatory procedures aiming at reducing ozone detrimental effects during an ozone episode. In this study, three machine learning models (Random Forest, Gradient Boosting Tree, feedforward neural network) are used to learn a prediction function that reveals the functional dependence of ozone on its predictors and can predict hourly ozone concentrations using hourly data of eight predictors (nitric oxide, nitrogen dioxide, shortwave ultraviolet-A radiation, wind direction, wind speed, relative humidity, ambient surface temperature, planetary boundary layer height). The best model, feedforward neural network with 92% accuracy, in conjunction with Shapely Additive exPlanations approach, is utilized to simulate high ozone concentrations and rank the predictors according to their importance in the build-up of ozone during a severe ozone smog episode that occurred in the period 6 - 18 March 2016. The research focuses on Mexico City, but it is equally applicable to any other city in the world.

How to cite: Ahmad, M., Rappenglück, B., O. Osibanjo, O., and Retama, A.: Application of Three Machine Learning Models for a Severe Ozone Episode in Mexico City , EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-21203, https://doi.org/10.5194/egusphere-egu24-21203, 2024.

Nitrogen oxides (NO_x = NO + NO₂) are of great concern due to their impact on human health and the environment. Machine learning (ML) techniques are increasingly employed for surface NO₂ estimation following fast-paced developments in artificial intelligence, computational power, and big data management. However, the uncertainties inherent in these retrievals are critical but are rarely studied in the rapid expansion of ML applications in atmospheric research.

In this study, we have developed a novel ML framework enhanced with uncertainty quantification techniques, named Boosting Ensemble Conformal Quantile Estimator (BEnCQE), to estimate surface NO₂ and assess the corresponding uncertainty arising from data. Quantifying such data-induced uncertainty is essential for ML applications as the ML models are data-driven. We apply the BEnCQE model with multi-source data to infer surface NO₂ concentrations over Western Europe at the daily scale and 1 km spatial resolution, from May 2018 to December 2021. The space-based cross-validation with in-situ station measurements shows that our model achieves accurate point estimates (r = 0.8, R² = 0.64, root mean square error = 8.08 ug/m3) and reliable prediction intervals (coverage probability, PI-66%: 66.4%, PI-90%: 90.4%). The model result is also in good agreement with the Copernicus Atmosphere Monitoring Service (CAMS) model output. Furthermore, the quantile estimation strategy used in our model enables us to understand the variations in the predictors’ importance for different NO₂ level estimates. Additionally, integrating uncertainty information can uncover potential exceedances of the World Health Organization (WHO) 2021 NO₂ limits in some locations, an exceedance risk that point estimates alone may fail to fully capture. Meanwhile, uncertainty quantification, by providing information on the uncertainty of each estimate, allows us to assess the robustness of the model outside of existing in-situ station measurements. The variations in uncertainty suggest that the model's robustness is related to conflicts between seasonal and spatial NO₂ patterns influenced by multi-source data. It also reveals challenges in urban and mountainous areas where NO₂ is highly variable and heterogeneously distributed.

How to cite: Sun, W., Tack, F., Clarisse, L., Schneider, R., Stavrakou, T., and Van Roozendael, M.: Inferring Surface NO2 over Western Europe: A Machine Learning Approach with Uncertainty Quantification, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-7936, https://doi.org/10.5194/egusphere-egu24-7936, 2024.

It is well known that degraded air quality affects human health and the surrounding environment. Air quality is related to emissions from various sources, meteorological (weather) conditions, topography, and vegetation. Weather conditions such as temperature, humidity, dew point, wind speed and precipitation vary significantly over the year and have the potential to affect the formation, transport, and dispersion of air pollutants. Therefore, it is important to understand the correlation between the seasonal variations of weather conditions on the air quality and be able to forecast it with reasonable accuracy. The aim of this study is to apply Artificial intelligence (AI) technique to investigate such correlation. For this purpose, four (4) AI algorithms namely: neural networks (NN), Decision tree (DT), Random forest (RF) and Gradient boosting (GB) have been applied to assess the correlation between Nitrous Oxide (NO₂) and seasonal variations of weather conditions (i.e. temperature, humidity, wind speed, wind direction, and pressure). NO₂ was selected as a target air pollutant which considered a serious air quality parameter and one of the greenhouse gases. The effect of seasonal variations of weather conditions on the air quality parameters was presented by the date of measurement as a parameter or feature. Air quality data were collected for the period between 2017 and 2021 from a local air monitoring station, while weather conditions were obtained from the weather station at the airport located the Eastern region of Saudi Arabia. The accuracy of the correlation between was tested using mean square error (MSE), mean root square error (MRSE), mean absolute error (MAE) and correlation coefficient (R²). Results of the study revealed a strong association between NO₂ levels and seasonal variations of weather conditions. The MEA ranges between 1.765 to 1.439 using NN, DT, RF and GB respectively.  The correlation coefficient (R²) ranges between 0.564 to 0.826 using NN, DT, RF and GB respectively. The results showed that GB algorithm generated better correlation for NO₂ compared to other algorithms. The study results can be used for better predicting air quality for NO₂ that can be used for the assessment of potential global warming and climate change phenomena

How to cite: Tawabini, B.: The Association Between Nitrous Oxide (NO2) Levels and Seasonal Variations of Weather Conditions Using Artificial Intelligence , EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-4279, https://doi.org/10.5194/egusphere-egu24-4279, 2024.

Volatile organic compounds (VOCs) play a crucial role in atmospheric chemistry, influencing global climate and posing potential health risks to humans. Accurate spatiotemporal estimation of VOCs is vital for establishing advanced early warning systems and controlling air pollution. However, research on high-resolution spatiotemporal prediction of VOCs concentrations using machine learning is still limited. This study conducted an extensive VOCs observational campaign in Shanghai, improving upon the LightGBM model with the integration of spatiotemporal information, satellite data, meteorological data, emission inventories, and geographical data for VOCs estimation. We achieved a high-precision distribution map of VOCs concentrations in Shanghai (1 km, 1 hour resolution), demonstrating the model’s excellent hourly VOCs estimation performance (R^2 = 0.92). Further analysis with SHapley Additive exPlanations (SHAP) regression revealed the significant contributions of each input feature to VOCs estimation. Compared to many traditional machine learning models, this approach offers lower computational demands in terms of speed and memory. Moreover, the model maintained good hourly spatiotemporal VOCs prediction performance during the COVID-19 lockdown. This research analyzed the spatiotemporal variations of VOCs concentrations in Shanghai, providing a scientific basis for future control of VOCs levels in the city and offering algorithmic support for comprehensive VOCs prediction in other regions.

How to cite: Lu, B. and Li, X.: High-resolution mapping of  VOCs using the fast space-time Light Gradient Boosting Machine (LightGBM), EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-331, https://doi.org/10.5194/egusphere-egu24-331, 2024.

Reducing aerosol mass loading requires targeted control of emissions from anthropogenic sources. Accurately tracking the changes in emission strengths of specific aerosol sources is vital for assessing the effectiveness of regulatory policies. However, this task is challenging due to meteorological influences and the presence of multiple co-existing emissions. Using multi-year data on ambient black carbon (BC) and PM2.5 from Tianjin, China, as a case study, we employed a data-driven approach that integrates a dispersion-normalized factor analysis receptor model with a machine learning technique for meteorological normalization. This approach enabled us to differentiate between the emission sources of BC and PM2.5 and their meteorological impacts. The source-specific aerosol exhibited abrupt changes in response to human-made interventions, such as those during COVID-19 and holiday periods, after accounting for weather-related variables. Notably, significant reductions were observed in emissions from coal combustion, vehicles, dust, and biomass burning over years, affirming the effectiveness of policies such as clean winter heating initiatives and the support for the Clean Air Actions. This coupled approach holds significant promise for advancing air quality accountability studies.

How to cite: Dai, Q., Dai, T., Bi, X., Wu, J., Zhang, Y., and Feng, Y.: Tracking changes in the emission strengths of source-specific aerosols by coupling a receptor model with machine learning, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-10670, https://doi.org/10.5194/egusphere-egu24-10670, 2024.

The atmosphere is governed by laws of atmospheric physics and chemistry. For decades even centuries, these laws are represented by differential equations, usually solved numerically for the scale defined by model grid cells. However, this representation paradigm reaches its limits when the underlying physics or chemistry is too complex or even unknown, especially when considering the multiscale nature of the atmosphere. In our data era these laws are stored in immense datasets of various observations and numerical simulations, while artificial intelligence techniques can retrieve them from data albeit often with limited physical interpretations. Hybrid modeling [1] can thus balance the physical modeling and the data-driven modeling for a more comprehensive representation of the atmosphere. By representation learning, the multiscale features of the atmosphere can be learnt and encoded in the weights of connected neurons in the deep networks of multiple layers, as is fundamentally different from the traditional atmospheric representation using formulae and equations. Here we elaborate this new deep learning-based representation paradigm with two demonstrating cases. In the first case [2], we reveal a multiscale representation of the convective atmosphere by reconstructing the radar echoes from the Weather Research and Forecasting (WRF) model simulations and the Himawari-8 satellite products using U-Net deep networks. We then diagnose the physical interpretations of the learnt representation with a sensitivity analysis method. We find stratified features with small-scale patterns such as echo intensities sensitive to the WRF-simulated dynamic and thermodynamic variables and with larger-scale information about shapes and locations mainly captured from satellite images. Such explainable representation of the convective atmosphere would inspire innovative data assimilation methods and hybrid models that could overcome the conventional limits of nowcasting. In the second case [3], we employ deep convolutional neural networks (CNN) to represent the errors associated with fine particulate matter (PM_2.5) forecasts of a chemistry-transport model (CTM), the Nested Air Quality Prediction Modeling System (NAQPMS), within 240-hour lead times across 180 monitoring sites in the Yangtze River Delta (YRD) region of China. The learnt multiscale error representation reduces the PM_2.5 forecasts’ root mean square error (RMSE) by 16.3-34.2% on test cases in 2017-2018. We then probe the physical interpretation of the multiscale error representation using the deep learning important features (DeepLIFT) interpretability method. We quantify the significant contribution from sulfur dioxide (SO₂, 31.3%) and ozone (29.4%), which are comparable to PM_2.5 (31.1%) and about three times higher than nitrogen dioxide (8.2%). Such interpretations would suggest that improvements are needed in formulating the SO₂-sensitive pollution in the ammonia-poor YRD region. We consider our representation studies as a step towards more comprehensive atmospheric hybrid models that take advantage of the mighty artificial intelligence technologies but are at the same time physically explainable.

[1] Liao, Q., Zhu, M., Wu, L. et al. 2020. https://doi.org/10.1007/s40726-020-00159-z

[2] Zhu, M., Liao, Q., Wu, L. et al. 2023. https://doi.org/10.3390/rs15143466

[3] Zhu, M., Liao, Q., Wu, L. et al. 2024. In submission.

How to cite: Zhu, M., Wu, L., Su, H., and Wang, Z.: Representing the atmosphere using deep learning techniques: applications in radar echo data reconstruction and PM2.5 forecast error reduction, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-13977, https://doi.org/10.5194/egusphere-egu24-13977, 2024.

Understanding the dynamics and characteristics of emission plumes from wildfires is of paramount importance for environmental monitoring and policy decisions. These plumes, composed of various greenhouse gases and pollutants, can have far-reaching consequences on global climate, air quality and health. In this study, a data-driven approach to detect and characterise emission plumes from wildfires utilising TROPOMI (Tropospheric Monitoring Instrument) of the Sentinel-5p satellite observations of nitrogen oxides (NOx), carbon monoxide (CO) and aerosols. The analysis leverages VIIRS active fire data to identify locations of fire occurrence, laying the foundation for plume detection. The primary hypothesis states that a plume image consists of three components: a plume body or core, a transitional zone from plume to clear sky, and the clear sky itself. To realise this hypothesis, a data-driven unsupervised algorithm to identify and map plumes is developed, which is based on kernel functions to pre-process the Sentinel-5p images. These kernels effectively highlight plume-related features, allowing for more precise delineation. Subsequently, Gaussian Mixture Models (GMM) are utilised to classify the images into three components of the plume according to the main hypothesis. In instances where multiple plume candidates exist, a Gaussian distance weighting function to identify the likeliest plume is employed. Furthermore, the mapping of the plume-clean air transition zones is further evaluated by employing Monte Carlo simulations to validate and refine the transition zone assessments. To verify the detections, plumes of methane (CH4), carbon monoxide (CO), formaldehyde (HCHO), nitrogen dioxide (NO2) and aerosols for several plumes over the Amazon and Alberta are extracted and the plume properties are related to different landcover types. The findings of this study provide valuable insights into the development of an advanced methodology for plume detection, which has broad implications for the understanding and monitoring of fire emissions and atmospheric research.

How to cite: Kinalczyk, D., Forkel, M., and de Laat, J.: Identification and description of fire emission plumes from Sentinel-5p observations   , EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-15297, https://doi.org/10.5194/egusphere-egu24-15297, 2024.

The knowledge of the surface reflectance is essential for the retrieval of atmospheric trace-gases from satellites. It is required in the conversion of the observed trace gas slant column to the total vertical column by means of a so-called air-mass factor. Although there exists climatological databases based on UV satellite data (e.g. OMI, GOME-2), these have a low spatial resolution and are not appropriate for current and future UV satellite missions like Sentinel-5p/TROPOMI or MTG-S/UVN (Sentinel-4) due their significantly higher spatial and spectral resolution. Current climatologies which are used in operational retrievals provide the Lambertian Equivalent Reflection (LER, e.g. OMI, GOME-2, TROPOMI, see [1,2,3]) and Directional-LER (DLER, e.g. GOME-2, TROPOMI see [3,4]) for selected wavelength in the UV-VIS range and are based on the so-called minimum LER approach, i.e. determine the minimum surface reflectance in the measurement timeframe.

We present here a new technique called GE_LER (Geometry-dependent Effective Lambertian Equivalent Reflectivity) based on Machine Learning, which retrieves the DLER from UV satellites in a wavelength range as opposed to the single wavelength approaches of existing climatologies. In this way, dedicated surface reflectivities for specific trace gas retrieval wavelength ranges can be determined. We train a Neural Network with simulated UV spectra, which have been calculated with (V)LIDORT (see [5]). This radiative transfer model is also used for the generation of Air Mass Factors in the operational TROPOMI trace gas retrieval. In this way we reduce the influence of using different radiative transfer models with respect to trace gas retrievals.

First results of our GE_LER retrieval for several trace-gases based on TROPOMI data will be shown.

References

[1] Kleipool (2010), OMI/Aura Surface Reflectance Climatology L3 Global Gridded 0.5 degree x 0.5 degree V3, Greenbelt, MD, USA, Goddard Earth Sciences Data and Information Services Center (GES DISC),, 10.5067/Aura/OMI/DATA3006

[2] Tilstra et al. (2017), Surface reflectivity climatologies from UV to NIR determined from Earth observations by GOME-2 and SCIAMACHY, J. Geophys. Res. Atmos. 122, 4084-4111, doi:10.1002/2016JD025940

[3] Tilstra et al. (2021), Directionally dependent Lambertian-equivalent reflectivity (DLER) of the Earth's surface measured by the GOME-2 satellite instruments, Atmos. Meas. Tech. 14, 4219-4238, doi:10.5194/amt-14-4219-2021

[4] Tilstra et al. (2023), A directional surface reflectance climatology determined from TROPOMI observations, Atmos. Meas. Tech. Discuss. [preprint], doi:10.5194/amt-2023-222, in review

[4] Spurr et al. (2008), LIDORT and VLIDORT: Linearized pseudo-spherical scalar and vector discrete ordinate radiative transfer models for use in remote sensing retrieval problems. Light Scattering Reviews, Volume 3, ed. A. Kokhanovsky, Springer

How to cite: Hedelt, P., Heue, K.-P., Lutz, R., Romahn, F., and Loyola, D.: Innovative surface reflectance retrieval from UV satellites, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-5695, https://doi.org/10.5194/egusphere-egu24-5695, 2024.

Black carbon (BC) plays an important role in air quality, public health, and climate, while its long-term variations in emissions and health effect were insufficiently understood for China. Here, we present the spatiotemporal evolution of BC emissions and the associated premature mortality in China during 2000-2020 based on an integrated framework combining satellite observations, a machine learning technique, a “top-down” inversion approach, and an exposure-response model. We found that the “bottom-up” approach likely underestimated BC emissions, particularly in less developed western and remote areas. Pollution controls were estimated to reduce the annual BC emissions by 26% during 2010-2020, reversing the 8% growth during 2000-2010. BC emissions in the main coal-producing provinces declined by 2010 but rebounded afterwards. By contrast, provinces with higher economic and urbanization levels experienced emission growth (0.05-0.10 Mg/km²/yr) by 2010 and declined greatly (0.07-0.23 Mg/km²/yr) during 2010-2020. The national annual BC-associated premature mortality ranged between 733,910 (95% confidence interval: 676,790-800,250) and 937,980 cases (864,510-1,023,400) for different years. The changing BC emissions contributed 78,590 cases (72,520-85,600) growth within 2000-2005 and 133,360 (123,150-145,180) reduction within 2010-2015. Strategies differentiated by region are needed for further reducing BC emissions and its health and climate impacts.

How to cite: Zhao, W. and Zhao, Y.: Long-term Variability in Black Carbon Emissions Constrained by Gap-filled Absorption Aerosol Optical Depth and Associated Premature Mortality in China, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-7393, https://doi.org/10.5194/egusphere-egu24-7393, 2024.

Machine learning (ML) techniques have been extensively applied in the field of atmospheric science. It provides an efficient way of integrating data and predicting atmospheric compositions. However, whether ML predictions can be extrapolated to different domains with significant spatial and temporal discrepancies is still unclear. Here we explore the answer to this question by presenting a comparative analysis of surface carbon monoxide (CO) and ozone (O₃) predictions by integrating deep learning (DL) and chemical transport model (CTM) methods. The DL model trained with surface CO observations in China in 2015-2018 exhibited good spatial and temporal extrapolation capabilities, i.e., good surface daily CO predictions in China in 2019-2020 and over 10% independent observation stations in China in 2015-2020. The spatial and temporal extrapolation capabilities of DL model are further evaluated by predicting hourly surface O₃ concentrations in China, the United States (US) and Europe in 2015-2022 with a DL model trained with surface O₃ observations in China and the US in 2015-2018. Compared to baseline O₃ simulations using GEOS-Chem (GC) model, our analysis exhibits mean biases of 2.6 and 4.8 µg/m³ with correlation coefficients of 0.94 and 0.93 (DL); and mean biases of 3.7 and 5.4 µg/m³ with correlation coefficients of 0.95 and 0.92 (GC) in Europe in 2015-2018 and 2019-2022, respectively. This analysis indicates the potential of DL to make reliable atmospheric composition predictions over spatial and temporal domains where a wealth of local observations for training is not available.

How to cite: Han, W., He, T.-L., Jiang, Z., Wang, M., Jones, D., Miyazaki, K., and Shen, Y.: The capability of deep learning model to predict atmospheric compositions across spatial and temporal domains, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-7133, https://doi.org/10.5194/egusphere-egu24-7133, 2024.

The air quality model is increasingly important in air pollution forecasting and controlling. Emissions significantly impact the accuracy of air quality models. This research studied the 3DVar (three-dimensional variational) emission inversion method based on machine learning in CMAQ (The Community Multiscale Air Quality modeling system). The ExRT(extremely randomized trees method) machine learning conversion matrixes were established to convert the pollutant concentration innovations to the corresponding emission intensity innovations, extended 3DVar to emission inversion. The O₃ and NO₂ concentration, NO_x and VOCs emissions are modeled using machine learning, taking account of the nonlinearity of the O₃-NO_x-VOCs processes. This method significantly improved the simulation ability of O₃. Taking the air pollution process in the BTH region from January 15 to 30, 2019 as an example, ExRT-3DVar (3DEx) and Nudging (Nud) emission assimilation experiments were caried out. Compared with the simulation without assimilation (NODA), the Nudging method has better assimilation effects on PM₁₀ and NO₂, with the regional errors reduced by 14%, 2%, and the temporal errors reduced by 31%, 34%; ExRT-3DVar has better effects on the assimilation of PM_2.5, O₃, SO₂, the regional errors were reduced by 40%, 29%, 13%, and the temporal errors were reduced by 49%, 10%, 33%. This simplicity, efficiently and extensibility framework of ExRT-3DVar method has been proved to be a good way to adjust emissions in CMAQ and still remains much to be done in the future.

How to cite: Huang, C., Wang, T., and Niu, T.: Study on the 3DVar emission inversion method combined with machine learning in CMAQ, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-2763, https://doi.org/10.5194/egusphere-egu24-2763, 2024.

Air pollution poses a substantial risk to both public health and the environment. Accurate forecasting of air quality is crucial in mitigating its detrimental impacts. The existing forecast method of air quality in India is computationally intensive and is not economical; hence, we utilize Advanced Machine and Deep Learning Models to forecast air quality. The objective of this research is to develop a novel hybrid model integrating Long Short-Term Memory (LSTM), Extreme Gradient Boosting (XGBoost), and Multi-Layer Perceptron (MLP) models to forecast concentrations over Kanpur. The study involved comprehensive data collection (Secondary air quality and meteorological data from the Central Pollution Control Board), analysis, and experimentation with multiple models. Root mean square error (RMSE) and coefficient of determination (R² score) are used for model validation. MLP-XGBoost-LSTM hybrid model works well with a decreased RMSE (12.6 μg/m³ ) and increased R² score (0.96) compared to individual models (XGBoost- 37 μg/m³, MLP-39 μg/m³, and LSTM-41 μg/m³).  The significance of the research lies in its potential to provide highly accurate forecasts, even with limited computational resources. These findings have significant implications for environmental policy, public health in heavily polluted regions, and the broader utilization of machine learning in environmental science.

How to cite: k k, V., Verma, S., and Vaibhav, V.:   PM2.5 concentration forecast using Hybrid models over Urban cities in India, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-5491, https://doi.org/10.5194/egusphere-egu24-5491, 2024.

The occurrence of extreme wind events poses a significant threat to human populations, putting lives at risk and causing substantial damage to vital infrastructures. Coastal regions, in particular, face heightened vulnerability due to the unpredictable nature of the land-sea interface, presenting a formidable   challenge for accurate modeling. Thus, there is an urgent need for robust and efficient predictive techniques to anticipate and manage the impact of these severe wind phenomena. In response to this imperative, our study explores the application of an innovative machine learning forecasting methodology tailored for extreme winds, specifically focusing on the Mediterranean west coast in the Valencian community. Our approach involves analysis of historical meteorological station data from the Spanish meteorological agency. This data, combined with an extensive set of reanalysis data spanning from 1961 to 2019, is utilized for the identification and classification of extreme wind events. Employing a train-test procedure, we implemented a Random Forest binary classification model, enabling successful forecasting of extreme wind episodes up to 48h in advance. Notably, the precision of our model exhibited a remarkable range between 73% and 92%, varying with the lead-time across the considered regions. This methodology not only enhances forecasting capabilities but also provides insights into the intricate dependencies of meteorological variables, thereby advancing our understanding of complex atmospheric processes. This pioneering study, driven by artificial intelligence, contributes to the ongoing exploration of the complex dynamics of winds in coastal regions. The insights gained from our research extend beyond the Mediterranean west coast and have the potential for broader applicability  in other coastal areas. The results underscore the pivotal role of adaptive strategies in mitigating the impact of extreme weather events, emphasizing the importance of proactive measures in the face of escalating climate-related challenges.

How to cite: Guerrero-Navarro, G. H., Martínez-Amaya, J., and Nieves, V.: Machine Learning Forecasting of Extreme Winds: A Study on the Mediterranean West Coast in the Valencian Community, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-150, https://doi.org/10.5194/egusphere-egu24-150, 2024.

The occurrence and passage of synoptic-scale systems modulate the local boundary-layer (BL) profile. In the Eastern Mediterranean (EM), a detailed clustering of the winter profiles over Beit-Dagan, at the Israeli central coastal plain, showed direct links to winter highs, lows and Red Sea troughs and further enabled the identification of the active RST, a longstanding challenge to objectively identify.

Since high resolution radiosondes profile data at Beit Dagan is available for the recent 20 years, and sometimes suffers lack of data, its application as synoptic tool is limited. Objective synoptic classification during longer periods is needed.

Our research  investigates the synoptic regimes according to upper tropospheric PV during the winter months (DJF 2011-2021). We utilize the self-organizing map (SOM) clustering method and the ERA5 reanalysis data to achieve this aim. Various domains, SOM parameters, quantization and topographical errors, standard deviations of each SOM class, and gradual size of maps were tuned and inspected respectively to select the final map. The synoptic regimes are later related to the boundary layer profile variability. The relation between the PV classes and the variability of the BL profile is found according to the frequencies of the PV classes under each BL profile class.

The ageostrophic balance next to the surface effect the BL profile. To include this important factor, extended synoptic classification, according to multi variable clustering of PV and 1000 hPa geopotential height (gph) was devised. SOM training and projection on the BL profile classes were accordingly preformed.

SOM clustering of 320K isentropic surface potential vorticity (PV) data presented 4X4 classes. Two PV classes relate to high PV (> 2 PVU) over the EM: the first presents wide northerly trough and the second a thinner trough with a north-easterly axis towards Israel due to anticyclonic shear. Most of the other classes present low PV values (< 2 PVU) over the EM relating to southerly wide ridge or anticyclonic wave breaking propagating to the east of the EM. Strong or weak PV activity over the EM is related to some of the BL profile classes (few classes with relatively high frequency (> 20%)). Under mild PV activity which is related to mild surface pressure gradients, no strong relation is found.

Multi-variable SOM clustering of gph and PV presented 4X5 classes which follow the variability of surface winter lows, highs and active Red Sea troughs. The active Red Sea trough relates to the north easterly relatively narrow PV stream. The main PV classes of the 4X4 single variable SOM classification resemble those of the combined (PV + gph) classification. The multi-variable clustering somewhat improves the indication of the BL profile classes. Better indication between BL profile pattern and strong winter highs is obtained.

This work suggests a new approach to inspect the co variability of synoptic regimes over the EM with various meteorological variables (beyond the BL profile) including examination of trends and persistence of each synoptic regime.

How to cite: Berkovic, S., Schloss, R., and Raveh-Rubin, S.: Potential-vorticity regimes over the Eastern Mediterranean and their relation to local boundary layer profiles, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-13317, https://doi.org/10.5194/egusphere-egu24-13317, 2024.