Evaluating machine learning models for ozone pollution forecasting

As global average temperatures rise, so too are wildfires projected to grow in size and frequency. This will cause an increase in wildfire-induced pollution, including ozone, a combustion byproduct, which is detrimental to human health. Accurate forecasts of air pollution are critical to provide early warnings to vulnerable communities, and in recent years different types of machine learning (ML) models have been created to predict the movement of pollutants in the atmosphere. However, there is little consensus about which type of model performs best, and very few studies consider the relationship between wildfires and ozone. We created several ML models to forecast tropospheric ozone concentrations over Africa between 2018 and 2022. Based on ML pollution forecasting literature, we chose to evaluate the Gradient Boosting Machine, Random Forest (RF), dense neural network (NN), convolutional NN, long-short term memory (LSTM) NN and Transformer NN models. Their inputs were daily wildfire activity, previous ozone concentration, wind speed/direction, and temperature; their output was the daily tropospheric ozone concentration. We evaluated the models’ forecasts using three metrics: mean-squared-error (MSE), ability to match the spatial heterogeneity of ozone concentrations in the target data, and correctly identifying ozone hotspots—concentrations above the 99th percentile. A convolutional NN coupled with a Transformer performed best overall, the RF was second-best, and the LSTM performed worst overall according to our metrics. To quantify how useful information about wildfires is to the accuracy of the forecasts, we removed fire from the training data and retrained and reevaluated all models. The results were inconsistent, and averaged across all models they were negligible: -0.955% MSE, +0.671% spatial variability mismatch, and +0.168% hotspot accuracy. We found a positive correlation (0.286) between daily wildfire activity and ozone concentrations and evidence that wildfire-produced ozone is consistently transported from East-to-West by wind. Our results show that convolutional-based models and the RF can and do accurately forecast ozone concentrations, and they outperform many other commonly used ML models used in similar domains.