PHENOMENA: a modular HPC model to facilitate automatic high-resolution greenhouse gas emission monitoring

This work presents the sPanisH EmissioN mOnitoring systeM for grEeNhouse gAses (PHENOMENA), a python-based, open-source, multiscale emission model that computes high resolution (up to 1km2 and daily) and low latency greenhouse gas (GHG) emissions for Spain. The system uses a bottom-up approach, based on emission factors and activity data, and consists of four different modules: First, the downloading module retrieves low latency activity data from multiple sources, including APIs, open data repositories, websites, and private providers, with error handling and automatic retrials to minimize manual intervention. Next, the preprocessing module standardizes the data and applies quality-control checks. The activity data is then combined with emission factors in the calculation module, which covers 11 emission sectors. Finally, the resulting emissions are post-processed to meet the requirements of an open web platform where the results are displayed.

PHENOMENA is based on the OOP paradigm and designed to run on High Performance Computing (HPC) infrastructures. While each one of the emission sectors can run in parallel using MPI strategies, it is still not feasible to run all of them at the same time or download all the activity data at once, as different data providers have different temporal availability. Thanks to the modularity of the system, it can be split into different HPC jobs to handle the heterogeneous data frequencies, increase robustness through automatic retrials, run different instances at the same time and automatize monthly uploads to the web portal, using the Autosubmit workflow manager.

The resulting product is a web app which provides daily 1 km x 1 km gridded emission maps and emission totals aggregated per region and sector. The system's latency is determined by the availability of the activity data from external providers, ranging from daily updates to delays of up to four months.

PHENOMENA allows monitoring low-latency GHG emissions for Spain at high temporal and spatial resolution, providing information in an accessible way to support national to local policymakers. The system is scalable, robust against failures, and easily adaptable to new data providers, regions and emission sectors.