Modeling the Variability of Terrestrial Carbon Fluxes using Transformers

The terrestrial carbon cycle is one of the largest sources of uncertainty in climate projections. The terrestrial carbon sink which removes a quarter of anthropogenic CO2 emissions; is highly variable in time and space depending on climate. Previous studies have found that data-driven models such as random forest, artificial neural networks and long short-term memory networks can be used to accurately model Net Ecosystem Exchange (NEE) and Gross Primary Productivity (GPP) accurately, which are two important metrics to quantify the direction and magnitude of CO2 transfer between the land surface and the atmosphere. Recently, a new class of machine learning models called transformers have gained widespread attention in natural language processing tasks due to their ability to learn from large volumes of sequential data. In this work, we use Transformers to model NEE and GPP from 1996-2022 at 39 Flux stations in the ICOS Europe network using ERA5 reanalysis data. We can compare our results with traditional machine learning approaches to evaluate the generalisability and predictive performance of transformers for carbon flux modelling.