Exploring Explainability for Graph-Based Weather Forecasting Models Using Layer-Wise Relevance Propagation

Machine Learning (ML) models, particularly deep neural networks, are often seen as black boxes, offering limited insight into how their predictions are made. This lack of transparency becomes especially important when ML is applied to critical domains such as numerical weather prediction, where traditional models are based on physical laws and differential equations.

Explainable AI (XAI) methods aim to address the black-box behavior by providing tools to interpret and understand model decisions. One such method is Layer-Wise Relevance Propagation (LRP), which traces the output of a neural network backward to assign relevance scores to input features based on their contribution to the prediction.

LRP has since been extended to Graph Neural Networks (GNNs) through the introduction of relevant walks, enabling interpretability in graph-structured data (GNN-LRP). These extensions have shown promise in areas such as image classification, sentiment analysis, and quantum chemistry. At the German National Weather Service (DWD), the AICON forecasting model employs a GNN architecture with message passing, similar in design to the GraphCast model.

In this work, we present an initial exploration of applying GNN-LRP to a simplified, toy version of a GNN model used as a representative of the AICON model. We investigate both saliency map-like visualizations and relevance walks, aiming to identify the most influential input features and their geographical location. While the current results are preliminary and limited in scope, this study tries to lay the groundwork for potential further research into explainability in graph-based weather prediction models.