An Inherently-Interpretable Approach to Uncover the Head Importance of Attention Networks

Attention-based deep learning models are becoming a ubiquitous approach for modeling the complex temporal dependencies in many vital Earth observation applications, such as agricultural monitoring. They typically consist of multiple attention heads, with each head containing attention weights that determine how temporal information is combined for the model's prediction. While analyzing the attention weights can provide insights into the model's workings, the existence of multiple heads makes it difficult to comprehend the extracted temporal information by the model. To overcome this issue, we propose an inherently interpretable approach that automatically weights the head importance during the model's forward pass. Our evaluation on the task of crop-type classification shows that the model maintains high accuracy while simplifying interpretation by highlighting only the most significant attention heads.