PlantTraitNet: A Multi-Modal, Multi-Task Approach to Learning Global Plant Trait Patterns Using Citizen Science Data and Noisy Labels

As primary producers in Earth's system, plants drive global matter and energy fluxes. Understanding the global distribution of plant functional traits and their biodiversity is, therefore, critical for understanding ecosystem behavior and Earth system dynamics in the face of climate and global change. However, we lack observations for various plant functional traits, such as plant height, leaf size, and nitrogen content, at a global scale.

These data gaps could be addressed through citizen science projects, where thousands of individuals have already recorded millions of plant photographs for species identification purposes. While these photographs do not include direct information about plant traits, trait data for thousands of plant species can be accessed from scientific databases. By linking these two data sources—crowd-sourced plant photographs and trait information from scientific databases—through plant species, we can supervise computer vision models to infer plant traits from plant images. The principle of "form follows function" suggests that a plant's appearance can provide valuable insights into its functional properties.

To assess the potential of citizen science data for plant trait estimation, we propose testing the feasibility of using weak and noisy labels for effective trait prediction. Considering that different plant traits are not independent of each other, we leverage multi-trait learning. Additionally, our approach incorporates plant images along with ancillary environmental data, such as soil conditions and Earth observation satellite data, to provide crucial context on factors like climate or land surface properties.

To fairly evaluate model performance, we curate a clean dataset spanning diverse geographic regions, as well as taxonomic and phylogenetic groups. We conduct a comprehensive study on the resilience of trained models across these distribution shifts. Furthermore, we assess which traits can be effectively learned from noisy labels and explore the extent of trait transferability under different conditions.

Our findings indicate that models trained on noisy data can, to a notable extent, predict a series of plant traits, including plant height, leaf area, and specific leaf area. This approach provides an efficient, scalable, and non-destructive method for estimating important plant functional traits. It could lay the groundwork for large-scale biodiversity monitoring and ecosystem assessment, with the potential to revolutionize how we track the functional properties of ecosystems at a global scale.