Foundation models (FMs) have shown great promise for image and text-based tasks, including applications to Earth Observation (EO) satellite imagery. However, with more and more models being published, model inter-comparison is becoming increasingly challenging. This session seeks to highlight works focussed on benchmarking and best practices for using pre-trained models for diverse tasks across EO, Weather and Climate research. We invite submissions focussed on creating geospatially-aware FMs, e.g. multi-modality, multi-temporality, multi-resolution and sensor independence, that enable new scientific understanding within their respective research domains.
To foster discussions on current applications, challenges and opportunities of FMs for EO, Weather and Climate, we encourage submissions from AI and domain researchers, climate modellers, industry experts and stakeholders from the AI4EO, High-Performance Computing (HPC) and Big Data communities.
The topics of the session are:
Benchmarking and Evaluating FMs: Establishing standardised fair evaluation metrics and benchmarks to assess the performance and capabilities of FMs in processing data, ensuring reliability and efficiency.
Best Practices: Guidelines for using existing pre-trained models for diverse applications, with a specific focus on how to decide which models are best for certain use cases.
Sensor independence: FMs can process data from various sensors, including multi- or hyper-spectral, SAR, Very High Resolution (VHR) satellite data and Earth System models, enabling comprehensive analysis of the Earth's dynamics holistically.
Multi-modality and multi-temporal models: FMs can adeptly handle diverse data modalities such as text, video, imagery and time-series, offering new approaches to data analysis, processing and interpretation, as well as to change detection capability.
Scientific understanding: In addition to understanding how to best create general-purpose models, it is of utmost importance to highlight what scientific insights are enabled through the creation of FMs. In particular, insights in relation to physical principles, spectral response, temporal and spatial performance and causality (e.g. in the areas of cloud dynamics, hydrology and oceanography) are strongly invited.
Implications of FMs for the Community: Understanding the potential societal, environmental and economic impacts of implementing FMs in EO applications, fostering informed decision-making and resource management.