In geoscientific applications, we observe a steady increase in the amount of data and in model complexity. This development is partly driven by technological advances that provide access to higher-resolution data, greater computational capabilities, and increasingly complex coupled numerical models. However, these advances also yield major challenges, since the resulting data amount and the number of model evaluations required are becoming infeasible with traditional evaluation methods.
Therefore, many applications take advantage of scientific machine learning (SciML) approaches to overcome these challenges. However, the rapid development of new methods within this field highlights one fundamental challenge: existing benchmarks typically do not adequately represent the complexity, diversity, and requirements of real-case geoscientific applications. Many commonly used benchmarks rely on relatively simple datasets, assume idealized problem setting and are defined for narrow tasks only. This makes it difficult to assess advances in SciML for their transferability and performance for a broader range of geoscientific applications.
To bridge this gap, this session welcomes contributions presenting complex, realistic benchmarks that enable the validation of scientific machine learning approaches for both numerical modelling and scientific data analysis. We particularly encourage benchmarks that capture current challenges such as: large and heterogeneous datasets, high-dimensional features in terms of the spatial-temporal relationships and the parameter space, multi-scale and complex coupled physical processes, limited or noisy observations, and the need for computationally efficient, resource-aware benchmark designs that enable model training and evaluation under limited computational budgets. We encourage submission from different geoscientific disciplines as benchmark requirements can differ significantly across the domains. At the same time, we welcome contributions from the application side that illustrate limitations of existing benchmarks and discuss which characteristics need to be preserved and represented by SciML methods to be beneficial in the given application context.
The session aims to bring together researchers developing SciML methods, benchmarks, and domain-specific applications. By connecting these perspectives, we seek to identify current gaps in benchmarking practices.