Multi-modal machine learning models for atmospheric and Earth system dynamics that are trained on many different datasets and predict a range of outputs, such as the WeatherGenerator, ESFM, or WeatherNext, are emerging as the next frontier in machine learning-based Earth system modeling. These models pose unique challenges, such as how to combine many input datasets at different resolutions in space and time, how different datasets should be scheduled at different training stages, or what data should be used as target during pre-training and what can be decoded afterwards. They also open research questions such as how model parameter count should depend on the number and resolution of input datasets or how to quantify the expected benefit of adding an additional dataset. This session collects contributions both on the development of multi-modal machine learning models as well as their use in applications, from short- and medium-range forecasting to scientific investigations that probe the model itself. Submissions can address either methodological questions, evaluations (e.g. conservation of physics), technical aspects (training on HPC systems) or focus on applications.
ESSI1
Multi-modal Machine Learning Models for Earth System Modeling