Comparing Machine Learning Methods for Dynamical Systems

Machine learning (ML) is becoming increasingly important in climate research and Earth system modeling. Our goal is to better understand how different ML methods compare, and to enable researchers to make informed decisions when choosing machine learning tools for a given task. In this work, we explore the problem of learning the dynamics of a system from observed data without prior knowledge of the laws governing the system. Our extensive simulation study focuses on ordinary differential equation (ODE) problems that are specifically designed to reflect key aspects of various ML tasks for dynamical systems - namely, chaos, complexity, measurement uncertainty, and variability in measurement intervals. The study evaluates a variety of methods, including neural ODEs, transformer networks, Gaussian processes, echo state networks, and spline-based estimators. Our results show that the relative performance of the methods tested varies widely depending on the specific task, highlighting that no single method is universally superior. Although our research is predominantly in low-dimensional settings, in contrast to the high-dimensional nature of many climate science challenges, it provides insightful comparisons and understanding of how different approaches perform in learning the dynamics of complex systems.