Advancing Progress on Earth System Digital Twins through an Integrated AI-Enabled Framework

The development of Earth System Digital Twins (ESDTs) represents an ongoing journey towards more accurate and integrated simulations of Earth processes. Inherently interdisciplinary, the endeavor grapples with the challenge of melding subsystems developed by experts in different fields and organizations, requiring communication between different science domains, technology stacks, and data modalities. The Coupled Reusable Earth System Tensor (CREST) framework is a key aspect of our efforts to address these difficulties: by implementing a generic abstraction layer over existing tensor libraries (e.g. TensorFlow, PyTorch, JAX), CREST provides the software foundation for building, operating, and deploying community developed ESDTs. This framework is designed to allow scientists to easily couple together process-based and data-driven models into broader digital twin workflows, while taking advantage of significant efficiency improvements from hardware accelerators.

CREST aims to be a step forward in combining traditional modeling techniques with emerging computational methods, particularly in the context of machine learning. Machine learning plays a foundational role in our approach, both contributing to the development of new data-driven models and aiding in efficient coupling with existing models. Through CREST, we aim to enhance model integration and foster more dynamic interactions within the modeling pipeline – primarily addressing the issues of limited support in current frameworks for gradient propagation, hardware acceleration, and federation with external models. In addition, CREST operational capabilities will include data assimilation, end-to-end distributed model training, black-box model coupling, what-if scenario analysis, and an easy-to-use GUI interface for end users.

In the context of practical applications, the Terrestrial Environmental Rapid-Replication and Assimilation Hydrometeorological (TERRAHydro) system serves as an example of applying these principles in practice. Using CREST, TERRAHydro couples together several hydrologic and land surface subcomponents, such as soil moisture, evapotranspiration, and net ecosystem exchange, into a coupled land surface model. The efficiency of AI-based LSDTs such as TERRAHydro are expected to be able to carry out scenario analysis beyond existing traditional land surface models. Here we show results and comparisons for this application, discuss progress on CREST and TERRAHydro overall, and outline our roadmap going forward.