Polar TEP – A Platform for Polar Big Data Analytics and Machine Learning

ESA has developed a series of seven TEPs on different subjects to provide insight into how our oceans, atmosphere, land and ice operate and interact as part of an interconnected earth system by exploiting the unprecedented flow of high-quality global data on the state of our planet, combined with long-term EO archives, in-situ networks and models. The Polar Thematic Exploitation Platform (Polar TEP) was developed to address the particular needs of the polar community.

Polar TEP provides a complete working environment where users can access algorithms and data remotely to obtain computing resources and tools that they might not otherwise have and avoid the need to download and manage large volumes of data. This new approach removes the need to transfer large Earth Observation data sets around the world, while increasing the analytical power available to researchers and operational service providers. Polar TEP provides new ways to exploit EO and other large datasets for research scientists, industry, operational service providers, regional authorities, and policy analysts. Polar TEP provides:

• Data Discovery - Polar TEP makes satellite and other polar data easily accessible for browsing or analysis within the cloud or within the user’s own environment. The infrastructure takes care of the complexity of handling satellite imagery archives and makes the data available via web services. Users can instantly access petabytes of Sentinel, Landsat, and other Earth observation imagery, both historic and the latest acquisitions.

• Interactive Development Environment - Polar TEP offers a managed JupyterLab instance with curated base images. The platform provides different flavors of computational resources and a network file system for persistent data storage. Headless notebook execution is supported.

• Machine Learning - Polar TEP has implemented the MLflow platform to support machine learning activities. MLflow manages all stages of the machine learning lifecycle, including experimentation, reproducibility, deployment, and a central model registry.

• Execution Environment - Docker containers are used to provide processors with a separate custom environment having minimal execution overhead. The computing resources used by the execution environment are scaled to the current demand.

• Application Hosting Environment - Users can host their own applications on a VM within the Polar TEP environment.

• Story Telling - Polar TEP provides tools to communicate analysis results to other researchers or the public.

Polar TEP is an integral part of the wider polar data ecosystem, contributing to data interoperability and fostering the use of information about the polar regions to support environmental protection, safety, and sustainable economic development.

This presentation will illustrate how the power of Polar TEP to process massive amounts of data is being applied to topics such as machine learning for operational sea ice charting, daily calculations of Greenland ice sheet albedo, and providing information to support traditional ways of life in the Arctic.