ClimateBenchPress: A Benchmark for Compression of Climate Data

The rapid growth of weather and climate datasets is increasing the pressure on data centres and hinders scientific analysis and data distribution. For example, kilometre-scale weather and climate models can generate 20 gigabytes of data per second when run operationally, making it generally infeasible to store all output unless advanced compression is applied. 

To address this challenge, novel lossy compression techniques, including recently so-called neural compressors which learn smaller representations of climate data, have been proposed with compression factors beyond 100x. However, if applied without care, lossy compression can remove valuable information from a dataset for often unknown downstream applications. It is therefore important to validate that the compression process does not alter scientific conclusions drawn from the data. Whether the compression error is tolerable is often easier to assess for domain experts and rarely well defined. 

Here, we address this challenge by presenting a benchmark suite for lossy compression of climate data (atmosphere, ocean, and land). We are defining data sets that can be used to train neural compressors as well as corresponding evaluation methods. Compressors have to pass a set of tests for each data set while compressing into the smallest file size possible at a reasonable (de)compression speed. To ensure evaluation on a diverse set of inputs, the benchmark covers climate variables following various statistical distributions at medium to very high resolution in time (hourly to yearly) and space (~1 km to 150 km). Evaluation tests are for single and multi-variable compression of gridded data with stable or changing statistics, random data access or large archives, in medium to very large datasets.

To provide references towards what compression levels can be achieved with current state of the art lossy compressors, we also evaluate a set of baseline compressors (SZ3, ZFP, Real Information) on our benchmark tasks. The benchmark is a quality check for new compressors towards a standardization of climate data compression, aiming to make compressors with high compression factors safe to use and widely supported.