Interactive Voxel Visualization of Large Earth System Data Cubes

As Earth system data streams and models grow larger, more complex and higher-dimensional, the demand for capable data visualization and exploration tools increases. While specialized data cube visualization tools have been developed in recent years, they typically rely on technical compromises to address the data access problems posed by large data sets. Some of the existing tools provide support for visualizing arbitrary 3D data chunks by making parts of the cube transparent, commonly known as volume or voxel rendering, i.e., “looking inside the data set”. This voxel rendering can communicate spatiotemporal patterns effectively and has a much higher information density of previous data cube visualization approaches, but is computationally demanding and scales strongly with the size of the visualized data set. 

Here we present an interactive voxel visualization for large Earth system data cubes, integrated into the existing Lexcube.org data cube visualization and its open-source Python package. The voxel visualization allows to highlight and visualize value ranges based on thresholds, creating "voxel clouds" in three-dimensional space-time. Additionally, users can highlight extreme values by selecting a quantile range, based on deviations from the mean seasonal cycle and other definitions of “extreme”. To enable this visualization for large data sets, we developed a novel lossy compression algorithm based on variable quantization of 3D blocks that significantly reduces both the required VRAM for the visualization and the computational effort for the ray tracing. The algorithm preserves high information content by encoding 3D chunks of high variance at a high resolution, while chunks of nearly uniform values get compressed a lot, respecting a user-set, configurable error metric. This way, the scientific accuracy of the visualization is guaranteed and quantified, while enabling the previously impossible voxel visualization and exploration of large data sets.

Based on the previous Lexcube software, the software stays compatible with a wide range of desktop and mobile devices by relying on WebGL 2 instead of adopting the modern successor WebGPU. Because the data backbone relies on Xarray, any gridded three-dimensional Zarr, NetCDF and other supported data sets can be ingested and visualized with our software - on Lexcube.org or using our open-source package for Jupyter notebooks, available on Github and PyPi.