Dimension reduction Kalman filtering: examples from high-dimensional dynamical systems

We consider a prior-based dimension reduction Kalman filter for state estimation in high-dimensional settings. The method extend ideas from prior-based dimension reduction in static inverse problems by projecting covariance equations to lower-dimensional space using a global reduction operator. In contrast to reduced rank Kalman filters the dimension reduction is defined entirery a priori. Here, it is constructed using standard wavelet transforms, yielding a stable and portable framework that does not depend on empirical parameter estimation to form the projection. 

The Kalman filter update step equations are projected onto a global wavelet basis, thereby avoiding explicit construction of covariance matrices in the full state space. This makes classical Kalman filtering tractable for large spatio-temporal systems otherwise computationally inaccessible. Combined with PyTorch implementation exploiting GPU acceleration, the approach leads to a drastic reduction in computational cost, while preserving the consistent filter and enabling Gaussian uncertainty quantification.

We demonstrate the method on two high-dimensional application, highlightning the wavelet representation's natural adaptation to different data patterns and structures. The first example concerns sparsely observed oceanographic data, where the reduced filter reconstructs the full state from limited measurements with uncertainty estimates with state model derived from modelled ocean current. The second focuses on satellite-derived cloud product with state dynamics provided by neural network estimates and the observations exhibit heterrogeneous quality and frequent gaps.

Overall, we demonstrate how reduced-basis Kalman filtering with a priori selected wavelet subspaces provides a general and computationally viable framework for nonstationary Gaussian inverse problems. The approach combines scalable data assimilation, uncertainty quantification, and the integration of data-driven dynamics in high-dimensional geophysical applications.