Transient low dimensional distribution of ensemble prediction in high dimensional chaos and control using the low dimensional latent representation

We report that in several chaotic, high-dimensional nonlinear systems, the evolution of multiple ensembles starting from nearby initial conditions exhibits a transient low-dimensional distribution in phase space. This low-dimensional distribution of the ensembles is primarily achieved by stretching the ensemble distribution along the unstable directions of the system's trajectories. Furthermore, we discuss the potential of using this transient low-dimensional distribution to significantly reduce the search space when controlling the system's future states. As a concrete example of a high-dimensional nonlinear system, we use the Lorenz 96 model under a parameter setting that produces chaotic behaviors. We generate ensembles by adding small random perturbations to the initial conditions and compute the trajectories starting from each initial condition. By applying principal component analysis (PCA) to the ensemble distributions at each time step, we evaluate the dimension of the ensemble spread using a statistics that we call PCA dimension. Our results demonstrate that the PCA dimension initially decreases to values much smaller than the number of ensembles or the system's dimension, before increasing and converging to a value approximately equal to the Kaplan-Yorke dimension of the attractor. This phenomenon is considered to correspond to the asymmetry of the local Lyapunov exponents. Moreover, we show that at times when the PCA dimension of the ensemble distribution transiently decreases, it is possible to accurately regress the system's state at the time of a future extreme event using the scores of the first two principal components of the ensemble distribution. Additionally, using a regression model trained in this low-dimensional latent space, we succeed in identifying an optimal perturbation to the initial conditions to demonstrate the possibility of avoiding extreme events. Since meteorological phenomena are ultra-high-dimensional systems, attempts to reduce the dimensionality of the control search space may contribute to the feasibility of implementing such control measures. This presentation includes the most recent progress, such as using a weather prediction model, at the time of the conference.