Performance Gains and Advantages of 4DVarNet in End-to-End Learning for Data Assimilation

The 4D variational assimilation (4DVar) framework is widely used in classical numerical weather prediction and geophysical data assimilation. However, a crucial assumption in 4DVar is that the model state that is close to the true state corresponds to the minimizer of the 4DVar cost function. Using a single-layer quasi-geostrophic (QG) model, we study scenarios where this assumption breaks down, particularly in the presence of model errors and suboptimal initialization. By introducing controlled perturbations in the initial conditions—we design experiments to investigate the sensitivity of 4DVar solutions. We find that minimizing the 4DVar score does not always correlate with achieving lower accuracy, suggesting the presence of local minima in the optimization process. 

4DVarNet, an end-to-end neural network based on variational data assimilation formulation, is trained in a supervised manner to solve the data assimilation task. This study aims to understand the advantage of trainable solvers that solve the same optimization problem using supervised learning, generating more accurate solutions efficiently. Through this case study based on observing system simulation experiments for sea surface geophysical fields, we show that supervised learning can overcome the minimization challenges of 4DVar when faced with observations that are irregular and highly sparse which are critical to address problems in ocean reconstruction. The advantage of learning allows 4DVarNet to discover hidden representations that are suitable for solving specific data assimilation tasks with better accuracy.