Calibration-driven parameterization development

We discuss the use of systematic ‘a posteriori’ calibration in the development of complicated (but theory-based) parameterizations. With ‘a posteriori’ calibration, model error is assessed using the results of forward simulations, thereby incorporating numerical error, numerical stability, model-specific implementation details,  and alleviating the need for explicit data for all parameterized model components. We show how calibration illuminates the parameterization development trade-off between reductions in model bias, producing better predictions, and increased parametric complexity, the latter which can decrease a model’s ability to extrapolate, increase both the data requirements and computational expense of the calibration. We illustrate the importance of a posteriori calibration by describing the iterative development of CATKE, a new parameterization we develop within CliMA for the fluxes associated with small- or "micro-scale" ocean turbulent mixing on scales between 1 and 100 meters. For calibration we use Ensemble Kalman Inversion to minimize the error between a set of large eddy simulations (="the truth") and predictions of the parameterization and this way find optimal values for the free parameters. Without systematic calibration we cannot make informed choices about parameterization development because we cannot distinguish between structural error and error due to non-optimal parameter values.