Assessing model performance by weather regime transitions

Having the ability to stratify a model’s performance by weather type is not only beneficial to a weather forecaster when making decisions, but it is also important for end users, whether they be scientists looking to improve the model, or a customer wishing to know the value of a forecast under a specific set of circumstances.

At the MET Office, Decider is a tool which assigns a dominant weather type to a set of ensemble members, to predict the probability of a weather type occurring. The weather type is chosen from either a set of 30 or 8 sub-types, where a weather type is pre-determined objectively by clustering a 154 year record of sea level pressure anomaly fields.  

There is also a record of daily weather type classifications derived from analysis fields and so information of model performance for these weather types could be invaluable in reducing model error if combined with the predictions from Decider.

Early trials of assessing model performance by weather type revealed that larger errors occur when the weather type persisted for a single day, rather than longer timescales, and so this suggests that it would be beneficial to examine weather type transition periods.

To examine this, we expand the weather type methodology to include multiple time periods. The current methodology uses 12Z analyses to identify the weather type, and so we first assess model performance as a sensitivity study to the analysis time.

Transition days are identified when the weather type changes during a pre-defined validation period, which allows separation into either night/day weather type transitions, or a change in weather type over a full 24-hour period.

We will present early results of this work and demonstrate the impact of model performance when stratifying by regime transitions.