Updating our Climate Monitoring Database to a simple, consistent and traceable solution

Approximately 25 years ago the Met Office designed and implemented a database of station-based monthly climate statistics using data from the UK's land network of weather stations. This database enables climate monitoring activities by providing pre-calculated monthly values for the entire record at each site.  

The variables include temperature, rainfall, sunshine duration, wind speed, pressure, humidity, snow and cloud cover. Statistics derived from these variables include monthly means and totals, highest and lowest daily values within each month, and counts of threshold exceedances (e.g. number of days with air temperature below 0°C). 

After 25 years this legacy code is being retired and a modern replacement system designed that will be written in Python.  

In this presentation we will describe the progress of this work, with particular focus on: 

• The source of the input data (and how this relates to the business logic) 

• The database table structure (and why we decided to change this) 

• The design of the software (including how it can be configured) 

• Synchronisation with the raw data (through database triggers and version numbers) 

• Improved traceability (of both calculated values and rescued data) 

• Population of the new database table (including data volumes and run times) 

We will also address data licensing considerations and highlight some of the downstream impacts of the new system on user applications. 

All National Meteorological Services must routinely replace legacy IT systems such as this if they are to remain fit for purpose in the long term. Our new system is designed to be modern, simple, consistent, efficient, traceable, flexible and scalable. We aim for it to provide a robust foundation for climate monitoring activities over the next 25 years!