Quantifying Uncertainty for Complex Systems

The ability to quantify uncertainty effectively in complex systems is not only useful, but essential in order to make good decisions or predictions based on incomplete knowledge.  Conversely, failure to quantify uncertainty, and a reliance on making assumptions, prevents a proper understanding of the uncertain system, and leads to poor decision-making. 

In our work to implement a geological disposal facility (GDF) for higher-activity radioactive waste, we need to be very confident in our demonstration of safety of the facility over geological timescales (hundreds of thousands of years). There are inevitably large uncertainties about the evolution of a system over such timescales.  We have developed a strategy for managing and quantifying uncertainty which we believe is more generally applicable to complex systems with large uncertainties.  At the centre of the strategy are three concepts: a top-down, iterative approach to building a model of the ‘total system’; a probabilistic Bayesian mathematical treatment of uncertainty; and a carefully designed methodology for quantifying uncertainty in model parameters by expert judgement that mitigates cognitive biases which usually lead to over-confidence.

Our total system model is a probabilistic model built using a top-down approach. It is run many times as a Monte-Carlo simulation, where in each realisation, parameter values are sampled from a probability density function representing the uncertainty. It is built with the performance measures of interest in mind, starting as simply as possible, then iteratively adding detail for those parts of the model where previous iterations have shown the performance measures to be most sensitive.  It sits well with a similar iterative approach to data gathering, the aim being developing understanding of what parts of the total system really matter. 

In our experience, to be both a tractable and effective strategy, it is essential that the level of detail and complexity in any quantitative analysis, is commensurate with the amount of uncertainty.  There needs to be a recognition that initial consideration of the system in too great a level of detail is futile when the uncertainty is large.  It is through iterative learning, understanding the sensitivities to the total system and refining our analysis and data gathering in areas of significance, that we can handle even complex uncertainty and develop a sound basis for confidence in decision making.

We are now looking to explore new techniques to iterative learning that involve maximising the information that can be gained, even from initially sparse datasets, to aid in confident decision-making.