Inconsistency and violation of causality in Bayesian inversion paradigms

Probabilistic formulations of inverse problems are most often based on Bayes Rule, which is considered a powerful tool for integration of data information and prior information about potential solutions. However, since its introduction it has become apparent that the Bayesian inference paradigm presents a number of difficulties, especially in the phase where the problem is mathematically formulated.

 

Perhaps the most notable difficulty arises because Bayes Theorem is usually formulated as a relation between probability densities on continuous manifolds. This creates an acute crisis because of a problem described by the French mathematician Joseph Bertrand (1889), and later investigated by Kolmogorov and Borel. According to Kolmogorov's (1933/1956) investigations, conditioning of a probability density is underdetermined: In different parameterizations (reference frames), conditional probability densities express different probability distributions. Surprisingly, this problem is persistently neglected in the scientific literature, not least in applications of Bayesian inversion. We will explore this problem and show that it is a serious threat to the objectivity and quality of Bayesian computations including Bayesian inversion, computation of Bayes Factors, and trans-dimensional inversion.

 

Another difficulty in Bayesian Inference methods derives from the fact that data uncertainties, and prior information on the unknown parameters, are often unknown or poorly known. Because they are required in the calculations, statisticians have invented

hierarchical methods to compute parameters (known as hyper-parameters) controlling these uncertainties. However, since both the data uncertainties and the prior information on the unknowns are supposed to be known 'a priori', but are calculated 'a posteriori', this creates another crisis, namely a violation of causality. We will take a close look at the consequences of this mixing of 'prior' and 'posterior', and show how it potentially jeopardizes the validity of Bayesian computations.