(Auto) Differentiating geology: Geological modelling with random Fourier features and neural fields

Neural fields (a.k.a. Spatial Neural Networks) are neural networks that take spatial coordinates as inputs and output target (interpolated) variable(s). They can learn arbitrarily complex functions and, because they are auto-differentiable, can be easily constrained by their spatial derivatives. In this contribution, we build on recent work to further explore applications of neural fields for geological modelling.

While scalar fields have been used to represent subsurface geology before, constraining these fields is a challenge. Geological models are under-constrained, requiring e.g. regularisation to derive geologically sensible results, making it difficult to learn high-frequency geometric details. Furthermore, unlike most applications of neural networks, neural fields have low dimensional inputs, which further limits their ability to learn high-frequency features during training. 

We address these challenges by using random Fourier feature encoding, a technique inspired by computer vision which transforms spatial inputs into a higher-dimensional feature space by applying sine and cosine functions weighted by randomly initialized parameters. Loss functions based on the value and gradient of the output scalar field are then used to learn the geometry of subsurface geology. Significantly, we also impose a weak-harmonic constraint on the field by minimising the divergence of the scalar field’s gradient, which penalises the formation of closed scalar field isosurfaces (i.e., “bubbles”) which violate the layered topology of stratigraphic sequences.

We demonstrate our approach on several synthetic geological datasets, and show how the neural field approach can explore the possible solution space using different random initialisations, thereby helping quantify uncertainty. To conclude, we suggest that neural fields could provide a powerful tool for future geological modelling workflows, due to their flexibility and ability to constrain diverse aspects of geological models.