Interpreting and evaluating digital soil mapping prediction uncertainties

There is an implicit quality associated with all soil maps which depends on a range of factors including the mapping algorithm, the extent and quality of the calibration data, and the quality and relevance of the co-variates. Spatially explicit prediction uncertainty information is increasingly provided when digital soil mapping methods are used. This may take the form of a 90% prediction interval such as that supplied by ISRIC in their global SoilGrids product. In this study we used independent data sets of particle size measurements from New Zealand and the Netherlands to investigate the accuracy of the prediction interval information in the SoilGrids clay, sand and silt layers. While prediction intervals were wide for both countries, we had expected that these would be narrower for the Netherlands as SoilGrids has more calibration points in the Netherlands than in New Zealand. Spatially, there was much more variation in the prediction interval width in the Netherlands than in New Zealand, with some areas being less uncertain and some highly uncertain. New Zealand had uncertain predictions for the entire country, although the prediction intervals were not as wide as in the most uncertain areas in the Netherlands.

Independent validation showed that the clay prediction intervals were too wide: for New Zealand between 95 and 98% of the validation data were within the 90% prediction interval, for the Netherlands this was between 95 and 97%. For sand we found the opposite, with only between 60 and 70% of the data falling in the 90% prediction interval for New Zealand and between 77 and 88% for the Netherlands. Comparison of prediction errors with prediction interval widths showed that the prediction errors tended to be larger at locations with wide prediction intervals, although this relationship was clearer for the Netherlands than for New Zealand. Estimates that fell outside the sand prediction interval were associated with narrow as well as wide prediction interval widths. Our analyses highlight the importance of users considering soil uncertainty information before using the soil data. It is also important that producers of soil information document the accuracy limitations to help guide potential users and that they evaluate the validity of the uncertainty information prior to release.