An advanced method to validate and compare susceptibility maps by investigating local-scale differences and highlighting the role of geomorphological features

In landslide studies, comparing the outcomes obtained by different models is a very robust test for their predictive capability and quantitative indexes are often used to assess which model provides the best predictions. The literature about landslide susceptibility is rich of works where two or more susceptibility models are validated in terms of AUC (area under ROC curve), then the AUC values are compared and the model that provided the highest AUC value is considered the best one.

The main purpose of this work is to expand this classical approach, which is too simplistic as it neglects any geomorphological consideration, and to propose a new approach that shifts the comparison at the pixel scale, linking the local-scale differences encountered with specific features of the study area. The proposed advanced comparison approach can be summarized with the following steps:

As a case of study, we used four susceptibility maps already defined with random forest (RF), index of entropy (IOE), frequency ratio (FR), and certainty factor (CF) in Wanzhou County (China). A classical validation procedure showed that RF provided the best outcomes, with a 0.801 AUC. After applying the advanced comparison procedure, we obtained deeper insights on the susceptibility models, explaining e.g. why and where RF performed significantly better than the other models and identifying systematic errors that could be associated to distinctive geomorphological features of the test site. Indeed, we discovered that RF is more able to exploit the very complex parameterization of the problem, with 13 parameters, sometimes interrelated each-other, with a total of 80 classes. Moreover, we found that the other models produced systematic errors in correspondence with some lithological units and in fluvial terraces. The area is characterized by 5 orders of relict fluvial terraces, clearly defined only in some small stretches, and the results obtained showed that landsliding has probably been one of the predominant geomorphological process responsible for their depletion.