An expert semi-quantitative evaluation approach to measure the quality of landslide inventories

The quality of a landslide inventory depends on its accuracy and the type and certainty of information shown on the map. Defining the accuracy of a landslide inventory is not straightforward, and there are no standards. Accuracy depends on the completeness of the map and the geographic and thematic correctness of the information displayed on the map.

In this study, an expert, semi-quantitative evaluation approach was developed to assess mapping errors and consequently determine the quality of a geomorphological historical landslide inventory map for a 2,000 km² area in the Southern Italian Apennines (Daunia, Puglia region) prepared through aerial photo-interpretation.

Quality control aims to quantify parameters, even in terms of binary choices. Whenever involving expert estimates, the assessment was carried out through a reference grid to limit inconsistencies. Furthermore, the expert evaluations were carried out by a team of evaluators working collegially, and who had not previously worked simultaneously on the same areas.

The general approach involves a systematic evaluation within 5 sample areas covering a total of 202 km² deemed a choice enabling the identification of areas that are sufficiently extensive (i.e., 10% of the total area and 20% of the total number of landslides) and adequately represent the diverse morphological and litho-structural characteristics of the study area. The assumption of representativeness of these areas forms the basis for extrapolating error data to the entire investigated area.

Geographic accuracy gauges the degree of correspondence between morphological and photographic evidence of landslides and their portrayal on the map. This correspondence was decomposed in terms of position, shape, and size and was evaluated, for each landslide portrayed in the 5 sites, at the declared scale of the final map (1:5,000). Each component was considered acceptable if at least 2/3 of the landslide was mapped correctly, and the overall accuracy satisfactory if at least two components were sufficient.

To assess the completeness of the inventory, the authors of the map were preliminarily asked to state the minimum size of landslides consistently mapped (A_CM) in the inventory. Then, within the sample areas, the ratio between the number of landslides exceeding A_CM and not represented in the map and the number of the landslides mapped expresses the degree of completeness of the inventory for the declared A_CM.

To evaluate the thematic accuracy, a percentage score was assigned representing the proportion of landslides with specific thematic errors within the inventory, considering the correct classification (type and relative age) of each landslide represented in the map and compared to its photographic and morphological evidence.

Considering the absence of qualitative standards in the scientific literature for landslide inventory maps, and, consequently, the lack of evaluative standards for the accuracy of such cartographic products, this work can be considered an attempt to define a procedure to evaluate the informative content of landslide inventory maps.