Deep-Image-Matching: an open-source toolbox for multi-view image matching of complex geomorphological scenarios

Geomorphometry and geomorphological mapping are essential tools for understanding landscape changes. The recent availability of 3D imaging sensors and processing techniques, including Artificial Intelligence, is offering interesting solutions for gemorphometric analyses and processes understanding. Photogrammetry stands as a pivotal image-based tool in geomorphology, enabling accurate 3D reconstruction of complex natural environments and effective tackling of multi-temporal monitoring challenges. A key step in photogrammetry is the identification of corresponding points between different images, traditionally achieved through the extraction and matching of local features such as SIFT and ORB. However, these methods face difficulties when using images of complex environments scenarios. Deep Learning (DL) methods have recently emerged as powerful tools to address challenges such as strong radiometric variations and viewpoint changes (Morelli et al., 2022; Ioli et al., 2023). However, their practical application in photogrammetry is hindered by the lack of libraries integrating DL matching into standard SfM pipelines.

The presentation will introduce the recently developed Deep-Image-Matching, an open-source toolbox designed for multi-view image matching using DL approaches, specifically tailored for 3D reconstruction in complex scenarios (https://github.com/3DOM-FBK/deep-image-matching). This tool can be used to achieve a 3D reconstruction with wide camera baselines and strongly varying viewpoints (e.g., with ground-based monitoring cameras), with datasets involving varying illumination or weather conditions typical of multi-temporal monitoring, with historical images, or in low-texture situations (e.g., snow or bare ice).

Deep-Image-Matching provides the flexibility to choose from a variety of local feature extractors and matchers. Supported methods include traditional local feature extractors, such as ORB or SIFT, as well as learning-based methods, such as SuperPoint, ALIKE, ALIKED, DISK, KeyNet + OriNet + HardNet, and DeDoDe. Matcher choices range from traditional nearest neighbor algorithms to state-of-the-art options like SuperGlue and LightGlue. Available semi-dense matching solutions include the detector-free matchers LoFTR and RoMa.

To handle high-resolution images, the tool offers a tiling process. In case of strong image rotations, such as aerial stripes, images are automatically rotated before matching. Image pairs for matching can be selected by exhaustive brute-force matching, sequential matching, low-resolution guided pairs selection, or global descriptor-based image retrieval. Geometric verification is used to discard outliers among matched features. The extracted image correspondences are stored in a COLMAP database for further processing (i.e. bundle adjustment and dense reconstruction) or can be exported in other formats useful for other open-source and commercial software.

The presentation will highlight how image-based geomorphometry and geomorphological mapping could benefit of the realized tool and how complex environmental scenarios (landslides, glaciers, etc.) could be analysed and monitored with the support of deep learning.

References:

Ioli, F., Bruno, E., Calzolari, D., Galbiati, M., Mannocchi, A., Manzoni, P., Martini, M., Bianchi, A., Cina, A., De Michele, C. & Pinto, L. (2023). A Replicable Open-Source Multi-Camera System for Low-Cost 4D Glacier Monitoring. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., 48, 137-144

Morelli, L., Bellavia, F., Menna, F., & Remondino, F. (2022). Photogrammetry Now and Then - From Hand-Crafted to Deep Learning Tie Points. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLVIII-2/W1-2022, 163–170