EGU24-16412, updated on 09 Mar 2024
https://doi.org/10.5194/egusphere-egu24-16412
EGU General Assembly 2024
© Author(s) 2024. This work is distributed under
the Creative Commons Attribution 4.0 License.

Deep-Image-Matching: an open-source toolbox for multi-view image matching of complex geomorphological scenarios

Francesco Ioli1, Luca Morelli2,3, Livio Pinto1, and Fabio Remondino2
Francesco Ioli et al.
  • 1Politecnico di Milano, Department of Civil and Environmental Engineering, Section of Geodesy and Geomatics, Milano, Italy (francesco.ioli@polimi.it, livio.pinto@polimi.it)
  • 2Bruno Kessler Foundation, 3D Optical Metrology (3DOM), Trento, Italy (lmorelli@fbk.eu, remondino@fbk.eu)
  • 3University of Trento, DICAM Department, Trento, Italy

Geomorphometry and geomorphological mapping are essential tools for understanding landscape changes. The recent availability of 3D imaging sensors and processing techniques, including Artificial Intelligence, is offering interesting solutions for gemorphometric analyses and processes understanding. Photogrammetry stands as a pivotal image-based tool in geomorphology, enabling accurate 3D reconstruction of complex natural environments and effective tackling of multi-temporal monitoring challenges. A key step in photogrammetry is the identification of corresponding points between different images, traditionally achieved through the extraction and matching of local features such as SIFT and ORB. However, these methods face difficulties when using images of complex environments scenarios. Deep Learning (DL) methods have recently emerged as powerful tools to address challenges such as strong radiometric variations and viewpoint changes (Morelli et al., 2022; Ioli et al., 2023). However, their practical application in photogrammetry is hindered by the lack of libraries integrating DL matching into standard SfM pipelines.

The presentation will introduce the recently developed Deep-Image-Matching, an open-source toolbox designed for multi-view image matching using DL approaches, specifically tailored for 3D reconstruction in complex scenarios (https://github.com/3DOM-FBK/deep-image-matching). This tool can be used to achieve a 3D reconstruction with wide camera baselines and strongly varying viewpoints (e.g., with ground-based monitoring cameras), with datasets involving varying illumination or weather conditions typical of multi-temporal monitoring, with historical images, or in low-texture situations (e.g., snow or bare ice).

Deep-Image-Matching provides the flexibility to choose from a variety of local feature extractors and matchers. Supported methods include traditional local feature extractors, such as ORB or SIFT, as well as learning-based methods, such as SuperPoint, ALIKE, ALIKED, DISK, KeyNet + OriNet + HardNet, and DeDoDe. Matcher choices range from traditional nearest neighbor algorithms to state-of-the-art options like SuperGlue and LightGlue. Available semi-dense matching solutions include the detector-free matchers LoFTR and RoMa.

To handle high-resolution images, the tool offers a tiling process. In case of strong image rotations, such as aerial stripes, images are automatically rotated before matching. Image pairs for matching can be selected by exhaustive brute-force matching, sequential matching, low-resolution guided pairs selection, or global descriptor-based image retrieval. Geometric verification is used to discard outliers among matched features. The extracted image correspondences are stored in a COLMAP database for further processing (i.e. bundle adjustment and dense reconstruction) or can be exported in other formats useful for other open-source and commercial software.

The presentation will highlight how image-based geomorphometry and geomorphological mapping could benefit of the realized tool and how complex environmental scenarios (landslides, glaciers, etc.) could be analysed and monitored with the support of deep learning.

References:

Ioli, F., Bruno, E., Calzolari, D., Galbiati, M., Mannocchi, A., Manzoni, P., Martini, M., Bianchi, A., Cina, A., De Michele, C. & Pinto, L. (2023). A Replicable Open-Source Multi-Camera System for Low-Cost 4D Glacier Monitoring. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., 48, 137-144

Morelli, L., Bellavia, F., Menna, F., & Remondino, F. (2022). Photogrammetry Now and Then - From Hand-Crafted to Deep Learning Tie Points. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLVIII-2/W1-2022, 163–170

How to cite: Ioli, F., Morelli, L., Pinto, L., and Remondino, F.: Deep-Image-Matching: an open-source toolbox for multi-view image matching of complex geomorphological scenarios, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-16412, https://doi.org/10.5194/egusphere-egu24-16412, 2024.

Supplementary materials

Supplementary material link

Comments on the supplementary material

AC: Author Comment | CC: Community Comment | Report abuse

supplementary materials version 1 – uploaded on 13 Apr 2024, no comments