Oral presentations and abstracts
Session assets
Scope
Planetary robotic missions contain vision instruments for various mission-related and science tasks such as 2D and 3D mapping, geologic characterization, atmospheric investigations, or spectroscopy for exobiology. One major application for computer vision is the characterization of scientific context, and the identification of scientific targets of interest (regions, objects, phenomena) for being investigated by other scientific instruments. Due to high variability of appearance of such potentially scientific targets it requires well-adapted yet flexible techniques, one of them being Deep Learning. Machine learning and in particular Deep Learning (DL) is a technique used in computer vision to recognize content in images, categorize it and find objects of specific semantics.
In its default workflow, DL requires large sets of training data with known / manually annotated objects, regions or semantic content. Within Mars-DL (Planetary Scientific Target Detection via Deep Learning), training focuses on a simulation approach, by virtual placement of known targets in a true context environment.
Mars-DL Approach
The Mars-DL workflow is depicted in Figure 1, with following details:
- 3D background for the training & validation scenes is taken from 3D reconstructions using true Mars rover imagery (e.g. from the MSL Mastcam instrument).
- Scientifically interesting objects to be virtually placed in the scene are chosen to be 3D reconstructions of shatter cones (SCs) and/or Meteorites as representative set. The SCs used for 3D reconstruction are from different terrestrial impact structures and cover a wide range of lithologies, including limestone, sandstone, shale and volcanic rocks. Typical features of SCs such as striations and horsetail structure are well developed in all selected specimens and allow a clear identification. The involved stony and iron meteorites are museum quality specimens (NHM Vienna) with fusion crust and/or regmaglypts.
- A capturing campaign at the NHM Vienna covered more than a dozen SC and meteorite specimen, imaged from various positions under approximate ambient illumination conditions. This was followed by a 3D reconstruction process using photogrammetric techniques (Figure 2) to obtain a seamless watertight high resolution mesh and albedo map for each object, gaining a 3D data base of objects to be randomly placed in the realistic scenes.
- Simulation of rover imagery for training is based on PRo3D, a viewer for the interactive exploration and geologic analysis of high-resolution planetary surface reconstructions [1]. It was adapted to efficiently render large amounts of images in batch mode controlled by a JSON file. SCs are automatically positioned on a chosen surface region with a Halton sequence, whereby collisions are avoided. The SCs are color adjusted to perfectly blend into the scene and in a future version will also receive and cast shadows. Special GPU shaders provide additional types of training images such as masks and depth maps. A typical rendering data set to be used for training is depicted in Figure 4.
- A random pose generator selects representative yet random positions and fields-of-view for the training images.
- Unsupervised generative-adversarial neural image-to-image translation techniques (GAN; [2]) are then applied on the background of the generated images, thus producing realistic landscapes drawn from the domain of actual Mars rover datasets while preserving the integrity of the high-resolution shatter cone models.
- The candidate meta-architectures for the machine learning approach are Mask R-CNN (instance segmentation) and Faster R-CNN (object detection), as these currently belong to the best performing architectures for these tasks. These meta-architectures can be configured with different backbone networks (e.g. ResNet-50, Darknet-53), which provide a tradeoff between network size and convergence rate.
- A remaining unused training set will be used for evaluation.
Summary and Outlook
So far science autonomy has been addressed only recently with increased mobility on planetary surfaces and the upcoming need for planetary rovers to react autonomously to given science opportunities, well in view of limited data exchange resources and tight operations cycles.
An Austrian Consortium in the so-called "Mars-DL" project is targeting the adaptation and test of simulation and deep learning mechanisms for autonomous detection of scientific targets of interest during robotic planetary surface missions.
For past and present missions the project will help explore & exploit further existing millions of planetary surface images that still hide undetected science opportunities.
The exploratory project will assess the feasibility of machine-learning based support during and after missions by automatic search on planetary surface imagery to raise science gain, meet serendipitous opportunities and speed up the tactical and strategic decision making during mission planning. An automatic “Science Target Consultant” (STC) is realized in prototype form, which, as a test version, can be plugged in to ExoMars operations once the mission has landed.
In its remaining runtime until end of 2020, Mars-DL training and validation will explore the possibility to search scientifically interesting targets across different sensors, investigate the usage of different cues such as 2D (multispectral / monochrome) and 3D, as well as spatial relationships between image data and regions thereon.
During mission operations of forthcoming missions (Mars 2020, and ExoMars & Sample Fetch Rover in particular) the STC can help avoid the missing of opportunities that may occur due to tactical time constraints preventing in-depth check of image material.
Acknowledgments
This abstract presents the results of the project Mars-DL, which received funding from the Austrian Space Applications Programme (ASAP14) financed by BMVIT, Project Nr. 873683, and project partners JOANNEUM RESEARCH, VRVis, SLR Engineering, and the National History Museum Vienna.
References
[1] Barnes R., Sanjeev G., Traxler C., Hesina G., Ortner T., Paar G., Huber B., Juhart K., Fritz L., Nauschnegg B., Muller J.P., Tao Y. and Bauer A. Geological analysis of Martian rover-derived Digital Outcrop Models using the 3D visualisation tool, Planetary Robotics 3D Viewer - PRo3D. In Planetary Mapping: Methods, Tools for Scientific Analysis and Exploration, Volume 5, Issue 7, pp 285-307, July 2018
[2] Jun-Yan Zhu, Taesung Park, Phillip Isola, Alexei A. Efros; Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. The IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2223-2232
How to cite: Paar, G., Traxler, C., Nowak, R., Garolla, F., Bechtold, A., Koeberl, C., Fernandez Alonso, M. Y., and Sidla, O.: Mars-DL: Demonstrating feasibility of a simulation-based training approach for autonomous Planetary science target selection, Europlanet Science Congress 2020, online, 21 Sep–9 Oct 2020, EPSC2020-189, https://doi.org/10.5194/epsc2020-189, 2020.
Abstract
In this work, we demonstrate techniques and results of a 3D mosaic of the whole of the Valles Marineris (VM) area of Mars using stereo images from the Mars Express High Resolution Camera (HRSC), as well as some examples of multi-resolution orthoimage and 3D mapping of 3 selected sites where recurring slope lineae (RSL) are common, at Coprates Montes, Nectaris Montes, and Capri Chaos, using Mars Reconnaissance Orbiter (MRO) Context Camera (CTX), High Resolution Imaging Science Experiment (HiRISE) and the Compact Reconnaissance Imaging Spectrometer for Mars (CRISM).
Introduction
Valles Marineris is the largest system of canyons on Mars that is more than 4,000 km long, 200 km wide, and up to 10 km deep. 3D mapping is essential to improving our understanding of the geological environment of Valles Marineris. Typically, deriving a co-aligned and mosaiced 3D base map covering the whole area is generally the starting point of such studies.This work focuses on a demonstration of multi-resolution 3D mapping and co-alignment of different data sources over the VM area using a co-registered and nested HRSC-CTX-CRISM-HiRISE dataset.
3D modelling
Previously, within the EU FP-7 iMars (http://www.i-mars.eu) project, we developed an automated multi-resolution Digital Terrain Model (DTM) processing chain for NASA CTX and HiRISE stereo-pairs, called the Co-registration ASP-Gotcha Optimised (CASP-GO), based on the open source NASA Ames Stereo Pipeline (ASP) [1], tie-point based multi-resolution image co-registration [2], and the Gotcha [3] sub-pixel refinement method. The CASP-GO system guarantees global geo-referencing congruence with respect to the aerographic coordinate system defined by HRSC, level-4 products and thence to the MOLA DTM, providing much higher resolution stereo derived DTMs.
3D area mosaic using HRSC
HRSC is now on its ~5,700thorbit onboard Mars Express, (totalling>20,000 HRSC products) covering ~98% of the Martian surface at a spatial resolution higher than 100 m/pixel. Among these, DLR has processed HRSC stereo images (level 4 DTMs, [4]), at 50-150m/pixel resolution, that now cover ~50% of the planet’s surface. At Valles Marineris, the DLR HRSC level 4 DTMs have covered a large portion of the canyons, leaving several large gaps unprocessed or having missing data.
We created a further 11 HRSC single strip DTMs and Orthorectified images (ORIs) using HRSC level 2 stereo images to cover these gaps. These are processed using CASP-GO [2] and co-registered with the existing DLR HRSC level 4 DTMs and ORIs to allow seamless mosaicing for the whole of Valles Marineris. Also, to eliminate the need for bundle block adjustment, both UCL and DLR HRSC level 4 DTMs are co-aligned with the MOLA DTM using an iterative closest point co-alignment which is part of ASP [1]. Figure 1 shows a colourised and hill-shaded DTM of the whole area. HRSC level 4 ORIs are then generated using this mosaiced DTM are radiometrically corrected and brightness/contrast adjusted to produce a corresponding HRSC ORI mosaic [5].
Figure 1 HRSC DTM merged DLR + UCL mosaic of VM at 50m/gridpoint
Figure 2 HRSC ORI mosaic of VM generated at 12.5m after brightness adjustment [5]
Automated co-registration of CTX, HiRISE and CRISM
In order to study RSL features, we produce stacks of multi-resolution products using the HRSC DTM and ORI as the basemap. We here show an example of one of the CTX DTMs and ORIs for Capri Chaos which is one of the three selected study sites. The CTX ORIs are co-registered with the HRSC ORI mosaic using the tie-point based image co-registration pipeline described in [2]. The CTX DTMs are then spatially adjusted according to the ORI tie-point and co-aligned vertically to the HRSC DTM mosaic using a B-Spline fitting method to eliminate any residual jitter. The spatial agreement between CTX ORIs and HRSC ORI mosaic is at subpixel level of the base image (~5m). The vertical agreement between CTX DTMs and HRSC/MOLA DTM mosaic can be reduced from several hundreds of metres to less than 10 m (on average) and 50 m (at maximum).
Subsequently, we have also corrected the UoA HiRISE ORIs and DTMs using the same joint image co-registration and vertical alignment method to CTX basemap. The CRISM data is also co-registered with the CTX ORI using a modified version of the tie-point based image co-registration pipeline, bringing several hundreds of metres of spatial misalignment to less than 18 metres. An example of the co-registered nested stack is shown in Figure 3 and of the CTX DTM in Figure 4.
Figure 3 Co-registered HiRISE ORI (at 90% transparency) and unco-registered UoA HiRISE ORI (in red), co-registered CRISM (at 90% transparency) and unco-registered raw CRISM (blue) image, shown on top of the UCL CTX ORI and HRSC ORI mosaic at Capri Chaos in the Valles Marineris.
Figure 4 Colourised and hillshaded figures of co-aligned and nested HiRISE DTM (in green bounding box), CTX DTM (in blue bounding box), and HRSC DTM mosaic at Capri Chaos at VM.
Conclusion
Precisely co-registered HiRISE, CRISM, CTX and HRSC data will be used as a basis for future comprehensive analyses of RSL sites using: time-series data to track RSL development, topographic analysis to constrain the slopes over which they propagate combined with information from compositional data derived from spectral and photometric data.
Acknowledgements
The research leading to these results is receiving funding from the UKSA Aurora programme (2018-2021) under grant no. ST/S001891/1. The research leading to these results also received partial funding from the European Union’s Seventh Framework Programme (FP7/2007-2013) under iMars grant agreement n˚ 607379. SJC is grateful for the financial support of CNES in support of her HiRISE work.
References
[1] Beyer, R., O. Alexandrov, S. McMichael (2018), Earth and Space Science, vol 5(9), pp.537-548. [2] Tao, Y., J.-P. Muller, W. Poole (2016), Icarus, vol 280, pp.139-157. [3] Shin, D. and J.-P. Muller (2012), Pattern Recognition, vol 45(10), pp.3795 -3809. [4] Gwinner et al., (2016) PSS, vol. 126, pp. 93-138 [5] Michael, G.G. et al. (2016), PSS, vol 121, pp. 18-26.
How to cite: Tao, Y., Michael, G., Muller, J.-P., and Conway, S.: 3D multi-resolution mapping of Valles Marineris to improve the understanding of RSLs , Europlanet Science Congress 2020, online, 21 Sep–9 Oct 2020, EPSC2020-430, https://doi.org/10.5194/epsc2020-430, 2020.
ABSTRACT
Impact crater identification and localization are vital to age estimation studies of all planetary surfaces. This study illustrates the utility of object detection based approach for impact crater identification. We demonstrate this using the You Only Look Once (YOLO) object detection technique to localize and identify impact craters on the imagery from Context Camera onboard the Mars Reconnaissance Orbiter. The model was trained on a chipped and augmented dataset of 1450 images for 13000 iterations and tested on 750 images. Accuracy and sensitivity studies reveal a Mean Average Precision of 78.35%, precision of 0.81, recall (sensitivity) of 0.76 and F1 score of 0.78. We conclude that object detection techniques such as YOLO may be employed for crater counting studies and database generation.
INTRODUCTION
Cumulative crater size-frequency distribution studies have been widely used in establishing the age of planetary landforms [1,2]. The crater identification process for such studies has been largely manual wherein impact craters are digitized on GIS platforms. Recent studies have explored the potential of using automatic crater detection techniques such as Canny Edge Detection and Circular Hough Transform [3], machine learning classifiers such as random forest [4] and support vector machines [5], and deep learning-based semantic classification [6,7]. These techniques are often computationally intensive[6,7] or result in lower accuracy[3,4,5] values. We have implemented a different approach utilizing bounding box based object detection. Here, we present an automatic crater detection algorithm that utilizes the YOLO object detection technique[8] implemented on the Google Colaboratory cloud service platform.
DATA PREPARATION
15 datasets near the Hesperia Dorsa and Tartarus Montes regions of the Martian surface imaged by the Context Camera payload onboard the Mars Reconnaissance Orbiter satellite were used for our study. Impact craters larger than 50m were annotated as circles using the QGIS software and attributes of centroid coordinates, minimum and maximum annotation extents and diameter were extracted. Clusters of craters were identified using the Density-based spatial clustering of applications with noise(DBSCAN)[9] algorithm. 512x512 pixel image chips containing the crater clusters were extracted. A dataset of 528 image chips was generated after clustering and chipping. We next performed data augmentation using the Keras image data generator class and generated a dataset of 2200 images. This dataset was split into 2:1 train and test sets and used for training the darknet implementation of YOLO for the custom dataset.
OBJECT DETECTION
The darknet implementation of YOLO was used for object detection. The images were subjected to a convolutional neural network-based feature extraction utilizing the Relu activation. A learning rate of 0.001 was employed and the dataset was divided into batches of 64 images with a mini-batch size of 16. The model was trained for 16000 iterations using Google Colaboratory with a GPU hardware accelerator.
RESULTS
Results were validated by computing the Mean Average Precision (mAP) and F1 scores for the test set every 1000 iterations. Training for 13000 iterations has resulted in optimal F1 and mAP as evident from Figure 1. The average loss reduced to 0.235. The detection resulted in an F1 score of 0.78, mAP (for Intersection over Union of 0.5 and higher) of 78.35%, precision of 0.81 and recall of 0.76. The weights computed at the end of 13000 iterations were utilized for detection. A sample detection is depicted in Figure 2.
DISCUSSION
This study establishes the utility and compatibility of object detection algorithms such as YOLO for automatic crater detection. Improvements to the model can be attempted by incorporating multi-source detection and improving the number and variety of training samples. This technique can also be extended for use with existing global databases for a well-tuned pre-trained model which can be utilized by researchers for crater identification over any region of their study. Further, they can be useful in augmenting and updating planetary databases with minimal manual intervention.
REFERENCES
1. Ivanov, B. A., et al. "The comparison of size-frequency distributions of impact craters and asteroids and the planetary cratering rate." Asteroids III 1 (2002): 89-101.
2. Michael, G. G., and Gerhard Neukum. "Planetary surface dating from crater size-frequency distribution measurements: Partial resurfacing events and statistical age uncertainty." Earth and Planetary Science Letters 294.3-4 (2010): 223-229.
3. Galloway, M. J., et al. "Automated Crater Detection and Counting Using the Hough Transform and Canny Edge Detection." LPICo 1841 (2015): 9024.
4. Yin, Jihao, Hui Li, and Xiuping Jia. "Crater detection based on gist features." IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 8.1 (2014): 23-29.
5. Salamunićcar, Goran, and Sven Lončarić. "Application of machine learning using support vector machines for crater detection from Martian digital topography data." cosp 38 (2010): 3.
6. Silburt, Ari, et al. "Lunar crater identification via deep learning." Icarus 317 (2019): 27-38.
7. Lee, Christopher. "Automated crater detection on Mars using deep learning." Planetary and Space Science 170 (2019): 16-28.
8. Redmon, Joseph, et al. "You only look once: Unified, real-time object detection." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
9. Ester, Martin, et al. "A density-based algorithm for discovering clusters in large spatial databases with noise." Kdd. Vol. 96. No. 34. 1996.
How to cite: Chaudhary, V., Mane, D., Anilkumar, R., Chouhan, A., Chutia, D., and Raju, P.: An Object Detection Approach to Automatic Crater Detection from CTX Imagery, Europlanet Science Congress 2020, online, 21 Sep–9 Oct 2020, EPSC2020-1029, https://doi.org/10.5194/epsc2020-1029, 2020.
