Oral presentations and abstracts

Scope

Planetary robotic missions contain vision instruments for various mission-related and science tasks such as 2D and 3D mapping, geologic characterization, atmospheric investigations, or spectroscopy for exobiology. One major application for computer vision is the characterization of scientific context, and the identification of scientific targets of interest (regions, objects, phenomena) for being investigated by other scientific instruments. Due to high variability of appearance of such potentially scientific targets it requires well-adapted yet flexible techniques, one of them being Deep Learning. Machine learning and in particular Deep Learning (DL) is a technique used in computer vision to recognize content in images, categorize it and find objects of specific semantics.

In its default workflow, DL requires large sets of training data with known / manually annotated objects, regions or semantic content.  Within Mars-DL (Planetary Scientific Target Detection via Deep Learning), training focuses on a simulation approach, by virtual placement of known targets in a true context environment.

Mars-DL Approach

The Mars-DL workflow is depicted in Figure 1, with following details:

• 3D background for the training & validation scenes is taken from 3D reconstructions using true Mars rover imagery (e.g. from the MSL Mastcam instrument).
• Scientifically interesting objects to be virtually placed in the scene are chosen to be 3D reconstructions of shatter cones (SCs) and/or Meteorites as representative set. The SCs used for 3D reconstruction are from different terrestrial impact structures and cover a wide range of lithologies, including limestone, sandstone, shale and volcanic rocks. Typical features of SCs such as striations and horsetail structure are well developed in all selected specimens and allow a clear identification. The involved stony and iron meteorites are museum quality specimens (NHM Vienna) with fusion crust and/or regmaglypts.
• A capturing campaign at the NHM Vienna covered more than a dozen SC and meteorite specimen, imaged from various positions under approximate ambient illumination conditions. This was followed by a 3D reconstruction process using photogrammetric techniques (Figure 2) to obtain a seamless watertight high resolution mesh and albedo map for each object, gaining a 3D data base of objects to be randomly placed in the realistic scenes.
• Simulation of rover imagery for training is based on PRo3D, a viewer for the interactive exploration and geologic analysis of high-resolution planetary surface reconstructions [1]. It was adapted to efficiently render large amounts of images in batch mode controlled by a JSON file. SCs are automatically positioned on a chosen surface region with a Halton sequence, whereby collisions are avoided. The SCs are color adjusted to perfectly blend into the scene and in a future version will also receive and cast shadows. Special GPU shaders provide additional types of training images such as masks and depth maps. A typical rendering data set to be used for training is depicted in Figure 4.
• A random pose generator selects representative yet random positions and fields-of-view for the training images.
• Unsupervised generative-adversarial neural image-to-image translation techniques (GAN; [2]) are then applied on the background of the generated images, thus producing realistic landscapes drawn from the domain of actual Mars rover datasets while preserving the integrity of the high-resolution shatter cone models.
• The candidate meta-architectures for the machine learning approach are Mask R-CNN (instance segmentation) and Faster R-CNN (object detection), as these currently belong to the best performing architectures for these tasks. These meta-architectures can be configured with different backbone networks (e.g. ResNet-50, Darknet-53), which provide a tradeoff between network size and convergence rate.
• A remaining unused training set will be used for evaluation.

Summary and Outlook

So far science autonomy has been addressed only recently with increased mobility on planetary surfaces and the upcoming need for planetary rovers to react autonomously to given science opportunities, well in view of limited data exchange resources and tight operations cycles.

An Austrian Consortium in the so-called "Mars-DL" project is targeting the adaptation and test of simulation and deep learning mechanisms for autonomous detection of scientific targets of interest during robotic planetary surface missions.

For past and present missions the project will help explore & exploit further existing millions of planetary surface images that still hide undetected science opportunities.

The exploratory project will assess the feasibility of machine-learning based support during and after missions by automatic search on planetary surface imagery to raise science gain, meet serendipitous opportunities and speed up the tactical and strategic decision making during mission planning. An automatic “Science Target Consultant” (STC) is realized in prototype form, which, as a test version, can be plugged in to ExoMars operations once the mission has landed.

In its remaining runtime until end of 2020, Mars-DL training and validation will explore the possibility to search scientifically interesting targets across different sensors, investigate the usage of different cues such as 2D (multispectral / monochrome) and 3D, as well as spatial relationships between image data and regions thereon.

During mission operations of forthcoming missions (Mars 2020, and ExoMars & Sample Fetch Rover in particular) the STC can help avoid the missing of opportunities that may occur due to tactical time constraints preventing in-depth check of image material.

Acknowledgments

This abstract presents the results of the project Mars-DL, which received funding from the Austrian Space Applications Programme (ASAP14) financed by BMVIT, Project Nr. 873683, and project partners JOANNEUM RESEARCH, VRVis, SLR Engineering, and the National History Museum Vienna.

References

[1] Barnes R., Sanjeev G., Traxler C., Hesina G., Ortner T., Paar G., Huber B., Juhart K., Fritz L., Nauschnegg B., Muller J.P., Tao Y. and Bauer A. Geological analysis of Martian rover-derived Digital Outcrop Models using the 3D visualisation tool, Planetary Robotics 3D Viewer - PRo3D. In Planetary Mapping: Methods, Tools for Scientific Analysis and Exploration, Volume 5, Issue 7, pp 285-307, July 2018

[2] Jun-Yan Zhu, Taesung Park, Phillip Isola, Alexei A. Efros; Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. The IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2223-2232

How to cite: Paar, G., Traxler, C., Nowak, R., Garolla, F., Bechtold, A., Koeberl, C., Fernandez Alonso, M. Y., and Sidla, O.: Mars-DL: Demonstrating feasibility of a simulation-based training approach for autonomous Planetary science target selection, Europlanet Science Congress 2020, online, 21 Sep–9 Oct 2020, EPSC2020-189, https://doi.org/10.5194/epsc2020-189, 2020.

The H2020 Europlanet-2020 programme, which ended on Aug 31^st, 2019, included an activity called VESPA (Virtual European Solar and Planetary Access), which focused on adapting Virtual Observatory (VO) techniques to handle Planetary Science data [1] [2]. The outcome of this activity is a contributive data distribution system where data services are located and maintained in research institutes, declared in a registry, and accessed by several clients based on a specific access protocol. During Europlanet-2020, 52 data services were installed, including the complete ESA Planetary Science Archive, and the outcome of several EU funded projects. Data are described using the EPN-TAP protocol, which parameters describe acquisition and observing conditions as well as data characteristics (physical quantity, data type, etc). A main search portal has been developed to optimize the user experience, which queries all services together. Compliance with VO standards ensures that existing tools can be used as well, either to access or visualize the data. In addition, a bridge linking the VO and Geographic Information Systems (GIS) has been installed to address formats and tools used to study planetary surfaces; several large data infrastructures were also installed or upgraded (SSHADE for lab spectroscopy, PVOL for amateurs images, AMDA for plasma-related data).

In the framework of the starting Europlanet-2024 programme, the VESPA activity will complete this system even further: 30-50 new data services will be installed, focusing on derived data, and experimental data produced in other Work Packages of Europlanet-2024; connections between PDS4 and EPN-TAP dictionaries will make PDS metadata searchable from the VESPA portal and vice versa; Solar System data present in astronomical VO catalogues will be made accessible, e.g. from the VizieR database. The search system will be connected with more powerful display and analysing tools: a run-on-demand platform will be installed, as well as Machine Learning capacities to process the available content. Finally, long-term sustainability will be improved by setting VESPA hubs to assist data providers in maintaining their services, and by using the new EU-funded European Open Science Cloud (EOSC). In addition to favoring data exploitation, VESPA will provide a handy and economical solution to Open Science challenges in the field.

The Europlanet 2020 & 2024 Research Infrastructure project have received funding from the European Union's Horizon 2020 research and innovation programme under grant agreements No 654208 & 871149.

[1] Erard et al 2018, Planet. Space Sci. 150, 65-85. 10.1016/j.pss.2017.05.013. ArXiv 1705.09727  

[2] Erard et al. 2020, Data Science Journal 19, 22. doi: 10.5334/dsj-2020-022.

How to cite: Erard, S., Cecconi, B., Le Sidaner, P., Rossi, A. P., Brandt, C., Rothkaehl, H., Tomasik, L., Ivanovski, S., Molinaro, M., Schmitt, B., Génot, V., André, N., Vandaele, A. C., Trompet, L., Scherf, M., Hueso, R., Määttänen, A., Millour, E., Schmidt, F., and Waldmann, I. and the VESPA team: Virtual European Solar & Planetary Access (VESPA): Progress and prospects , Europlanet Science Congress 2020, online, 21 Sep–9 Oct 2020, EPSC2020-190, https://doi.org/10.5194/epsc2020-190, 2020.

The European Space Agency conducts planetary science and heliophysics investigations with a set of interplanetary missions such as Mars Express and ExoMars 2016 –currently in operations-, Rosetta, Venus Express, SMAR-1, Huygens and Giotto –in legacy phase- and will do so with BepiColombo, Solar Orbiter, ExoMars 2020, Jupiter Icy Moons Explorer (JUICE) and Hera missions. To support the study phase, planning, operations, data analysis and exploitation processes, the availability of ancillary data and other geometrically relevant models (spacecraft and related natural bodies trajectory and orientation, reference frames, digital shape models, science payload modeling, etc.) is a must.

SPICE is an information system the purpose of which is to provide scientists the observation geometry needed to plan scientific observations and to analyze the data returned from those observations. SPICE is comprised of a suite of data files, usually called kernels, and software -mostly subroutines.  A customer incorporates a few of the subroutines into his/her own program that is built to read SPICE data and compute needed geometry parameters for whatever task is at hand. Examples of the geometry parameters typically computed are range or altitude, latitude and longitude, phase, incidence and emission angles, instrument pointing calculations, and reference frame and coordinate system conversions. SPICE is also very adept at time conversions.

The ESA SPICE Service (ESS) leads the SPICE operations for ESA missions. The group generates the SPICE Kernel datasets (SKD) for missions in missions in and legacy). ESS is also responsible for the generation of SPICE Kernels for Solar Orbiter and Hera. The generation of these datasets includes the operation software to convert ESA orbit, attitude and spacecraft clock correlation data into the corresponding SPICE format. ESS also provides consultancy and support to the Science Ground Segments of the planetary missions, the Instrument Teams and the science community. ESS works in partnership with the Navigation and Ancillary Facility (NAIF), a group at the Jet Propulsion Laboratory (JPL/NASA) originator and responsible of evolving and maintaining the SPICE system components.

The quality of the data contained on a SKD is paramount. Bad SPICE data can lead to the computation of wrong geometry and wrong geometry can jeopardize science results. ESS, in collaboration with NAIF is focused on providing the best SKDs possible. Kernels can be classified as Setup Kernels (Frame Kernels that describe Reference Frames of a given S/C, Instrument Kernels that describe a given sensor FoV and other characteristics, etc.) and Time-varying Kernels (SPK and CK kernels that provide Trajectory and Orientation data, SCLK that provide Time Correlation Data, etc.). Setup Kernels are iterated with the different agents involved in the determination of the data contained in those kernels (Instrument Teams the Science Ground Segment, etc.) and Time-varying kernels are automatically generated by the ESS SPICE Operational pipeline to feed the Operational kernels that are used in the day-to-day work of the missions in operations (planning and data analysis). These Time-varying kernels are peer-reviewed a posteriori for the consolidation of SKDs that are archived in the PSA and PDS. This contribution will outline the status of the SKDs maintained by ESS.

ESS offers other services beyond the generation and maintenance of SPICE Kernels datasets, such as configuration and instances for WebGeocalc (WGC) and SPICE-Enhanced Cosmographia (COSMO) for the ESA Missions WGC provides a web-based graphical user interface to many of the observation geometry computations available from SPICE. A WGC user can perform SPICE computations without the need to write a program; the user need have only a computer with a standard web browser. COSMO is an interactive tool used to produce 3D visualizations of planet ephemerides, sizes and shapes; spacecraft trajectories and orientations; and instrument field-of-views and footprints. This contribution will outline how the Cosmographia and WGC instances for ESA Planetary missions can be used, out of the box, for the benefit of the science community.

How to cite: Costa Sitjà, M., Escalante López, A., and Arviset, C.: An update of SPICE for ESA solar system exploration missions, Europlanet Science Congress 2020, online, 21 Sep–9 Oct 2020, EPSC2020-204, https://doi.org/10.5194/epsc2020-204, 2020.

The ESAC Science Data Centre (ESDC) is in the process of registering Digital Object Identifiers (DOIs) for datasets or group of datasets accessible across the ESA Space Science Archives managed by ESDC. These DOIs are persistent URL that point to DOI landing pages setup and managed by ESDC, actually located at https://www.cosmos.esa.int/web/esdc/doi

In the heliophysics domain, the first step has been to register DOIs related to the datasets measured by experiments onboard ESA heliophysics spacecraft. Around 60 experiments are flying or have been flown so far onboard ESA heliophysics missions. At the moment, 47 DOI have been registered with CrossRef pointing to DOI landing pages for each experiment onboard SOHO, Proba-2, Cluster, Double Star, Ulysses and ISS-Solaces and are publicly accessible at https://www.cosmos.esa.int/web/esdc/doi/heliophysics. Discussions are on-going with editors of major journal to promote and acknowledge the use of data from any of these experiments by citing these DOIs. Eventually, this would improve the traceability of the usage of datasets from these experiments.

Additionally, a JavaScript Object Notation (JSON) script has been added to these DOI landing pages to make them discoverable through Google Dataset Search (GDS). GDS is a new search engine that helps users to find datasets online. Twenty five millions datasets were already indexed and searchable when this service was launched in early 2020, after a beta version released in late 2018. Thanks to a close interaction with the PI teams, DOIs for ESA heliophysics experiments are now easily discoverable through GDS and not only by looking for a particular experiment name. The use of structured metadata, based on schema.org, has enabled the search through physical processes investigated by these experiments, the type of experiments, the time coverage, the PI names or their affiliation… Various examples will be provided while the next steps of development will be outlined.

How to cite: Masson, A., De Marchi, G., Merin, B., Henar, M., and Wenzel, D.: Google Dataset Search and DOI for data in the ESA Space Science Archives, Europlanet Science Congress 2020, online, 21 Sep–9 Oct 2020, EPSC2020-285, https://doi.org/10.5194/epsc2020-285, 2020.

Solar System Treks Mosaic Pipeline (SSTMP) is a new, open-source tool for generation of planetary DEM and orthoimage mosaics. Opportunistic stereo reconstruction from pre-existing orbital imagery has in the past typically required significant human input, particularly in the pair selection and spatial alignment steps. Previous stereo mosaics incorporate myriad human decisions, compromising the reproducibility of the process and complicating uncertainty analysis. Lack of a common framework for recording operator input has hindered the community's ability to collaborate and share experience to improve stereo reconstruction techniques. SSTMP provides a repeatable, turnkey, end-to-end solution for creating these products. The user requests mosaic generation for a bounding box or polygon, initiating a workflow which results in deliverable mosaics usable for site characterization, science, and public outreach.

The inital release of SSTMP focuses on production of elevation and orthoimage mosaics using data from the Lunar Reconaissance Orbiter's Narrow Angle Camera (LRO NAC). SSTMP can automatically select viable stereo pairs, complete stereo reconstruction, refine alignments using data from the LRO's laser altimeter (LOLA), and combine the data to produce orthoimage, elevation, and color hillshade mosaics.

SSTMP encapsulates the entire stereo mosaic production process into one workflow, managed by Argo Workflow opensource Kubernetes-based software. Each process runs in a container including all tools necessary for production and geospatial analysis of mosaics, ensuring a consistent computing environment. SSTMP automatically retrieves all necessary data. For processing steps, it leverages free and open-source software including Ames Stereo Pipeline, USGS ISIS, Geopandas, GDAL, and Orfeo toolbox.

How to cite: Curtis, A., Law, E., Malhotra, S., Day, B., Trautman, M., Gallegos, N., and Nainan, C.: The Solar System Treks Mosaic Pipeline (SSTMP), Europlanet Science Congress 2020, online, 21 Sep–9 Oct 2020, EPSC2020-425, https://doi.org/10.5194/epsc2020-425, 2020.

Abstract

In this work, we demonstrate techniques and results of a 3D mosaic of  the whole of the Valles Marineris (VM) area of Mars using stereo images from the Mars Express High Resolution Camera (HRSC), as well as some examples of multi-resolution orthoimage and 3D mapping of 3 selected sites where recurring slope lineae (RSL) are common, at Coprates Montes, Nectaris Montes, and Capri Chaos, using Mars Reconnaissance Orbiter (MRO) Context Camera (CTX), High Resolution Imaging Science Experiment (HiRISE) and the Compact Reconnaissance Imaging Spectrometer for Mars (CRISM).

Introduction

Valles Marineris is the largest system of canyons on Mars that is more than 4,000 km long, 200 km wide, and up to 10 km deep. 3D mapping is essential to improving our understanding of the geological environment of Valles Marineris. Typically, deriving a co-aligned and mosaiced 3D base map covering the whole area is generally the starting point of such studies.This work focuses on a demonstration of multi-resolution 3D mapping and co-alignment of different data sources over the VM area using a co-registered and nested HRSC-CTX-CRISM-HiRISE dataset.

3D modelling

Previously, within the EU FP-7 iMars (http://www.i-mars.eu) project, we developed an automated multi-resolution Digital Terrain Model (DTM) processing chain for NASA CTX and HiRISE stereo-pairs, called the Co-registration ASP-Gotcha Optimised (CASP-GO), based on the open source NASA Ames Stereo Pipeline (ASP) [1], tie-point based multi-resolution image co-registration [2], and the Gotcha [3] sub-pixel refinement method. The CASP-GO system guarantees global geo-referencing congruence with respect to the aerographic coordinate system defined by HRSC, level-4 products and thence to the MOLA DTM, providing much higher resolution stereo derived DTMs.

3D area mosaic using HRSC

HRSC is now on its ~5,700^thorbit onboard Mars Express, (totalling>20,000 HRSC products) covering ~98% of the Martian surface at a spatial resolution higher than 100 m/pixel. Among these, DLR has processed HRSC stereo images (level 4 DTMs, [4]), at 50-150m/pixel resolution, that now cover ~50% of the planet’s surface. At Valles Marineris, the DLR HRSC level 4 DTMs have covered a large portion of the canyons, leaving several large gaps unprocessed or having missing data.

We created a further 11 HRSC single strip DTMs and Orthorectified images (ORIs) using HRSC level 2 stereo images to cover these gaps. These are processed using CASP-GO [2] and co-registered with the existing DLR HRSC level 4 DTMs and ORIs to allow seamless mosaicing for the whole of Valles Marineris. Also, to eliminate the need for bundle block adjustment, both UCL and DLR HRSC level 4 DTMs are co-aligned with the MOLA DTM using an iterative closest point co-alignment which is part of ASP [1]. Figure 1 shows a colourised and hill-shaded DTM of the whole area. HRSC level 4 ORIs are then generated using this mosaiced DTM are radiometrically corrected and brightness/contrast adjusted to produce a corresponding HRSC ORI mosaic [5].

Figure 1 HRSC DTM merged DLR + UCL mosaic of VM at 50m/gridpoint

Figure 2 HRSC ORI mosaic of VM generated at 12.5m after brightness adjustment [5]

Automated co-registration of CTX, HiRISE and CRISM

In order to study RSL features, we produce stacks of multi-resolution products using the HRSC DTM and ORI as the basemap. We here show an example of one of the CTX DTMs and ORIs for Capri Chaos which is one of the three selected study sites. The CTX ORIs are co-registered with the HRSC ORI mosaic using the tie-point based image co-registration pipeline described in [2]. The CTX DTMs are then spatially adjusted according to the ORI tie-point and co-aligned vertically to the HRSC DTM mosaic using a B-Spline fitting method to eliminate any residual jitter. The spatial agreement between CTX ORIs and HRSC ORI mosaic is at subpixel level of the base image (~5m). The vertical agreement between CTX DTMs and HRSC/MOLA DTM mosaic can be reduced from several hundreds of metres to less than 10 m (on average) and 50 m (at maximum).

Subsequently, we have also corrected the UoA HiRISE ORIs and DTMs using the same joint image co-registration and vertical alignment method to CTX basemap. The CRISM data is also co-registered with the CTX ORI using a modified version of the tie-point based image co-registration pipeline, bringing several hundreds of metres of spatial misalignment to less than 18 metres. An example of the co-registered nested stack is shown in Figure 3 and of the CTX DTM in Figure 4.

Figure 3 Co-registered HiRISE ORI (at 90% transparency) and unco-registered UoA HiRISE ORI (in red), co-registered CRISM (at 90% transparency) and unco-registered raw CRISM (blue) image, shown on top of the UCL CTX ORI and HRSC ORI mosaic at Capri Chaos in the Valles Marineris.