Flyover videos from annotated 3D Martian environments within minutes: The PRo3D Sequence Bookmark Facility

Background: Visualizing Mastcam-Z Stereo Reconstructions on Global-to-Local Scale

Mars 2020 Mastcam-Z [1] 3D products enable the visualization and analysis of rocks, outcrops, and other geological and aeolian features. The Planetary Robotics Vision Processing framework PRoViP [2], among others, generates panoramas, textured meshes and derived products from Mastcam-Z sequences on tactical (Sol-by-Sol) time frame. OPCs (Ordered Point Clouds) [3] georeferenced by the cm-level accurate tactical rover localization provided by the Mapping Specialists on the Mars 2020 science team, are loaded in PRo3D (Planetary Robotics 3D Viewer; [4]) for interactive 3D exploration, measurement and annotation. A “scene” contains several OPCs, overlaid to textured HiRISE [5] Digital Terrain Models – DTMs  – again OPCs – for large-scale spatial context (Figure 1).

Figure 1: Mastcam-Z Sol 369 OPCs on HiRISE OPC. NASA/JPL/CalTech/USGS/ASU/MSSS/JR/VRVis

Narrative Videos for Immersive Story Telling of Scientific Rationales

A typical workflow to explain physical processes on a planetary surface based on its 3D virtual representation incorporates the following steps:

• A spatial and thematic overview shows the environment in large-scale context, typically in km-scale, involving also the mission progress such as Rover trajectory, local landmark features and scale bars. The latter is true of all steps to follow
• Moving towards a specific region, zooming-in, letting new surface layers appear, or annotating a target of interest attracts attention to a more focused part of the landscape
• Speed & timing of virtual motion (“Navigation” in the scene) varies tension, or allows time for spoken or annotated explanations
• Different viewpoints and smoothly transiting between them highlights specific features.

PRo3D’s Sequenced Bookmarks: Key for Efficient Video Production

PRo3D implements such a workflow in a repeatable way. “Bookmarks” define 3D viewpoints, their sequence, duration and the transition times between them (Figure 2). Disabling/enabling the display of scale bars, individual surfaces and annotations, rover tracks and stations, dip-and-strike annotations, measurements and sky projections of annotations, in conjunction with the embedding of photospheres and multi-temporal surface layers, realizes a comfortable production of highly informative yet ergonomic video stories launched by a “play” button. Up to 8k resolution offline rendering produces smooth videos even on low-budget graphics cards.

Figure 2: PRo3D Sequenced Bookmarks user interface (Mastcam-Z Sol 422) ASU/MSSS/JR/VRVis

Science Case 1: Explaining the 3D Assessment of Soil Crust

Understanding the Martian soil is critical for future human exploration, and samples that will be returned [6]. 3D observations of cracks, fractures and tracks sheds important light on the crust’s characteristics and extent of induration [7]. Figure 3 shows key frames from a PRo3D video explaining a soil crust assessment process. Important building blocks are the Rover tracks’ depth or the steepness of micro slopes. The viewing direction should be parallel to the rills of the rover tracks, the viewing distance from the Rover and the virtual viewing distance shall be small.   

Figure 3: Soil crust evaluation stages documented using PRo3D’s Sequenced Bookmarks video-rendering. a) Environment, b) approaching the rover tracks, c) side view parallel to track rills with numeric assessments, d) close-range numeric assessments of a micro slope. Two Mastcam-Z OPCs from Sol 329 (zcam07106) and Sol 333 (zcam08354) overlain on HiRISE DTM/Ortho OPC. NASA/JPL/CalTech/ASU/MSSS/ÖAW/JR/VRVis

Science Case 2: Explaining the Result of a Geologic Annotation

A Digital Outcrop Model’s nature can be difficult to convey in simple 2D images, due to the different types of data collected – interpreting layering characteristics by lines, the orientation or dip and strike of exposed layers and bedding, thickness of layers, variation in thickness (up and down as well as laterally), and locations to build the interpretation on the spatial distribution of key textural and geometrical characteristics [4]. Collating such measurements carried out over multiple outcrops further increases the complexity of concisely sharing observations. Sequenced bookmarks allow for accessible visualisation of such multifaceted data presentation. This is highlighted in the interpretation of the Mure outcrop imaged by Mastcam-Z on Sols 168-169 (sequences 8180/8182/8183, Figure 4).

Figure 4: PRo3D’s Sequenced Bookmarks video rendering documenting geologic workflow stages. a. – Initial approach and (scaled) assessment of outcrop. b. – Layer contacts’ interpretation and annotation, visualizing geometry and dimensions. c. – Dip and strike of key surfaces. d. –Layer true thickness.

Outlook: Dynamically Adapting the Scene and Further Exploitation Cases

Functions for dynamic change of the scene whilst animating, sequenced toggling of surfaces, annotations and scale bars, and additional features such as color change in a predefined sequence are presently implemented. Such way, the 3D dynamic presentation of an exploration scenery will get a standard and low-effort means for immersively sharing observations and findings. Manifold application cases, not restricted to the Martian environment, contain the 3D explanation of dynamic processes such as landing [8], atmospheric processes (e.g. dust devils’ motion), the accumulation of measurements as the rover drives, or 3D change documentation [9].

Acknowledgements

JR, VRVis and ÖAW receive funding from ESA PRODEX Contract PEA4000117520 and FFG ASAP Projects 882828 and 885326. VRVis is funded by BMK, BMDW, Styria, SFG, Tyrol and Vienna Business Agency in the scope of COMET - Competence Centers for Excellent Technologies (879730) managed by FFG. We thank Rebecca Nowak for implementing the Sequenced Bookmarks suite.

References

[1] Bell et al., Space Science Reviews, 2021;  [2] Traxler et al., 2022 https://doi.org/10.1002/9781119313922.ch3; [3] Ortner et al., Real Corp, 2010; [4] Barnes et al., ESS, 2018; [5] McEwen et al., JGR-Planets, 2007; [6] Beaty et al., MPS, 2019; [7] Hausrath, in submission, 2022; [8] Perseverance Descent Reconstruction in PRo3D, https://www.youtube.com/watch?v=-z1KySwtnuU; [9] QZCAM 0268 0255 ZCAM03258 8278 L0R0 110 034 HiRISE PRo3D Brac SampleHole1 E01, https://www.youtube.com/watch?v=5Sn4MpZdgpg