A new tool for studying active surface processes with improved image alignment.

Introduction
NASA's Mars Reconnaissance Orbiter (MRO [1]) has been in operation for over 20 years, with a suite of instruments allowing a detailed study of martian surface and subsurface, most notably the High Resolution Imaging Science Experiment (HiRISE [2]), which offers the highest spatial resolution in the history of martian orbital imagery - down to 25 cm/px. Numerous sites have been photographed more than once, either by chance, for the sake of generating stereopairs for local digital elevation models, or for other contextual investigations of planned rover landing sites or locations of interest [3]. This can be advantageous for researching periglacial, eolian, mass wasting or other active surface processes on Mars[4-5]. However, due to geometrical effects, such as observation emission and phase angle differences, georeferencing cannot provide sufficiently accurate information to align such high resolution images in geospatial software, which hinders efficient studies based on comparing such images. Although computer vision algorithms for aligning images based on keypoint matching have been known for a long time, they had not yet been leveraged to alleviate this issue.  

HiLiner
We present to the Mars research community a newly developed desktop application, under the name of HiLiner - High resolution image aLi(g)ner. The tool allows to inspect a pair of overlapping images by means of interactive panning and zooming, with two display modes: side-by-side and overlaid on top of each other with varying opacity. The latter mode also allows the user to activate flickering, which alternates between full opacity and full transparency of the overlaid image - the most convenient feature for scanning the area in search of changes between the two images. It is also possible to adjust the color space of each image (brightness/contrast/pseudocolor) and the blending mode in the overlay case.

While the above mentioned capabilities are standard for a geospatial software, the focus of HiLiner is on alignment between the images, where several possibilities are offered. When the user decides to bind the images, alignment based on georeferencing is applied instantaneously, and meanwhile a computation of a shift dictionary is launched in the background. The algorithm recursively descends down the cached deep zoom pyramid, with which the image is represented in the software, finding corresponding tile pairs and using the SIFT algorithm [6] to infer the shift between them. The alignment is then applied as a correction to the georeferencing case which is obtained by averaging shifts inferred for the four nearest tiles to the current central pixel, in a manner analogous to bilinear interpolation, but weighted by the number of keypoints found in each tile. A significant improvement can be observed thanks to this correction, particularly in areas with a lot of detail that can serve as relevent keypoints (e. g. crater rims).

Nevertheless, in many cases a more satisfying alignment can be achieved by using a dynamic alignment mode, which performs an on-the-fly comparison of the scopes of both images currently visible in the software, and applies an optimal matching homography as a transformation on the latter. In this mode, a small delay required for the computation of the on-the-fly transformation is inevitable, nevertheless, it does not have a strong negative impact on the user flow.

Finally, it is also possible to activate a manual mode, where a user can impose additional corrections on the respective shift of images by means of mouse dragging or arrow movement. Obviously, this mode should be treated as a last resort solution, since the point of the software is to alleviate the burden of manual alignment.
To facilitate transmitting locations of observed changes to other geospatial software, we also incorporated a feature called Interest Site Manager. Regardless of the current visualization and alignment mode, the user can select and name points of interest, which can then be exported as a shapefile, sharing the coordinate reference system of the original images.  

Technical aspects
The tool is open source, written entirely in Python (using the PyQt6 framework) and published on GitLab. For Linux users, a dedicated installation script is provided, which install HiLiner as a full-fledged desktop application. In particular, saved projects with a custom .hlnr extension can then be opened directly in HiLiner by double-clicking on the file. On other operating systems, the tool can be run from Anaconda Prompt after creating a dedicated environment. Projects can then still be saved and loaded from within the tool.
As an important practical note, the application folder can grow considerably in size over time, due to the need of storing cached deep zoom pyramids and shift dictionaries. If using extensively and for a broad suite of images (the same image will only be cashed once as it is saved under the folder named after a hash of its contents), it is recommended to occasionally purge these cashes, either manually in a selective manner, or by launching a dedicated cleansing action within HiLiner.

Future prospects
Development and testing of HiLiner have been specifically focused on pairs of monochrome HiRISE images. Although the abbreviated name makes a direct reference to HiRISE, the extended name High resolution image aLi(g)ner expresses a wish for usability with pairs of any kind of images both within, and outside of the realm of geospatial raster data. These capabilities have partially already been implemented - indeed, it possible to view not only monochrome, but also color or even multispectral images, and basic georeferencing alignment can be properly visualized regardless whether the resolutions of both images match or not. However, the tool has yet to be optimized for use with such other kinds of data. 

Acknowledgements
This research work was funded by GeoPlaNet programme (https://geoplanet-impg.eu/). 

References

[1] Zurek et al. (2007), JGR: Planets, 112.

[2] McEwen et al. (2007), JGR: Planets, 112.

[3] Kirk et al.(2008), JGR: Planets, 113.

[4] Dundas et al. (2017), Nature Geoscience, 10, 903-907.

[5] Diniega et al. (2013), Icarus, 225, 526-537.

[6] Lowe (2004), International Journal of Computer Vision, 60, 91-110