Combining rover images and satellite data in the process of automatic mapping of aeolian formations on the Martian Surface

The purpose of this research was to test the feasibility of using multi-source data acquired from both satellite imagery (MRO mission's HIRISE camera) and the Opportunity rover NAVCAM camera in the process of automatic mapping of aeolian formations on Mars.Both image data, digital terrain model and its derived models, i.e. topographic position index and terrain ruggedness index were used in this process.The research tested the possibility of combining these diverse data sources in the process of supervised learning of a convolutional neural network performing the task of semantic segmentation of the Martian surface.The possibilities of automatic mapping of aeolian forms were verified on the Meridiani Planum area.HiRISE satellite data in the form of stereo pairs were downloaded from the Planetary Data System , McEwen (2007).The data were processed into orthoimages and terrain models using NASA ASP (Beyer A. ( 2018)).The NAVCAM camera images were used, as they provides the most information about the terrain context.A modified approach used in Ceo et al. ( 2018) was utilized to process the data acquired from the rover.In the first step, it was necessary to convert the individual images taken at each location into panoramic image.Using additional metadata, spatially localized panoramas were produced, which were then used for feature extraction and interpolation.The semantic segmentation process used an encoder -decoder neural network based on the VGG-16 architecture (Simonyan K. et al. (2015)).The mapping system outlined above was then trained using expert-labelled learning data.This yielded very good semantic segmentation results both with and without using information from the rover.It achieved an overall accuracy of 94.38% with only the use of image orbital data, 94.90% adding terrain information, and 95.94% using all available data.The entire study was published in Choromaski et al. (2022).