Exploiting facial side similarities to improve AI-driven sea turtle photo-identification systems

Animal photo-identification (photo-ID), the process of identifying individual animals from images, has proven to be a valuable tool for various studies on sea turtles, increasing the knowledge of their ecology and informing conservation efforts. Photo-ID in sea turtles is predominantly based on the geometric patterns of the scales of their two head sides, which are unique to every individual and different from side to side. As such, both manual and automated photo-ID techniques are traditionally performed under a side-specific setting. There, an image showing a single profile of an unknown individual is compared only to images showing the same side of previously identified individuals. In this paper, we show for the first time an inherent visual similarity between left and right facial profiles of the same individuals in three sea turtle species. We do so by employing two state-of-the-art automated neural network-based photo-ID methods, one local feature-based and one deep embedding-based, designed to rank profiles based on their similarities. Both methods rank the similarity of the left and right profiles of the same individual higher than those of different individuals. These similarities are detectable even when images are taken years apart under diverse conditions. We further show that the exploitation of this similarity results in improved accuracies when compared to the traditional side-specific photo-ID setting. Our results indicate two concrete guidelines for improving automated sea turtle photo-ID workflows. When trying to match a photo of a given profile, searches should not be restricted only to photos of the same profile. As the first method of choice, a deep embedding model finely-trained using a photo-database of the focal sea turtle population should be used. In the absence of such training database, a neural network-based local feature method is preferable, but in that case searches should be performed with both the original query image and its horizontally flipped version. • We show for the first time an inherent visual similarity between left and right facial profiles of the same individuals in three sea turtle species. • Deep learning based methods rank the similarity of the left and right profiles of the same individual higher than those of different individuals. • Exploitation of this similarity can be translated to improved accuracies when compared to the traditional side-specific photo-ID setting. • Both profiles should always be included in the search database when matching a photo of a given profile. • As the first method of choice, a deep embedding model finely trained with a part of the database should be used. Without such a training database, a neural network-based local feature method is preferable.