Potentials of Animal-Environment Interactions for Remote Sensing Research: Employing Movement-Tracked Animals as Environmental Informants using Deep Learning

Land use/land cover (LU/LC) classifiers are models that transform environmental data acquired using Earth observation (EO) techniques, such as remote sensing, into interpretable, quantifiable information. Both inside and outside academia, this information plays a crucial role in the understanding of land surface dynamics that result from global processes such as anthropogenic pressure or climate change. Conventional LU/LC classification approaches rely on human-defined classes and the collection, interpretation and labelling of ground truth data to train a classifier, e.g. through field and labelling campaigns. Thus, ground truth data generation is costly, labour-intensive, and tends to be subjective, since labelling relies on human perception and conception of the environment. Animals, like humans, perceive, interpret and differentiate their environment, resulting in distinctive movement behavior. Such is governed by the animals’ needs and the surrounding conditions. Based on this relationship, environmental data, e.g. from EO, have been successfully used to explain animal movement. However, there have been much less examples of conversely attempting to use movement-tracked animals as informants on the environment. This research proposes a semi-supervised Deep Learning (DL) approach employing movement-tracked animals as environmental informants to generate ground truth data for classifying animal land use (ALU) from environmental data. A neural network using bidirectional Long-Short Term Memory (LSTM) learned the relationship between movement trajectory sequences and associated environmental conditions. For this, data of more than 167 million trajectory locations, globally recorded for 7165 individuals of 87 species tracked in the context of 140 independent research studies over 30 years were collected. Alongside, more than 3 billion associated environmental observations were acquired using EO. The resulting model allowed the detection of potential animal-environment interactions (AEI). Using an adaptation of the Siamese neural network architecture, clustering-friendly representations were derived from AEI patterns, based on their similarity and dissimilarity. In an unsupervised setting, AEI representations were clustered into global AEI types. These were then used as ground truth labels to classify ALU from environmental data. The ALU classifier’s applicability was measured within 153 areas of interest (AOI) within Europe, covering an area of 53,167.55 km2 and a decade of yearly composited environmental data, for which ALU was predicted. The robustness of the approach with respect to changes in both trajectory and environmental input data was evaluated by a sensitivity analysis. In a subsequent co-occurrence analysis, ALU was compared to seven well-known human-defined LU/LC reference products to examine how reference class definitions matched with ALU. The proposed approach detected potential AEI in circa 80 % of the provided samples. It achieved an accuracy of 0.881 in AEI representation learning, showing that similarities and dissimilarities between AEI could be learned from the data. Within the AOIs, 343,111 AEI samples were found, nearly all of which were assigned to 45 different AEI types found in unsupervised clustering. Using 80 % of these samples as ground truth in training, ALU classification reached an accuracy of 0.806 (kappa = 0.627). This showed that, overall, classes defined by AEI types were well-differentiable in the environmental data. With an average ground truth spatial density of one sample per 0.51 km2 every three years, the presented approach exceeded sampling densities of state-of-the-art human labelling campaigns such as the European Land Use/Cover Area frame Survey (LUCAS) with 15 km2 per sample/three years. This showcases that AEI modelling and clustering can be used to produce classification ground truth data at a rate difficult and costly for humans to achieve. Sensitivity analysis revealed that ALU classification strongly relied on optical surface reflectances, evaporation, elevation, pressure and temperature data, showing how essential the combination of multiple data sources covering both land surface and atmosphere was. The more species were represented in the input data, the higher was the observed ALU classification accuracy. Larger datasets derived from less species performed poorer, demonstrating that pattern diversity across species was more important than the bare number of included individuals and records. Co-occurrence analysis showed that in comparison to human LU/LC, thematic resolution of ALU varied per land surface type, with higher thematic resolution in natural habitats such as grasslands and lower thematic resolution in urban areas. This showed the potential of increasing thematic resolution by balancing the input data, e.g. towards it explicitly representing species that thrive in urban environments. The study showed that it was possible to model the relationship between animal movement patterns and environmental conditions on a large scale. It also proved that similar interactions of animals with their environment could be globally grouped without requiring a-priori segmenting and labelling of known behavior. Moreover, AEI types were successfully used as labels to classify environmental data. The resulting ALU classifications reflected how the included animals as a whole differentiated their environments, providing a new type of LU/LC product not reliant on human interpretation of the environment. The incorporation of this animal-centric perspective on the land surface into EO could be of interest for many research applications, including land use change analyses, the quantification of the effects of anthropogenic pressure, and potentially any modelling approach currently relying on human LU/LC information. Future research should investigate, how the selection of species specifically adapted to certain habitat types affects thematic resolution of ALU classifications. In conclusion, the study demonstrates how animals can be employed as environmental informants in remote sensing research by using them as landscape interpreters to generate ground truth for land use classifications. It shows how conventional LU/LC ground truth, which is costly, spatio-temporally sparse and subjective due to human interpretation, could be complemented by animal tracking – an emerging, new instrument of Earth observation.