animal2vec and <scp>MeerKAT:</scp> A self‐supervised transformer for rare‐event raw audio input and a large‐scale reference dataset for bioacoustics

Abstract Bioacoustic research, vital for promoting conservation and understanding animal behaviour and ecology, faces a monumental challenge: analysing vast datasets where animal vocalizations are rare. While deep learning techniques are becoming standard, adapting them to bioacoustics remains difficult. We address this challenge with animal2vec, an interpretable large transformer model and a self‐supervised training scheme tailored for sparse and unbalanced bioacoustic data. It learns from unlabelled audio and then refines its understanding with labelled data. Furthermore, we introduce and publicly release MeerKAT: Meer kat K alahari A udio T ranscripts, a dataset of meerkat ( Suricata suricatta ) vocalizations with millisecond‐resolution annotations, the largest labelled dataset on a non‐human terrestrial mammal currently available. Our model sets a baseline on the MeerKAT corpus, outperforming other transformer models, and improves on existing methods on the publicly available NIPS4Bplus birdsong dataset. Moreover, animal2vec performs well even with limited labelled data (few‐shot learning). animal2vec and MeerKAT provide a new reference point for bioacoustic research, enabling scientists to analyse large amounts of data even with scarce ground truth information.