Deep machine learning allows classification of male and female songbird songs in a species with highly variable repertoires

Abstract Deep machine learning is a powerful tool used to streamline data extraction and analysis of large and varied datasets in many biological fields, including animal behaviour. Specifically, these methods can improve detection and classification of acoustic signals to automate extraction of qualifiers such as age, sex, or individual, including in highly variable signals. Still, trade-offs occur between accuracy and precision when signal classes are variable, graded, or rare. Here we evaluate the usefulness of a deep learning approach to detect and classify sex-specific songs in the northern house wren ( Troglodytes aedon ), a songbird with highly variable, sexually dimorphic songs in which females sing during brief periods early in the breeding season. To do this, we used ANIMAL-SPOT, a ResNet18-based deep learning toolkit for training multiclass models. Using multiple datasets and parameters, we achieve an 82–87% accurate detection rate for female house wren songs and 87–88% accurate detection rate for male songs. Therefore, our model could accurately detect most instances of male and female house wren song. False positive rates were low for males (precision 0.94–0.96), but high for females (precision: 0.31–0.58). Female house wren songs are both highly variable, can occur infrequently, and share features with non-sex-specific vocalizations in their repertoires. This posed challenges for creating a single model and an entirely automated workflow to distinguish female songs from other vocalizations without lowering model accuracy or missing rare vocalizations. The higher false positive rate was a conscious decision to favor manual proofing over missing female songs. Our results show that we can detect sex-specific vocalizations with high accuracy, but precision depends on the distinctness of the songs from other vocalizations in the repertoire. We provide suggestions for improved workflows and handling false positives. We envision such approaches will be useful to automate sex-specific monitoring for both behaviour and conservation purposes.