A Dual-Stream Multi-Feature Approach to Whistle Classification Across Species for Passive Acoustic Monitoring

Passive acoustic monitoring (PAM) has emerged as a non-intrusive method for detecting and classifying marine mammal vocalizations in near real time. Yet reliable classification from PAM remains challenging, particularly for whistles—transient, narrowband calls whose rapid temporal variations are difficult to capture from conventional spectral analyses alone. Consequently, existing approaches often rely on species-specific classifiers, which require extensive data collection, annotation, and computational resources, limiting their application across diverse taxa. To overcome these limitations, this study introduces a unified multi-feature framework that integrates (i) temporal, spectral, and cepstral (TSC) features and (ii) spectrogram representations for whistle classification across multiple marine mammal taxa. In addition, to fully exploit these complementary representations, a novel dual-stream architecture comprising a bidirectional long short-term memory (BiLSTM) stream and a time-distributed convolutional neural network (TCNN) + BiLSTM stream is designed to learn from both feature types, enabling richer characterization of whistle dynamics. The framework was evaluated using five-repetition group-aware Monte Carlo cross-validation (MCCV), with recording groups kept mutually exclusive across the training, validation, and test partitions within each repetition. Across beluga, dolphins, narwhal, killer whale, and pilot whale whistles, as well as noise, the proposed model achieved a macro F1 score of 0.944 ± 0.004, outperforming the LSTM (0.918 ± 0.009), CNN (0.712 ± 0.037), and ANN (0.538 ± 0.134) baselines. Paired statistical comparisons also showed higher accuracy for the proposed model than the three baselines across the five MCCV repetitions, with the corresponding differences remaining significant after Holm–Bonferroni correction. These results demonstrate the effectiveness of integrating complementary acoustic representations with bidirectional sequence modeling for robust multi-species whistle classification across unseen recording groups. The proposed framework provides an effective approach for marine mammal whistle classification in PAM applications.