The Use of Pretrained Convolutional Neural Networks in Recognizing Phytoplankton Species. Cases from a marine, a brakishwater and a freshwater site.

Introduction Phytoplankton are microscopic organisms that form the foundation of aquatic food webs. Accurate identification and classification of phytoplankton species are crucial for monitoring all aquatic ecosystems, from marine to freshwater, understanding ecological dynamics, and assessing environmental changes. Traditional methods of phytoplankton identification, which rely on manual microscopy, are time-consuming and require expert knowledge. Recent advancements in machine learning, particularly Convolutional Neural Networks (CNNs), offer promising solutions for automating this process. This abstract explores the application of pre-trained CNNs in recognizing phytoplankton species, highlighting their advantages, methodologies, and potential impacts. Methodology We present three approaches from a marine site, the Gulf of Venice site of the LTER-Italy network (DEIMS.ID https://deims.org/758087d7-231f-4f07-bd7e-6922e0c283fd), which includes the 'Acqua Alta' Oceanographic Tower (AAOT) (Fig. 1), the brackishwater site Utö Atmospheric and Marine Research Station (ResNet-18, located at 59°46.84’ N, 21°22.13’ E) https://en.ilmatieteenlaitos.fi/uto, and the freshwater site the IGB-LakeLab in Lake Stechlin NE Germany (DEIMS.ID https://deims.org/2223bc9c-12b2-49fe-af73-4299f553e054). Three different architectures of CNN were used: VGG16 for the Gulf of Venice, ResNet-18 for the Finnish station and a YOLOv11-cls for the German Lake Stechlin LakeLab station. These CNN models were pre-trained on the ImageNet dataset and subsequently fine-tuned with specific datasets for the respective geographic areas. These CNNs were chosen for their ability to autonomously extract features from images without external assistance, making them efficient, fast tools for analyzing large amounts of data and due to their specificity regarding the characteristics of the observational site. The process involves several steps: Data Collection and Preprocessing : several public datasets are available (Ciranni et al. 2024), where each image is annotated according to its class. Each model is structured to require input images in a specific format, so depending on the chosen model, it is necessary to preprocess the images accordingly. With an Imaging Flow Cytobot (IFCB, an in-situ automated submersible imaging flow cytometer that generates images of particles in-flow taken from the aquatic environment.), the produced images are of good quality (Fig. 2), and the main modification applied is resizing the images to fit the model requirements; Transfer Learning : Transfer learning allows the weights of a pre-trained neural network to be retained and updated (only if specified) for specific tasks. It has been demonstrated that using pre-trained models leads to significant results, reducing both training time and the amount of data required compared to an untrained model (Maracani et al. 2023); Training and Validation : The modified CNN is trained on the annotated phytoplankton images. Techniques such as data augmentation (to increment the number of images), dropout, and batch normalization are employed to enhance model performance and prevent overfitting. The model's accuracy is validated using a separate dataset; Evaluation Metrics : Performance metrics, including accuracy, precision, recall, and F1-score, are used to evaluate the model. Confusion matrices and receiver operating characteristic (ROC) curves provide additional insights into the model's classification capabilities. Data Collection and Preprocessing : several public datasets are available (Ciranni et al. 2024), where each image is annotated according to its class. Each model is structured to require input images in a specific format, so depending on the chosen model, it is necessary to preprocess the images accordingly. With an Imaging Flow Cytobot (IFCB, an in-situ automated submersible imaging flow cytometer that generates images of particles in-flow taken from the aquatic environment.), the produced images are of good quality (Fig. 2), and the main modification applied is resizing the images to fit the model requirements; Transfer Learning : Transfer learning allows the weights of a pre-trained neural network to be retained and updated (only if specified) for specific tasks. It has been demonstrated that using pre-trained models leads to significant results, reducing both training time and the amount of data required compared to an untrained model (Maracani et al. 2023); Training and Validation : The modified CNN is trained on the annotated phytoplankton images. Techniques such as data augmentation (to increment the number of images), dropout, and batch normalization are employed to enhance model performance and prevent overfitting. The model's accuracy is validated using a separate dataset; Evaluation Metrics : Performance metrics, including accuracy, precision, recall, and F1-score, are used to evaluate the model. Confusion matrices and receiver operating characteristic (ROC) curves provide additional insights into the model's classification capabilities. Results Studies have demonstrated that pre-trained CNNs can achieve high accuracy in phytoplankton classification. In our case, models like ResNet and VGG have shown classification accuracies exceeding 80% on diverse phytoplankton datasets (Fig. 3, Kraft et al. 2022). These models effectively distinguish between species with subtle morphological differences, which are often challenging for human experts. Discussion The use of pre-trained CNNs in phytoplankton recognition offers several advantages: Efficiency : Automated classification significantly reduces the time and effort required for phytoplankton identification compared to manual methods. Scalability : CNNs can handle large volumes of image data, making them suitable for Long Term Ecological Research. Consistency : Machine learning models provide consistent and objective classifications, minimizing human error and variability. Efficiency : Automated classification significantly reduces the time and effort required for phytoplankton identification compared to manual methods. Scalability : CNNs can handle large volumes of image data, making them suitable for Long Term Ecological Research. Consistency : Machine learning models provide consistent and objective classifications, minimizing human error and variability. However, challenges remain. The automatic taxonomic identification level is still not as detailed as that of human expertise. The quality and diversity of training data are critical for model performance. Inadequate or biased datasets can lead to poor generalization. Additionally, the interpretability of CNNs is limited, making it difficult to understand the decision-making process fully. Conclusion Pretrained CNNs represent a powerful tool and a pipeline for phytoplankton species recognition, offering significant improvements in efficiency, scalability, and consistency over traditional methods. Continued advancements in machine learning and the availability of high-quality datasets will further enhance the capabilities of these models. Future research should focus on addressing current limitations, such as data quality and model interpretability, to fully realize the potential of CNNs in marine science. In this work, we will present the results as discussed to demonstrate possible workflows to fully realize the potential of CNNs in marine science and potentially contribute to the Standard Observations (SOs) addressing current limitations. We will also bring a workflow proposal to manage and perform actions related to harmonization, interoperability, quality control and sharing of the data obtained througth the CNNs recognitions following the directives proposed by Torstensson (2025).