EP01.42: Saving expert time with AI assisted labelling workflow for view classification of ultrasound images

Medical software development often requires data annotation by clinical experts. This is a significant burden due to its time-consuming nature, so finding experts for these tasks is difficult. An alternative approach is an iterative workflow where preliminary labels generated by AI simplify this process, necessitating less human effort. The framework of the idea is the SUOG (https://www.suog.org/) application, more specifically the search of ultrasound images in a large reference database, where view labelling is necessary. Our workflow was simulated using 9000 fetal images and 150 possible view labels. The workflow starts with manually annotating small portion of data for initial reference. The next step is feature extraction (image characteristic calculation) on the whole dataset using AI model pretrained on natural images (DINO), followed by a traditional classification algorithm to identify similarities in the data. Images with similar features are assigned to the same label as in the expert-labelled initial subset. The prelabels are reviewed by an expert then they can be added to the reference subset. This cycle is repeated with refitting the classification algorithm adding the reviewed data until the whole dataset is ready. Prelabelling gets more precise with each iteration. Partitioning our dataset into 10 random batches the workflow was simulated. During evaluation we assumed that accepting a prelabel is significantly faster and correcting a false prelabel is slightly slower than manual annotation. Real usage of the workflow also showed promising results with the evaluated prelabels being accurate more than 50% of the time (using expert-selected views only). Both simulation and real-life results showed that using our AI-based prelabelling reduces the time needed for annotation by at least 25%. This leads to notable cost reduction of labelling activities executed during medical AI application development.