Development of an Artificial Intelligence-Aided Software for Annotating Image Datasets

Context: Deep learning is a highly successful class of methods in the field of artificial intelligence (AI) that has a variety of applications. To perform well, deep learning models require a large amount of high-quality annotated data. Problem: Data annotation is a time-consuming and laborious task that requires a significant amount of human labor, which makes it expensive. Solution: This work aims to reduce the time required to annotate image datasets by building an easy-to-use software tool that has semi-automated annotation powered by an artificial intelligence model. IS Theory: The work is based on the Socio-technical theory because we developed and evaluated a tool to be acceptable and useful for the users. Method: We developed a web-based tool and employed HQ-SAM, a deep neural network for image segmentation based on Vision Transformers, to generate polygon annotations based on the user’s prompts. Although HQ-SAM has a good zero-shot generalizability, we fine-tuned it on the Bean Leaf Dataset to evaluate how well the network adapts to specific tasks. Summary of results: We observed an increase in accuracy of the fine-tuned model compared to the pre-trained one. We tested our tool with 20 participants, all of whom are from the computer vision and graphics fields. We asked them to annotate the same two images both manually and AI-aided, and recorded the annotation times. Lastly, we asked the participants to fill out a usability form about their user experience. In our evaluation, we registered a median speedup of 1.5× regarding the AI-aided annotation compared to manual annotation and overly positive answers regarding our tool’s ease of use and usefulness. Contribution: We expect the proposed system to significantly reduce the human effort required for image dataset annotation, leading to faster annotation times. This offers a valuable contribution to the computer vision and AI communities, speeding up the dataset creation process.