Drug Pills Identification System using Google Gemini LLM: A Generative AI approach

Generative AI is emerging as a disruptive force in the healthcare industry, bringing novel solutions ranging from drug development and clinical decision support to personalized patient care. This study is focused on drug discovery using the Generative AI model. In this paper, a system is proposed for providing drug descriptions from drug pill images. The system is implemented by utilizing Large Language Models (LLMs) in combination with computer vision to detect and provide detailed information about drugs from pill images. In the proposed system, the identification process begins by taking the medicinal drug pills and their cover images. Then, the image is converted into binary values using a standard built-in function. In addition, the target language for providing audio descriptions about the drugs is also used. Then, the Google Gemini LLM model is customized by using binary values of the image, target language, and ontology-based prompt engineering. As a result, the LLM model provides drug descriptions in text. Then, the textual description of the drug is converted into the target language audio format by using the Google Text to Speech Converter. The system is experimented by using 807 medicinal drug images which are collected from web resources. The performance of the system is measured by using accuracy. The system achieved an accuracy of 95.04% which is a little higher when compared with the current state-of-the-art model.