P072 Developing Generative AI Products for Sleep Health Support: A Risk-Based Approach

Abstract Introduction Generative AI tools can assist individuals with sleep problems by providing information about sleep disorders, sleep health, and product support. However, it is crucial to mitigate the risks of misinformation and ensure that tools do not unintentionally provide medical advice. Methods A set of topics and patient/customer queries were identified to define the use case. Despite not meeting the definition of a medical device in Australia, a multidisciplinary team employed risk management principles to meet user needs while complying with regulatory, legal, and clinical requirements. Key considerations included selecting the LLM and grounding content to create a prototype. The prototype was tested by a multidisciplinary team to evaluate performance and introduce additional requirements via prompt engineering, disclaimers, and topic exclusions. Upon meeting requirements, the product launched with 100% human oversight of customer responses, planning to reduce oversight as the product consistently met all requirements. Results The digital assistant has successfully answered over 1000 user inquiries, meeting all requirements. Data sources are regularly updated to align with current information and the risk analysis is continuously updated based on performance, and. Discussion The risk-based process used for developing the digital assistant has resulted in a generative AI tool to support people with sleep problems. Collaboration with a multidisciplinary team is crucial to ensure the responsible use of generative AI tools that are fit for the purpose to support people with sleep problems as they look for reliable information about sleep health and sleep disorders.