PatientLens: AI-Enabled Interactive Avatars for Patient Report Summarization and Visualization

In clinical environments, doctors often must review large amounts of patient reports prior to consults and check-ups --- a task that is time-consuming, cognitively taxing, and prone to errors. We investigate how to improve this workflow via the use of AI-driven virtual avatars that enable clinicians to query, and summarize information from text-based patient reports. While the use of AI (specifically LLMs) brings significant potential benefits for clinical settings, it also presents critical challenges such as hallucination. Based on robust discussions and iterative prototyping with clinicians, we develop a human-in-the-loop approach that supports interactively creating and refining virtual avatars that visually present patient information while efficiently supporting LLM oversight and transparency. Evaluations help validate that the developed tool, nicknamed PatientLens, supports clinical review and summarization workflows. We also discuss how lessons learned during this project can enhance healthcare communication, optimize clinical workflows, and support improved health equity and outcomes.