An Embedding-Based Method for Processing Medical Audio into Structured Reports

Healthcare professionals spend a significant portion of their time on electronic health records documentation, reducing patient interaction time and increasing operational costs. Our solution implements a two-stage pipeline combining voice-to-text transcription using WhisperV3 and text-to-structure conversion through an embedding-based approach. We address critical challenges in medical documentation automation, including specialized vocabulary processing and the prevention of hallucinations in generated content. The system was developed with continuous input from medical domain experts, resulting in a comprehensive field structure covering 78 essential information categories organized into six distinct sections. Our application processes clinical conversations locally, prioritizing data privacy and security while transforming unstructured medical notes into structured clinical documentation. The resulting system enables healthcare professionals to focus more time on patient care while simultaneously improving the quality and accessibility of medical records for better clinical decision-making.