IEDA4Med: Iterative EDA Method for Medical Examination Data Based on Diagnostic Text

This paper proposes an iterative exploratory anal-ysis method for medical examination data based on the op-timization of medical text features, addressing the challenges of integrating text and numerical features in healthcare data. We designed a Text Vectorization and Dimensionality Reduction (TVDR) approach that combines Transformer models with traditional data mining techniques. The BERT model is used for text encoding, while PCA and multidimensional spatial mapping are employed to achieve a unified representation of high-dimensional medical data. Additionally, we introduce a Multi-dimensional Patient Data Classification (MPDC) method based on BILSTM-CRF technology, which analyzes extracted disease symptoms and related organ information from diagnostic texts to enhance the accuracy and interpretability of patient data classification. To further improve the efficiency of data exploration, we propose Feature Coordinate Fine-tuning (FCF) to refine the distribution of data in visualizations. Finally, we develop an Iterative Explore Data Analysis (IEDA) framework based on parallel coordinates, which validates the effectiveness and practicality of the framework through iterative analysis of multidimensional data in medical data analysis. The framework's advantages in enhancing medical data analysis efficiency and insights were verified through experiments conducted in collaboration with multiple medical institutions, utilizing real-case data covering various disease types.