Automated extraction of key parameters and detection of inconsistencies in clinical documentation using large language models

This study investigates unstructured text data on clinical trials. The task addressed relates to the fact that analyzing such data involves a laborious and error-prone process, hard-to-tackle even for specialists. In turn, this leads to an increase in the duration of studies and delays in the release of new drugs to the market. This work reports an approach to constructing a dataset on clinical trials, as well as subsequent extraction of key information using state-of-the-art large language models. A study was conducted on extracting such indicators as the eligible gender of participants, a research phase, as well as the study's therapeutic area. A total of 11,703 experiments were performed, most of which achieved high results. In particular, the average values when using the GPT-4o-mini model were as follows: F1-measure – 0.92; accuracy – 0.98; recall – 0.99; precision – 0.87. Extraction of information from clinical documentation in Ukrainian demonstrated similar results compared to English-language counterparts. In some cases, a significant number of false positives were observed, and the indicators were significantly lower (the lowest recorded values: F1-measure – 0.52; accuracy – 0.82; recall – 0.78; precision – 0.35). For such cases, the reasons were analyzed, and the corresponding conclusions and recommendations were formulated. In addition, the results of the experiments helped identify a number of discrepancies and errors in official registries, which is a vivid example of practical application. Other examples of using the result are the possibility of scaling the technology to additional data types, as well as supporting digital transformation in the medical field. Such results are prerequisites for automating the clinical trial process and accelerating the release of new drugs to the market.