Cross language transformation of free text into structured lobectomy surgical records from a multi center study

In a recent study, the effectiveness of GPT-4 Omni in transforming lobectomy surgical records into structured data across multiple languages was explored. The aim was to improve both efficiency and accuracy in documenting thoracic surgical oncology procedures. Involving 466 records from seven specialized hospitals, the process started with OCR and text normalization. A manual restructuring by thoracic oncologists set the benchmark for fine-tuning Generative Pre-trained Transformer 4 Omni (GPT-4o). Experts reviewed the AI's output, assessing it on accuracy, precision, recall, and F1 scores. GPT-4o demonstrated high performance across both Chinese and English records, achieving an accuracy of 0.966, precision of 0.981, recall of 0.982, and an F1-score of 0.982 in both language settings. Results showed that GPT-4o was highly effective in both Chinese and English, significantly speeding up documentation compared to traditional methods. While it performed well across languages and reduced review times, common error types included terminology misinterpretations (2.82%), procedural sequence errors (1.41%), and omissions of key details (0.47%). While it performed well across languages and reduced review times, these limitations highlight areas for further refinement, particularly in enhancing contextual understanding and mitigating minor errors. Nonetheless, GPT-4o shows great potential in standardizing surgical records, streamlining workflows, and boosting care and research in thoracic oncology.