Exploring Methods to improve Context-Aware text Extraction and Categorization of a Large Language Model

Debriefing training exercises and high-stakes operations often involves analyzing radiocommunicationto better understand interactions and decision-making. This research investigates methods forextracting and categorizing sender-recipient information from transcribed air traffic control (ATC) communication,aiming to support more effective analysis of such data. A quantitative experimental approachwas used to compare the accuracy of zero-shot and few-shot prompting techniques, as well as their finetunedcounterparts using Low-Rank Adaptation (LoRA).Zero-shot prompting provided a strong baseline for evaluating the effectiveness of few-shot andfine-tuned approaches. The few-shot method generated unexpectedly strong results, and its combinationwith a LoRA fine-tuned model achieved the highest accuracy. These improvements highlight how simpleadjustments can enhance a model’s ability to understand domain-specific communication.The findings suggest that these methods could benefit other high-stakes domains, such as militaryoperations, emergency response, and other uniformed services, by supporting faster and more accurateinterpretation of critical communications during both training and real-world events. The results aresupported by Fisher’s exact test, performed to determine statistical significance.