Utilizing Large Language Models for Quality Enhancement of Text Annotation Data: A Chain of Thought Reasoning Approach

Large language models (LLMS) have shown great potential for automating and improving the quality of data annotation. This study proposes a quality improvement pipeline, which combines Chain of Thought (CoT) reasoning with supervised fine-tuning (Effective fine-tuning by Parameters (PEFT) strategies) to address the common problem of inconsistencies and inaccuracies in the annotations generated by LLMS. The pipeline consists of task-specific tag acquisition, guided inference generation for tag proof, targeted fine-tuning, and sequential evaluation and filtering to guarantee the quality of multidimensional annotations. Using a distributed multi-GPU setup, inference accuracy is improved when the method is evaluated on a modified cmre2018 dataset, and the sucess rate goes from a benchmark of 55.1 percent to more than 80 percent. This shows the efficiency of the pipeline in enhancing the LLM-based annotation process, and it also highlights the potential of llm to simplify and improve annotation quality for complex dataintensive tasks.