Improving LLM-Driven Data Annotation Through Structured Instruction Engineering

This paper introduces a method for enhancing the performance of an LLM-based AI agent designed for data annotation. We present a technique for structuring the agent's instructions and evaluate its efficacy across diverse models from multiple providers. The proposed method supports the ongoing paradigm shift in data an-notation-from human-human to human-machine interaction between a manager and an annota-tor-and underscores the significance of LLMs in facilitating this transformation.