PROMPT ENGINEERING TECHNIQUES IN LARGE LANGUAGE MODELS : A Comparative Study

Even though LLMs can solve different kinds of tasks without training, the quality of the answer depends on the formulation of the request. Prompt Engineering is the art of formulating a question right. There are many ways to do it, they differ in terms of their design, cost, use cases and computational requirements, thus it may be challenging for a beginner to choose the right one. In this paper, we compare and contrast seven of them: Zero-shot Prompting, Few-shot Prompting, Chain of Thought (CoT), Tree of Thoughts (ToT), Graph of Thoughts (GoT), Self-Refine, and Retrieval-Augmented Generation (RAG). All of them will be described in the same categories: idea, example, pros and cons, uses, computational requirements. No experiments have been conducted, however, the paper is based on the literature review. All the information comes from original sources. They can not be compared directly since different models and methods were used in different papers. The literature suggests that reasoning steps enumeration aids to solve multi-step problems if the model is big enough. The tree and graph approaches prove to be helpful if one thought is not enough; however, they are invoked much more often in that case. Self feedback improves the answer in some cases but not in others. The external knowledge in RAG comes in addition to formulating a prompt. The last part of the paper includes a brief comparison between the techniques, the costs associated with the use of each one and gaps in the research.