Can LLaMA 3 Understand Monetary Policy?

This study provides an in-depth examination of LLaMA 3's performance on a domain-specific task, specifically classifying monetary policy texts. The aim is to categorize these texts as hawkish, dovish, or neutral by using a prompt to assess LLaMA 3'<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$s$</tex> understanding of finance-related knowledge anticipated to be acquired during its pretraining. Experimental results demonstrate that LLaMA 3 has indeed developed the necessary commonsense knowledge to interpret monetary policy, as it surpasses both major and random baselines. To further understand its performance, we also analyze the consistency of its impressive results across different model inputs, including the number of tokens, nouns, and verbs.