Responsible and Sustainable AI: Considering Energy Consumption in Automated Text Classification Evaluation Tasks

Text classification is one of the typical and fundamental natural language processing tasks. With the advent of large language models (LLMs), text classification has evolved much further. Based on the growing sizes of LLMs and the increased demands for hardware, and especially energy, questions about sustainability and environmental impacts and responsibility also arise. To assess text classification approaches, researchers usually only use common performance metrics like precision, recall, and f<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf>-score. Green AI, i.e., improving environmental aspects while maintaining performance, is regularly disregarded and not a standard in the evaluation of automated text classification approaches. Yet, minor performance improvements might not justify, e.g., much higher energy consumption. In this paper, we aim to raise awareness for this issue and the corresponding trade-off discussions and decisions. Therefore, we present novel sustainability metrics and provide guidelines for text classification approaches that are suitable for Green AI. In a text classification use case, we showcase the applicability of our proposed metrics and discuss corresponding trade-off decisions.