GeoFactory: an LLM performance enhancement framework for geoscience factual and inferential tasks

Large Language Models (LLMs) have demonstrated a remarkable capacity to efficiently learn, comprehend, and process extensive datasets and academic literature, leading to their widespread application across a variety of disciplines. However, the adoption of LLMs in geoscience remains constrained by several inherent limitations, including a lack of geoscience knowledge, difficulties in updating information, insufficient autonomy, high operational costs, and low efficiency. Overcoming these challenges will not only facilitate the broader application of LLMs in geoscience but also lay the foundation for the development of multimodal large language models (MLLMs) tailored to the field. LLM enhancement methods can augment their capabilities and broaden their applicability by tailoring them to specific tasks and specialized domains. This study introduces a framework named GeoFactory, designed to enhance the capabilities of LLMs in addressing geoscience tasks. Initially, we empirically evaluated the performance of four LLM enhancement methods and their associated 14 algorithms for addressing geoscience factual and inferential tasks. We analyzed the strengths and weaknesses of the LLM enhancement methods and specific algorithms to offer comprehensive guidance for addressing the inherent limitations associated with the adoption of LLMs in geoscience. Ultimately, we propose the GeoFactory framework to seamlessly ensemble these four LLM enhancement methods for improving the geoscience capabilities of LLMs. This framework enables Mistral-7b to outperform GPT-4 and 14 baseline algorithms in terms of the average performance on geoscience tasks. These findings show that, through appropriate LLM enhancement, a model with fewer parameters can achieve a performance comparable to that of GPT-4 in geoscience tasks. This study offers high-level guidance for the selection and application of LLM enhancement methods in geoscience.