LLMOverTab: Tabular data augmentation with language model-driven oversampling

In recent years, Large Language Model (LLM) have seen significant advancements, attracting attention for their applications in various fields. These models have shown promising results in handling tabular data, especially in cases with limited datasets, by leveraging pre-trained knowledge. However, their effectiveness in addressing imbalanced data in tabular formats is less explored. To bridge this gap, our study introduces LLMOverTab, a novel approach using LLMs for oversampling in imbalanced tabular data. We conducted comprehensive experiments on diverse tabular datasets to assess the effectiveness of LLMOverTab, demonstrating its potential in improving the handling of imbalanced data. The study also explores application of LLMOverTab in zero-shot and few-shot learning contexts, providing insights into its adaptability. Additionally, we analyze the oversampled data, offering reflections on the quality of generated samples. Our research not only showcases the utility of LLMOverTab in managing imbalanced tabular data, but also opens new avenues for the application of language models in various tasks of tabular data. This study adds to the increasing interest in applying LLMs to various task domains. It provides new perspectives for the innovative use of LLMs in structured tabular data fields, highlighting their potential in a range of applications. • Introduces LLMOverTab for oversampling in imbalanced tabular data. • Surpasses traditional methods like SMOTE and other LLM approaches. • Uses prompt engineering to generate meaningful synthetic instances. • Finds LLMOverTab excels especially with LLM prediction models. • Suggests exploring different LLM architectures and prompt techniques.