Flexible And Faithful Data Insights Generation

With the ever-growing volume of data, corporate users want to understand their data more efficiently. In addition to data visualization, they like text-based insights (e.g., a summary of trends, seasonality, and anomalies in the data) so they can digest them easily and/or share them with other stakeholders. In this paper, we introduce an efficient framework that generates faithful and flexible insights that resemble human writing. It is a hybrid approach that takes advantage of both template-based summaries and the latest Generative AI technology. With a fine-tuned compact Large Language Model (LLM) and a Gatekeeper, we can achieve a lower error rate than state-of-the-art LLMs, while using only a fraction of resources. We have released a service based on the proposed approach and received very positive feedback.