Why Big Data Projects Fail? A Systematic Literature Review

Big data projects have become increasingly important in today's data-driven world, significantly influencing sectors such as healthcare, finance, and retail. However, these projects often face high failure rates, with estimates suggesting that between 80% and 87% fail to produce sustainable solutions. This systematic literature review aims to investigate the factors contributing to the failure of big data projects. We conducted a comprehensive analysis of 26 academic studies and 3 industry reports, covering literature from 2010 to 2024. Our review reveals five primary themes contributing to big data project failures: technical challenges, organisational factors, ethical and legal considerations, financial constraints, and methodological challenges. Technical issues, particularly in data quality and integration, emerged as the most prevalent, closely followed by organisational factors such as skills shortages and cultural resistance. Ethical considerations and financial constraints also play significant roles, while methodological challenges, though less frequently mentioned, highlight important areas for future research. The review underscores that big data project failures rarely stem from a single factor but rather from the interplay of multiple challenges. This insight calls for a holistic approach to big data initiatives, integrating technical solutions with organisational change management, ethical considerations, and strategic alignment. Our findings provide insights for researchers, practitioners, and policymakers, emphasising the need for interdisciplinary approaches and industry-specific frameworks to enhance the success rate of big data projects.