Hybrid Models for Big Data Processing Combining Hadoop, Spark, and Machine Learning Frameworks
People utilize MapReduce implementations to process and Analyze massive volumes of data efficiently. Search engines like Google and other data-intensive applications rely on these frameworks to aggregate and manage vast datasets from heterogeneous sources. Such data is essential for enabling data-driven decision-making, predictive modelling, and the delivery of intelligent services. With the exponential growth of data, scalability, flexibility, and ease of use have become critical for high-performance analytics. MapReduce remains one of the most widely adopted frameworks for large-scale data processing due to its scalability, fault tolerance, and simplicity of programming. However, despite these advantages, MapReduce demonstrates certain performance limitations, particularly in iterative and computationally intensive analytical workloads often required by modern Machine Learning algorithms. Consequently, integrating Machine Learning techniques with MapReduce and Hadoop ecosystems has emerged as a powerful approach for enhancing data processing efficiency, model training, and intelligent pattern discovery. This study explores the intersection of Big Data analytics and Machine Learning within MapReduce-based environments. It investigates performance optimization strategies, hybrid frameworks such as Spark MLlib and Mahout, and discusses how distributed ML algorithms can overcome the limitations of traditional MapReduce processing. The paper aims to provide a comprehensive understanding of these advancements, guide the planning and administration of Big Data and ML-driven projects, and identify promising avenues for future research in scalable intelligent data analytics
Authors
- S.Nagaparameshwara Chary
Institutions
- National Institute of Technology Warangal (IN)
Publication Details
- Journal
- Journal of Emerging Technologies and Innovative Research
- Published
- 2025-01-01
- DOI
- https://doi.org/10.56975/jetir.v12i10.570680
- Primary Topic
- Cloud Computing and Resource Management
- Type
- article
- Field-Weighted Citation Impact
- 0.00