Hybrid Models for Big Data Processing Combining Hadoop, Spark, and Machine Learning Frameworks

People utilize MapReduce implementations to process and Analyze massive volumes of data efficiently. Search engines like Google and other data-intensive applications rely on these frameworks to aggregate and manage vast datasets from heterogeneous sources. Such data is essential for enabling data-driven decision-making, predictive modelling, and the delivery of intelligent services. With the exponential growth of data, scalability, flexibility, and ease of use have become critical for high-performance analytics. MapReduce remains one of the most widely adopted frameworks for large-scale data processing due to its scalability, fault tolerance, and simplicity of programming. However, despite these advantages, MapReduce demonstrates certain performance limitations, particularly in iterative and computationally intensive analytical workloads often required by modern Machine Learning algorithms. Consequently, integrating Machine Learning techniques with MapReduce and Hadoop ecosystems has emerged as a powerful approach for enhancing data processing efficiency, model training, and intelligent pattern discovery. This study explores the intersection of Big Data analytics and Machine Learning within MapReduce-based environments. It investigates performance optimization strategies, hybrid frameworks such as Spark MLlib and Mahout, and discusses how distributed ML algorithms can overcome the limitations of traditional MapReduce processing. The paper aims to provide a comprehensive understanding of these advancements, guide the planning and administration of Big Data and ML-driven projects, and identify promising avenues for future research in scalable intelligent data analytics

Authors

Institutions

Publication Details

Journal
Journal of Emerging Technologies and Innovative Research
Published
2025-01-01
DOI
https://doi.org/10.56975/jetir.v12i10.570680
Primary Topic
Cloud Computing and Resource Management
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Hybrid Models for Big Data Processing Combining Hadoop, Spark, and Machine Learning Frameworks

S.Nagaparameshwara Chary
Journal of Emerging Technologies and Innovative Research
Cloud Computing and Resource Management
article

Hybrid Models for Big Data Processing Combining Hadoop, Spark, and Machine Learning Frameworks

S.Nagaparameshwara Chary
article en

Abstract

People utilize MapReduce implementations to process and Analyze massive volumes of data efficiently. Search engines like Google and other data-intensive applications rely on these frameworks to aggregate and manage vast datasets from heterogeneous sources. Such data is essential for enabling data-driven decision-making, predictive modelling, and the delivery of intelligent services. With the exponential growth of data, scalability, flexibility, and ease of use have become critical for high-performance analytics. MapReduce remains one of the most widely adopted frameworks for large-scale data processing due to its scalability, fault tolerance, and simplicity of programming. However, despite these advantages, MapReduce demonstrates certain performance limitations, particularly in iterative and computationally intensive analytical workloads often required by modern Machine Learning algorithms. Consequently, integrating Machine Learning techniques with MapReduce and Hadoop ecosystems has emerged as a powerful approach for enhancing data processing efficiency, model training, and intelligent pattern discovery. This study explores the intersection of Big Data analytics and Machine Learning within MapReduce-based environments. It investigates performance optimization strategies, hybrid frameworks such as Spark MLlib and Mahout, and discusses how distributed ML algorithms can overcome the limitations of traditional MapReduce processing. The paper aims to provide a comprehensive understanding of these advancements, guide the planning and administration of Big Data and ML-driven projects, and identify promising avenues for future research in scalable intelligent data analytics

Journal of Emerging Technologies and Innovative ResearchVol. 12(10)
National Institute of Technology Warangal (IN)
Openalex Percentile: Top 55%
Cloud Computing and Resource Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Hybrid Models for Big Data Processing Combining Hadoop, Spark, and Machine Learning Frameworks — S.Nagaparameshwara Chary · Journal of Emerging Technologies and Innovative Research (2025) | TGRS Research Map | TGRS