Analysis of Parallel Optimisation Strategies Based on MapReduce Models

The aim of this paper is to provide an in-depth analysis of parallel analysis strategies for MapReduce models, and to explore how to improve the overall performance by optimising task allocation and scheduling, improving data locality and increasing node utilisation. The research methodology includes an analysis and overview of existing MapReduce frameworks and proposes a series of improvement strategies. These strategies improve the utilisation of computing resources by adjusting the granularity of task division, optimising data slicing and distribution, and improving task scheduling algorithms. The results show that by reasonably optimising the parallel analysis strategy of the MapReduce model, its performance in large-scale dataset processing can be significantly improved, especially in resource-constrained distributed environments. Ultimately, this paper concludes that although the MapReduce model has been used in more mature applications, there is still much room for optimising its parallel strategy when facing larger scale and complex data processing tasks. In the future, further research should be devoted to finer-grained task scheduling, dynamic resource allocation, and more efficient fault-tolerance mechanisms to continuously improve the parallel processing capability of the MapReduce model.

Authors

Publication Details

Journal
International Journal of Computer Science and Information Technology
Published
2024-12-21
DOI
https://doi.org/10.62051/ijcsit.v4n3.39
Primary Topic
Graph Theory and Algorithms
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Analysis of Parallel Optimisation Strategies Based on MapReduce Models

Shiyue Wu
International Journal of Computer Science and Information Technology
Graph Theory and Algorithms
article

Analysis of Parallel Optimisation Strategies Based on MapReduce Models

Shiyue Wu
article en

Abstract

The aim of this paper is to provide an in-depth analysis of parallel analysis strategies for MapReduce models, and to explore how to improve the overall performance by optimising task allocation and scheduling, improving data locality and increasing node utilisation. The research methodology includes an analysis and overview of existing MapReduce frameworks and proposes a series of improvement strategies. These strategies improve the utilisation of computing resources by adjusting the granularity of task division, optimising data slicing and distribution, and improving task scheduling algorithms. The results show that by reasonably optimising the parallel analysis strategy of the MapReduce model, its performance in large-scale dataset processing can be significantly improved, especially in resource-constrained distributed environments. Ultimately, this paper concludes that although the MapReduce model has been used in more mature applications, there is still much room for optimising its parallel strategy when facing larger scale and complex data processing tasks. In the future, further research should be devoted to finer-grained task scheduling, dynamic resource allocation, and more efficient fault-tolerance mechanisms to continuously improve the parallel processing capability of the MapReduce model.

International Journal of Computer Science and Information TechnologyVol. 4(3)
Openalex Percentile: Top 72%
Graph Theory and Algorithms
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.