Construction and Alignment Features of English Corpus based on Particle Swarm Optimization Algorithm

With the continuous advancement of natural language processing technology, building a high-quality corpus has become a hot topic in current research. However, the traditional corpus construction method requires a lot of manual intervention, which is time-consuming and laborious, and cannot meet the requirements of large amounts of data. To overcome these limitations, this paper applies an automated method based on the particle swarm optimization (PSO) algorithm. First, the scale and text type of the corpus are determined, and then an adaptive function suitable for text similarity evaluation is designed and evaluated. The method mainly consists of three parts: initialization of the particle swarm, updating of particle positions, and selection of the best solution. Through experimental evaluation, we found that in a large sample scenario, the perplexity of the PSO algorithm can be stabilized at around 2, showing good robustness. In addition, the evaluation using the BLEU (bilingual evaluation) score also reached a high score of 0.99, which indicates the high quality of the alignment. The research results will further verify the feasibility of the PSO algorithm in the automatic construction and alignment of the English corpus, and provide a new and efficient method for corpus construction in natural language processing. Through automated methods, manual participation can be significantly reduced, construction efficiency can be improved, and alignment quality can be maintained or improved.

Authors

Publication Details

Published
2024-09-20
DOI
https://doi.org/10.1109/icdscnc62492.2024.10939378
Primary Topic
Educational Technology and Pedagogy
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Construction and Alignment Features of English Corpus based on Particle Swarm Optimization Algorithm

Xin Li, Wei Shang
Educational Technology and Pedagogy
article

Construction and Alignment Features of English Corpus based on Particle Swarm Optimization Algorithm

Xin Li, Wei Shang
article en

Abstract

With the continuous advancement of natural language processing technology, building a high-quality corpus has become a hot topic in current research. However, the traditional corpus construction method requires a lot of manual intervention, which is time-consuming and laborious, and cannot meet the requirements of large amounts of data. To overcome these limitations, this paper applies an automated method based on the particle swarm optimization (PSO) algorithm. First, the scale and text type of the corpus are determined, and then an adaptive function suitable for text similarity evaluation is designed and evaluated. The method mainly consists of three parts: initialization of the particle swarm, updating of particle positions, and selection of the best solution. Through experimental evaluation, we found that in a large sample scenario, the perplexity of the PSO algorithm can be stabilized at around 2, showing good robustness. In addition, the evaluation using the BLEU (bilingual evaluation) score also reached a high score of 0.99, which indicates the high quality of the alignment. The research results will further verify the feasibility of the PSO algorithm in the automatic construction and alignment of the English corpus, and provide a new and efficient method for corpus construction in natural language processing. Through automated methods, manual participation can be significantly reduced, construction efficiency can be improved, and alignment quality can be maintained or improved.

Openalex Percentile: Top 74%
Educational Technology and Pedagogy
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Construction and Alignment Features of English Corpus based on Particle Swarm Optimization Algorithm — Xin Li, Wei Shang · (2024) | TGRS Research Map | TGRS