Dynamic Record Linking Using Multi-Agent Machine Learning: An Architecture for Noisy and Variable Datasets

The aim of this article is to present a dynamic record linkage solution using high level machine learning to solve the problems associated with noisy, inconsistent and dynamic data sets. Classic deterministic and probabilistic models fail to work in such environments because they are static and based on assumptions. Incorporating supervised learning algorithm, active learning, and ensemble techniques such as random forests and boosting allows the proposed structure to change in response to different data types and thus become more accurate and scaleable. Some of its highlights are feature selection to ensure match accuracy, clustering to support noisy inputs, and active learning to minimize reliance on large labeled datasets. Simulations with real data, such as government or healthcare data, show that compared with the traditional approach, linkage is significantly more accurate and efficient. The flexible model led to 15% higher F1-scores on noisy datasets and was scalable across large data sets. This work demonstrates how adaptive machine learning can revolutionize the modern record linkage tasks and provides a powerful and effective solution to ever-changing data conditions.