ENTITY RESOLUTION: META-BLOCKING AND KLSH

We perform an exploration to solve an entity resolution problem on tabular data based on blocking rules and a K-Means clustering algorithm. We create this imaginary dataset that contains piano manufacturer information with intentionally introduced variations and duplicates, designed for testing entity resolution algorithms, specifically meta-blocking and K-LSH (Locality Sensitive Hashing) methods. The piano model names are purposefully omitted in the dataset to test record linkage within brands for different models without the model name feature. The proposed approach has three sequential stages. Firstly, the generation of a hierarchical graph for records in blocks, followed by the generation of a record relationship graph with components; we further enhance the pipeline by tuning each component’s feature weights via Bayesian optimization prior to k-means clustering enabling separation of piano models despite omitting their explicit name labels; finally, we cluster the records for each component using a K-Means algorithm. We conclude the results are encouraging and we identify multiple further research opportunities to improve our results.