Unsupervised Dendrogram Text Index and Search Using Hierarchical Clustering

Text data retrieving depends initially on the data indexing, emphasizing the role of different indexing techniques used for query processing. In this context, traditional Information Retrieval (IR) techniques based on the inverted index share a number of challenges including limited scalability, accuracy, and semantic awareness. In this study, we provide an improved solution by introducing a new Machine Learning (ML) indexing technique based on hierarchical clustering. Our Unsupervised Dendrogram Index and Search (UDIS) approach introduces a dendrogram-like tree index structure that allows linking data items according to their clustering in the document corpus, providing data co-occurrence and semantic context in index building and querying. UDIS aims at improving the IR imbedding with a semantic tree structure, allowing improved quality and seamless integration of semanticaware search results. We have empirically evaluated the impact of UDIS index size and query execution, and their relationship with search quality. Experiments show improved results compared with unigram and bigram AND-and-OR inverted indices.