Probabilistic Modeling and Pattern Discovery-Based Sindhi Information Retrieval System

Natural language processing is the technology used to interact with computers using human languages. An overlapping technology is Information Retrieval (IR), in which a user searches for the demanded or required documents from among a number of documents that are already stored. The required document is retrieved according to the relevance of the query of the user, and the results are presented in descending order. Many of the languages have their own IR systems, whereas a dedicated IR system for Sindhi still needs attention. Various approaches to effective information retrieval have been proposed. As Sindhi is an old language with a rich history and literature, it needs IR. For the development of Sindhi IR, a document database is required so that the documents can be retrieved accordingly. Many Sindhi documents were identified and collected from various sources, such as books, journal, magazines, and newspapers. These documents were identified as having potential for use in indexing and other forms of processing. Probabilistic modeling and pattern discovery were used to find patterns and for effective retrieval and relevancy. The results for Sindhi Information Retrieval systems are promising and presented more than 90% relevancy. The time elapsed was recorded as ranging from 0.2 to 4.8 s for a single word and 4.6 s with a Sindhi sentence, with the same starting time of 0.2 s. The IR system for Sindhi can be fine-tuned and utilized for other languages with the same characteristics, which adopt Arabic script.