SCALABLE AND EFFICIENT ALGORITHMS FOR EXTRACTING DATA IN BIG DATA SYSTEMS

This paper is centered on the development of an information system for the extraction of emails addresses of staff/students in an organization with big data to assist in the dissemination of vital information in a very short time using two algorithms that are rule and machine learning based. The rulebased uses regular expression technique while the machine based is implemented with the decision tree classifier. The proposed tool can be used i n the banking sector to extract customers email addresses for posting transaction details and goodwill messages to improve customers’ relationship, in the educational sector the tool can be used to send students’ progress report, registration and payment r eceipt and lastly in the health sector to send medical reports, globalization and monitoring the hospital quality. The database was generated online since most organizations are very discreet with staff details. Tokenization of the generated data takes place immediately where the domains are determined. The constructive research methodology and object-oriented design technique was used to analyze, design and implement the tool. A customized software application was developed in python programming language f or its implementation ensuring the system evaluation met with system requirements and potential users’ expectations. The research extensively carried out testing using different data sizes to execute email addresses extraction and showed the limitations of the tool and potential for further work on the software package.