Improved Hierarchical Agglomerative Clustering Based Multivariate Gaussian Outlier Detection for Education Data

Data Mining is the procedure of analyzing dataset in any field to have meaningful patterns or relationship from the variables. Outliers refer to the data that has special insight and valuable information which deviates from the rest that can be in the form Local or Global. Noise and Outliers are used interchangeably based. Precisely Noise data refers to the erroneous data and the Outliers implies deviated data. This work proposes Cluster based Outlier Detection method on Education Data to filter the Outliers that implies the students or learners who are deviated or considered extraordinary from the rest of the students. The existing Outlier detection method has computational complexity and more processing time. In order to handle these issues, this work initially applies Edited Nearest Neighbor method to filter the global Outliers, then implements Cluster based Outlier Detection method to filter local Outliers. For clustering, the existing Hierarchical Agglomerative method is improved with Fuzzy Partition values continued with Flat Clustering to split the Dendrogram, which is the common output form of Hierarchical Clustering. Furthermore, the work calculates Multivariate Gaussian Outlier Score in order to filter the local Outliers in each clusters formed by Flat clusters. The proposed method is implemented in Rapidminer tool, assessed with the appropriate clustering evaluation measures. The proposed method proves its efficiency by having less Cluster Density, Sum of Squares and Processing Time while compared with the existing benchmark methods.