Risk Minimization-Based Weighted Naive Bayesian Classifier

Naive Bayesian classifier(NBC),which is famous for its sound theoretical basis and simple model structure,is a classical classification algorithm which has been deemed as one of the top 10 algorithms in the fields of data mining and machine lear-ning.However,the dependence assumption of NBC limits its prediction performance when attribute dependence exists.Weighted NBC(WNBC) is an improved version of NBC,which has good generalization performance and low training complexity.This paper proposes a risk minimization-based WNBC(RM-WNBC) by considering both empirical risk and structural risk,in which the empirical risk measures the classification performance of RM-WNBC and structural risk depicts the dependence expression capability of RM-WNBC.Unlike existing improvements to NBC,RM-WNBC alleviates the dependence assumption and further enhances the generalization capability of NBC by considering with the internal characteristics of NBC rather than its external characteristics.The empirical risk is represented by the estimation quality of posterior probabilities,while the structural risk is represented by the mean squared error of joint probabilities.The minimization of empirical risk and structural risk guarantees that RM-WNBC can achieve both good classification performance and appropriate dependence representation.To obtain the optimal weights of marginal probabilities,an efficient and convergent updating strategy is designed by minimizing the empirical and structural risks.A series of persuasive experiments is conducted to validate the feasibility,rationality and effectiveness of RM-WNBC on 31 benchmark data sets.The experimental results show that the optimization process of RM-WNBC weights is convergent and RM-WNBC not only well deals with the attribute dependence but also obtains better training and testing accuracies than the classical NBC,three typical Bayesian networks,four WNBCs and feature selection-based NBC.