Tree- and Forest-structured Discriminant Analysis for Mixed Data

We deal with the problem of classification of observations into two populations with binary and continuous variables, in this context we propose using the ratio of two decomposable tree- or forest-structured conditional Gaussian (CG) densities as classification rule. The simplicity of CG densities with these structures alleviates the problem of the need of large sample sizes, whereas the decomposability property ensures the existence of analytic expressions of the maximum likelihood estimators and the use of a modified version of the Kruskal's algorithm to find the minimum spanning tree for the structure estimation. Since the selection of features often improves the classification performance, a stepwise procedure based on the cross-entropy loss is also proposed. We compare the empirical performance of the proposed method with that of other methods, classical and modern, using test error rates for a real data set and for simulated samples of different sizes. The empirical performance of the proposed method in the real data was four among various methods. In the simulation, the method was able to recover the structure of the CG densities from which the samples were generated and produced the lowest error rate; it was also observed that the error rates for all the methods were substantially larger than the Bayes error for small sample sizes. The results suggest that the use of tree-- and forest--structured graphical models as representations of class-conditional densities within a discriminant analysis framework leads to a competitive method, sufficiently fast computationally, and worth considering for the classification of mixed data.