STATISTICAL LEARNING USING BINARY EXPANSION

As model complexity and data volumes increase, there arises a pressing need for methods capable of capturing and interpreting dependence information effectively. In this dissertation, we focus on improving the effectiveness and interpretability of statistical learning with a powerful approach known as binary expansion framework. In the first project, we propose a nonparametric resolution-wise regression procedure to estimate distribution by considering marginal binary expansions of the response and explanatory variables, and modeling the relationships with penalized logistic regressions on the frequency domain. In order to speed up the computation of binary expansion statistics, in the second project, we develop a fast testing algorithm, SorBET (i.e., Shifting and XOR operator based BET). The SorBET processes data and carries out statistics through bitwise operations, which substantially reduces the computational expense. A recent framework of binary expansion linear effect (BELIEF) provides a "divide and conquer" approach to decompose any complex form of dependency into small linear regressions over data bits. Although BELIEF can be used to approximate any relationship, it faces an important challenge of high dimensionality. To overcome this obstacle, we propose a novel definition of smoothness for binary interactions and create a regularization of BELIEF under smoothness interpretations. We show that there is a one-one correspondence between each marginal binary interaction and the smoothness we defined. Moreover, we have shown that in higher dimensions, the smoothness can be expressed as a product of that for marginal binary interactions. In order to model the smooth form of dependency, we propose a smooth LASSO with larger penalty on less smooth terms. The numerical studies demonstrate advantages of smooth LASSO in clear interpretability and effectiveness for nonlinear and high dimensional data.