UNVEILING RELATIONS: STATISTICAL INDEPENDENCE TEST AND LOW-DIMENSIONAL MANIFOLD LEARNING

Determining dependence between two continuous random vectors in high dimensions ($d>>1$) from noisy observations is a fundamental problem in scientific research. Inspired by the recent work on non-parametric tests of independence using Hadamard matrices (Walsh wavelets) in search of dependencies between random variables of a noisy sample, we present a frequency domain analysis-based framework and design a class of non-parametric tests of independence. This dissertation first expands the Walsh wavelets to other basis function choices. We found that the polynomial-based basis choices often perform better when the underlying relations can be approximatedby polynomial equation-based relations. When the Legendre polynomial basisfunctions are used, we apply the canonical component analysis (CCA) to find an optimal algebraic variety (zeros of a polynomial equation) on which data may be located. Rigorous mathematical analysis is performed to justify the optimization problem and its CCA-based solution. Numerical experiments were performed to understand the stability of this low-dimensional manifold learning algorithm. To address high-dimensional challenges, we explored hierarchical tree-based Haar wavelet basis and corresponding fast transforms, randomized algorithms, and k-nearest neighbors based sensor functions. Extensive numerical experiments were performed in 2+1 dimensionsto understand the mathematical and statistical properties of different approaches.Finally, out of curiosity, we study the application of modern deep neural network (DNN)approaches for independence tests. We discovered that the convolutional neural network (CNN) architecture can effectively distinguish between different distributions, however, in its current training settings and with the current feature inputs, it is unable to handle independence testing for arbitrary distributions in high dimensions.