Statistical inference for large-scale multi-source heterogeneous data

In the era of digital information, the data with which people face may not only be large-scale but also heterogeneous. In this paper, we study statistical inference for the overall population mean function of large-scale multi-source heterogeneous datasets. By borrowing hierarchical sampling methods and divide-and-conquer techniques, we propose a weighted local linear estimator for the overall population mean function of multi-source heterogeneous data. Through studying the pointwise convergence properties and extreme value distribution properties of the estimator, we construct asymptotically accurate simultaneous confidence bands and pointwise confidence intervals for large-scale multi-source heterogeneous data. Our proposed methods are applicable not only to scenarios of heterogeneous data but also to scenarios of homogeneous data using divide-and-conquer methods. Numerical simulation studies show that the proposed methods perform well in analyzing both large-scale multi-source heterogeneous data and homogeneous data. As an illustration, we apply the proposed methods to hypothesis testing problems on Beijing multi-site air-quality data and U.S. census data.