Variational bayes for high-dimensional linear models

The emergence of large-scale high-dimensional datasets has presented unparalleled scientific and commercial opportunities in areas such as drug discovery, personalized medicine, financial modeling, and climate science. However, these datasets, characterized by their vast number of features, pose unique challenges to traditional statistical methodologies, including: parameter estimation, uncertainty quantification, and feature selection. These challenges have spurred a flurry of research and methodological development by the statistical community; however, it is only in recent years that progress has been made in delivering methods addressing all three of these needs while ensuring computational scalability. This thesis builds on this line of research, introducing scalable methodologies tailored to a variety of linear models commonly used in practice. Specifically, we turn to variational inference (VI), an inferential framework that relies on optimization rather than sampling to perform posterior inference. Throughout, we demonstrate how VI can be used to construct scalable methods for logistic regression and Gaussian process classification, sparse survival analysis with censored data, group sparse linear models, and high-dimensional additive models. Through extensive simulations and real-world applications, we showcase the effectiveness and competitiveness of this approach, highlighting the potential to provide accurate and interpretable inference in the analysis of high-dimensional data, paving the way for their application in various scientific and industrial contexts.