Specification curve analysis of the TEDDY study reveals large variation in microbiome-based T1D predictive performance

The microbiome may play a role in predicting future Type 1 Diabetes (T1D) risk. Associations between the microbiome and T1D onset are well documented but observational microbiome studies are difficult to interpret and reproduce due to differences in study designs. To evaluate if the microbiome is a robust predictor of T1D or T1D associated autoantibodies, we performed a "specification curve analysis" from a longitudinal cohort of 783 individuals at high risk of T1D, that attempts to parameterize and systematically test all possible study design specifications. We predicted T1D and autoantibodies using 11,189 different specifications. We show a large amount of variation in the predictive ability of the microbiome across specifications. 72.5% of models that only use microbial features had an area under the curve (AUC) of 0.5 and the "best" model had an AUC of 0.78. Results for every specification can also be found in an interactive app at: http://apps.chiragjpgroup.org/teddy .