Enhancing the Interpretability of Radial Velocity Analysis Using Leave-one-out Cross Validation

Abstract Radial velocity (RV) observations are reaching increasingly high precision, enabling the targeting of low-amplitude signals. Assessing results obtained at the limits of our capabilities requires understanding the relationship between data, models and inference. Leave-one-out cross validation (LOO-CV) has become an increasingly popular tool for interpretable model criticism in astronomical data analysis. In this work, we demonstrate how LOO-CV can provide insight into the RV analysis of exoplanetary systems through three case studies. We first investigate the anomalously high mass of HD 119130 b inferred from CARMENES RVs, later revised significantly downward after follow-up observations with other instruments. We show that this high mass was strongly driven by two individual measurements, whose removal significantly reduces the tension between data sets. Next, we use LOO-CV as a model selection tool for the GJ 4276 system. Here, LOO-CV prefers an eccentric single-planet model over a two-planet model, and identifies the orbital phases at which follow-up observations would best discriminate between them. Finally, we extend LOO-CV to models with correlated noise, applying it to the GJ 357 system, whose RVs are modeled with a Gaussian process. We find no predictive support for a fourth planetary signal that is moderately favored by the Bayesian evidence, strengthening the interpretation of a three-planet architecture previously reported in the literature. We provide practical guidance for implementing LOO-CV in RV analysis, together with an online tutorial. Such methods will become ever more important as future RV surveys push the limits of planet detection and characterization.