Discovering and modeling hidden periodicities in science data

Abstract Hidden periodicities in science data have long been a popular topic of investigation. The popularity stems from the fact that detecting and characterizing periodicities can provide a means for extracting information from science data—information that might not otherwise be accessible. In other words, periodicities in data can be exploited for the purposes of statistical inference and decision making. The long history of this topic is briefly reviewed with heavy reference to a historical essay on the topic by H.O.A. Wold, written more than half a century ago, following which the treatise focuses on a paradigm shifting advance in theory and methodology for characterizing hidden periodicities that was initiated by the second author in the mid-1980s and further advanced by both authors since then, including a plethora of algorithms for performing the needed computations in applications. The data models this theory is based on are generally called cyclostationary but include variations that are labeled with modifiers like wide-sense, strict sense, n-th order for n = 1, 2, 3,..., almost, poly, and irregular . The theory is probabilistic, but is intentionally not based on stochastic processes which, it is argued, are inappropriate for many, if not most, applications. The basis used is Fraction-of-Time (FOT) Probability . The concept, theory, and methodology of FOT Probability is itself a major paradigm shift, also initiated by the second Author more than half a century ago, and it is an integral part of the (preferred) non-stochastic theory of cyclostationarity. Since the birth of this topic, both authors have continued to advance these paradigm shifts, including further development of theory, associated methodology, and computational algorithms. The most advanced of the concepts described (viz., irregular poly-cyclostationarity) is illustrated with an application of the associated algorithms to science data consisting of time series of Sunspot numbers containing approximately 75,000 daily measurements representing a period of about 200 years. The results include the first methodical characterization of the irregularity of the poly-periodicity hidden in the data.