High-Latitude Ionospheric Irregularities Characterized Through Machine Learning Methods

This study uses Machine Learning and data-driven techniques to understand plasma irregularities in high-latitude regions better. By combining observations and recent findings from modeling, the goal is to identify and classify scintillation signatures caused by different types of irregularities in the ionosphere. The focus is on irregularities from electron precipitation in the auroral oval and ExB drifts in the polar cap. Using Machine Learning tools, the study aims to distinguish between different scintillation signatures and link them to their sources, improving our ability to detect and characterize these events. Using multiple instruments and advanced filtering, the aim is to enhance the accuracy of the scintillation event database and classify irregularities more effectively in high-latitude areas, ultimately advancing our understanding of plasma dynamics in these regions. A novel approach to distinguish the most similar ionospheric time series scintillation signatures from a random group of signatures without any prior information is introduced. This classification is based on the correlation between the input signatures in phase and power in Timeseries Clustering. The observed similarity of the signatures was likely due to steepening spectra during an auroral front. Based on the successful distinction, a hypothesis was formulated: Predominant irregularity mechanisms in the auroral oval are expected to vary from those in the polar cap due to high-latitude ionospheric dynamics. If different mechanisms generate different plasma structures as predicted by theoretical models, then it could be presumed that the scintillation signatures in these two regions have different characteristics. This was tested with a classification algorithm during five geomagnetic storm days. Using time-series hierarchical clustering to compare groups of signatures for pairs of stations, one from the auroral oval vs. one from the polar cap, a decision tree model was trained to classify the signatures. For hours in which both stations were located in their characteristic regions as confirmed by electron energy flux observations, the model achieved a good performance, whereas in hours with similar structures over both stations, the signatures could not be distinguished well. The model performance also depended on the structure of the decision tree classifier, the resolution of the available datasets also for the station labels, and the data quality. The scintillation signature database generated in the previous step was extended by additional observations and estimates that also play a major role in theoretical models: plasma drift, irregularity thickness and layer height, and spectral characteristics. The importance of each parameter for the classification was analyzed using a factor analysis, and the estimation process was optimized for those major factors. The major steps to use this enhanced database and theoretical predictions to train a Support Vector Machine to attempt to link irregularity types with their corresponding scintillation signatures were discussed. Since the time series signatures are crucial to the validity of the machine learning database, the data processing was optimized to remove non-irregularity-related effects. Also, the irregularity scale sizes and dynamics considered in the filtering approach and the effect of an adaptive cutoff frequency on the ML model were investigated. Similarly to the findings from the Timeseries Clustering approach, large-scale contributions to the signature occur to be an important parameter to distinguish different high-latitude source regions. Due to the sparse plasma drift velocity observations available at GNSS-relevant intermediate scale sizes, a technique using spaced GNSS receivers was implemented at a new location in Svalbard, Norway. In collocation with the Incoherent Scatter Radar in Svalbard, this one-dimensional array can probe polar cap scintillation events. However, during the selected case studies, no significant events were detected.