Transformer networks for continuous gravitational-wave searches

Wide-parameter-space searches for continuous gravitational waves using semicoherent matched-filter methods require enormous computing power, which limits their achievable sensitivity. Here we explore an alternative search method based on training neural networks as classifiers on detector strain data with minimal preprocessing. Contrary to previous studies using convolutional neural networks (CNNs), we investigate the suitability of the transformer architecture, specifically the vision transformer (ViT). We establish sensitivity benchmarks using the matched-filter <a:math xmlns:a="http://www.w3.org/1998/Math/MathML" display="inline"> <a:mi mathvariant="script">F</a:mi> </a:math> -statistic for ten targeted searches over a ten day timespan, and ten directed and six all-sky searches over a one day timespan. We train ViTs on each of these benchmark cases. The trained ViTs achieve essentially matched-filter sensitivity on the targeted benchmarks, and approach the <d:math xmlns:d="http://www.w3.org/1998/Math/MathML" display="inline"> <d:mi mathvariant="script">F</d:mi> </d:math> -statistic detection probability of <g:math xmlns:g="http://www.w3.org/1998/Math/MathML" display="inline"> <g:mrow> <g:msub> <g:mrow> <g:mi>p</g:mi> </g:mrow> <g:mrow> <g:mi>det</g:mi> </g:mrow> </g:msub> <g:mo>=</g:mo> <g:mn>90</g:mn> <g:mo>%</g:mo> </g:mrow> </g:math> on the directed ( <i:math xmlns:i="http://www.w3.org/1998/Math/MathML" display="inline"> <i:mrow> <i:msub> <i:mrow> <i:mi>p</i:mi> </i:mrow> <i:mrow> <i:mi>det</i:mi> </i:mrow> </i:msub> <i:mo>≈</i:mo> <i:mn>85</i:mn> <i:mi>–</i:mi> <i:mn>89</i:mn> <i:mo>%</i:mo> </i:mrow> </i:math> ) and all-sky benchmarks ( <k:math xmlns:k="http://www.w3.org/1998/Math/MathML" display="inline"> <k:mrow> <k:msub> <k:mrow> <k:mi>p</k:mi> </k:mrow> <k:mrow> <k:mi>det</k:mi> </k:mrow> </k:msub> <k:mo>≈</k:mo> <k:mn>78</k:mn> <k:mi>–</k:mi> <k:mn>88</k:mn> <k:mo>%</k:mo> </k:mrow> </k:math> ). Unlike the CNNs in our previous studies, which required extensive manual design and hyperparameter tuning, the ViT achieves better performance with a standard architecture and minimal tuning.