Machine Learning Methods for Controlling Seismic Attenuation Systems In Gravitational Waves Detectors

On September 14, 2015, the first detection of gravitational waves (GW) opened a new window into the cosmos. Today, the second generation of ground-based detectors — Advanced LIGO, Advanced Virgo, and KAGRA — is fully operational with a total of 90 confirmed events in the third catalog (GWTC-3) and the O4 run currently underway. Two decades later, the third-generation of GW ground-based detectors, and in particular the Einstein Telescope (ET), will promise to broaden our understanding of the Universe even more, expanding the frequency range to 3 Hz, thus potentially enabling breakthroughs in areas such as high-mass black hole formation, binary neutron star early warning, and high-redshift black hole physics. In this framework, the problem of seismic attenuation at low frequency gains a key role. Achieving ET seismic isolation requirements demands a new approach to attenuation systems, as the current SuperAttenuator (SA) design - 17 m high multi-pendula chain - poses challenges in cost and feasibility. Additionally, ET will reasonably feature a greater number of degrees of freedom compared to current systems, necessitating a control approach capable of managing this increased complexity. A solution for a compact suspension is being developed in Pisa and it is called Pendulum Inverted Pendulum (PIP), designed to ideally fit within 10 m. Together with the passive attenuation, an active control of the SA is required to keep the interferometer locked: if the SA architecture changes, also the active control method must be redesigned accordingly. To address these challenges, we propose a novel Reinforcement Learning (RL)-based control system. In fact, RL-controls are highly flexible, adaptable and capable of managing multiple degrees of freedom simultaneously, making it an appealing alternative for the complexities of the ET design. The control consists of a Proximal Policy Optimization (PPO) algorithm with enhancements like generalized State-Dependent Exploration (gSDE) and continuity costs and is initially tested on a Cart-Pole model intending to approximate the dynamics of a single PIP leg. As expected, the system appears to adapt dynamically to noise and retain memory for optimization. We further extend the model to three dimensions, incorporating the PIP leg deformable joint dynamics via simplified springs-based simulations, ensuring compatibility with RL environments. We also re-model the joint to be suitable for a Finite Element Method (FEM) simulation to be integrated in an RL environment. This work provides a first exploratory results for an automated, adaptable control systems for seismic isolation that could be applied to seismic isolation systems for the gravitational wave detectors, with particular attention to next-generation ET, Advanced Virgo post-O5 and the PIP itself.