Techniques for mass peak reconstruction in searches for long-lived heavy neutral leptons decaying to a lepton and a $$\rho $$ meson

Abstract The precise reconstruction of the mass peak of long-lived heavy neutral leptons (HNLs) helps to improve the sensitivity for sterile neutrino searches in both fixed-target and collider environments (e.g., SHiP and the LHC). We present an analytical framework for reconstructing the HNL mass peak in the semileptonic HNL decay channel $$N \rightarrow \ell \rho $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>N</mml:mi> <mml:mo>→</mml:mo> <mml:mi>ℓ</mml:mi> <mml:mi>ρ</mml:mi> </mml:mrow> </mml:math> with $$\rho \rightarrow \pi \pi ^0$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>ρ</mml:mi> <mml:mo>→</mml:mo> <mml:mi>π</mml:mi> <mml:msup> <mml:mi>π</mml:mi> <mml:mn>0</mml:mn> </mml:msup> </mml:mrow> </mml:math> , using only the lepton and the charged pion emerging from the decay vertex together with kinematic constraints and the known particle masses. Incorporating mass constraints from intermediate resonances (e.g., the $$\rho $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>ρ</mml:mi> </mml:math> meson) or the parent particle (e.g., the $$W$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>W</mml:mi> </mml:math> boson at the collider experiments), we propose two methods, suitable for experiments with displaced vertex tracking capabilities. The particle-level simulation’s results demonstrate that the $$\rho $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>ρ</mml:mi> </mml:math> -mass constraint method yields promising HNL mass resolution in both beam-dump and collider-based environments. The W-mass constraint method, limited to the HNLs produced via $$W$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>W</mml:mi> </mml:math> -boson decays at the collider-based experiments, shows better HNL mass resolution than the $$\rho $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>ρ</mml:mi> </mml:math> -mass constraint method.