A radiative seesaw model in a non-invertible selection rule with the assistance of a non-holomorphic modular <i>A</i> 4 symmetry

Abstract We propose a two-loop neutrino mass model where fermionic and bosonic dark matter (DM) candidates are simultaneously connected to the neutrinos. But the fermionic DM candidate is favored compared to the bosonic one due to generating the fermionic DM mass at one-loop level. In order to obtain our desired Lagrangian and Higgs potential, we introduce a Z3 gauging TY non-invertible fusion rule with the assistance of a non-holomorphic modular A4 symmetry. The fusion rule forbids the mass of DM candidate at tree level but its mass is generated at one-loop level where the DM mass term dynamically violates the fusion rule. After that, the neutrino mass matrix is induced at one-loop level where a remnant Z2 symmetry is still remained. The symmetry assures the stability of our DM candidate. The non-holomorphic modular A4 symmetry plays a role in forbidding the interactions between the SM particles and heavier fermions XR and an isospin singlet inert scalar boson S0 that run in the DM mass loop, in addition to reduction our free parameters that leads to our predictions for lepton sector. We perform χ2 numerical analysis for the lepton masses, mixing angles, and phases, and we show several predictions for NH and IH. Then, we demonstrate our lepton flavor violations, muon anomalous magnetic dipole moment, and the relic density of the DM candidate fixing the best fit point of the lepton sector.