Evolution of a supermassive binary black hole in a nonspherical nuclear star cluster

(abridged) We consider a secular orbital evolution of a supermassive binary black hole (SBBH) with unequal masses $M_p$ and $M_s < M_p$ in a central part of a non-spherical nuclear star cluster (NSC). When the mass of NSC inside the orbit is smaller than $M_{s}$ dynamical friction becomes inefficient. The subsequent orbital evolution of SBBH is largely governed by perturbing tidal potential of NSC arising from its non-sphericity. When the perturbing potential is mainly determined by quadrupole harmonics with azimuthal number $|m|=2$ the secular dynamics of the SBBH does not conserve any components of the angular momentum and can lead to the formation of highly eccentric orbits. Such orbits can experience an efficient circularization due to emission of gravitational waves (GW). In this Paper we consider this situation in some detail. We study analytically and numerically the orbital evolution taking into account the important effect of Einstein apsidal precession and estimate the largest possible value of $e$, which can be obtained. We then estimate a possibility of fast orbital circularization through emission of gravitation waves on a highly eccentric orbit, with circularization timescale of the order of the orbital period. We find that our mechanism could result in such events on a time scale of the order of or smaller than a few Gyr. It is stressed that for particular values of the model parameters such events may sometimes be distinguished from the ones expected in more standard scenarios, since in our case the eccentricity may remain substantial all the way down to the final merger. It is also noted that our results can be applied to other astrophysical settings, e.g. to study the orbital evolution of a binary star or a proto planetary system inside a massive deformed gas cloud.