Irregular linear-quadratic two-person nonzero-sum differential games

In this paper, we consider the explicit form of open-loop Nash equilibria for irregular linear-quadratic (LQ) two-person nonzero-sum differential games, where the control weighting matrices in the cost functional are only positive semi-definite. The existence of an open-loop Nash equilibrium is characterized by the solvability of a system of constrained linear forward-backward differential equations (FBDEs). In classical LQ nonzero-sum games, the associated Riccati equation admits a regular solution is adopted as a basic assumption. Thus the feedback representation of open-loop Nash equilibria can be obtained by decoupling the FBDEs through the regular solution to a Riccati equation. However, by an example it is shown that the feedback representation still exists even when the regular condition is not satisfied. The essential differences between regular and irregular LQ nonzero-sum games are investigated. Therefore one of the research purposes of this paper is to relax the basic assumption of regularity. The irregular feedback representation can be derived from two equilibrium conditions in two different layers by using the ``two-layer optimization" approach. The results show that the irregular LQ game is totally different from the regular game as the irregular controller must guarantee the terminal state constraint of $\mathbf{P}_1(T)x^*(T)=0$.