Convergence of a novel penalty method for quasi variational-hemivariational inequality with application

The paper deals with an elliptic quasi-variational inequality with a set-valued map and implicit constraints depending on a solution. First, a result on nonemptiness and compactness of the solution problem is shown by the penalty method. The convergence of the solution set of the penalized problem is proved in terms of the upper Kuratowski limit. Then, the existence of solution to the penalized problem and the convergence of the penalty method are established for a class of quasi variational-hemivariational inequalities with constraint. Finally, the results are illustrated by a mathematical model of the incompressible Navier-Stokes equation with damping and a mixed multivalued nonmonotone slip boundary condition.