Dynamic Analysis of a Predator–Prey Model with Group Defense in Prey and Cooperative Hunting in Predators

This article discusses a single-prey and single-predator model by incorporating two behavioral mechanisms, namely group defense in prey modeled through a Holling type IV response function and cooperative hunting in predators represented by a predation rate dependent on predator density. Through system analysis, three critical points were obtained, namely the total extinction point, the predator extinction point, and the interior point that describes the coexistence of both species. The total extinction point is always unstable, while the other two critical points are conditionally stable depending on the predator attack rate parameter. Numerical simulations in the form of phase portraits were obtained by varying the parameters related to the predator attack rate. The simulation results show double stability (bistability) when α = 3, where the system can lead to predator extinction or stable coexistence depending on the initial population conditions. Furthermore, numerical continuation was performed to track changes in equilibrium point stability. The results show a Hopf bifurcation at α = 0.4595, marked by a change in the stability of the interior point, a limit point bifurcation (saddle-node) at α = 0.0478 due to the merging of two interior equilibrium points, and a transcritical bifurcation at α = 3.0505, which shows an exchange of stability between equilibrium points.