Higher-order evolutionary dynamics with game transitions

Higher-order interactions are prevalent in real-world complex systems and exert unique influences on system evolution that cannot be captured by pairwise interactions. We incorporate game transitions into the higher-order prisoner's dilemma game model, where these transitions consistently promote cooperation. Moreover, in systems with game transitions, the proportion of higher-order interactions has a dual impact, either enhancing the emergence and persistence of cooperation or facilitating invasions that promote defection within an otherwise cooperative system. Correspondingly, bistable states, consisting of mutual defection and either mutual cooperation or coexistence, are consistently identified in both theoretical analyses and simulation results.