A CTMDP Modeling for Multi-Stage Software Aging and Rejuvenation

The seminal work on the software rejuvenation model used a continuous-time Markov chain to analyze the effectiveness of software rejuvenation for improving system availability. The effectiveness in terms of steady-state availability is determined by a condition on the transition rates, known as the rejuvenation threshold. However, such a theoretical condition has not been studied for software aging models with more than two stages. This paper formulates the software rejuvenation decision problem using a continuous-time Markov decision process (CTMDP) and theoretically demonstrates the conditions to determine the optimal rejuvenation policy, maximizing the steady-state availability for the three-stage software aging model. Furthermore, we develop an approximated policy evaluation algorithm for the CTMDP-based software aging and rejuvenation model that allows us to evaluate the steady-state availability of the system with a given decision policy. Through a numerical study, we confirm that the results of the policy evaluation algorithm correctly fit the boundary conditions of the optimal policy for the three-stage model. Our numerical results also demonstrate the boundary conditions for models with more than four stages.