Application of the One-Step Second-Derivative Method for Solving the Transient Distribution in Markov Chain

Markov chains are an application of stochastic models in operation research, helping the analysis and optimization of processes with random events and transitions. The method that will be deployed to obtain the transient solution to a Markov chain problem is an important part of this process. The present paper introduces a novel Ordinary Differential Equation (ODE) approach to solve the Markov chain problem. The probability distribution of a continuous-time Markov chain with an infinitesimal generator at a given time is considered, which is a resulting solution of the Chapman-Kolmogorov differential equation. This study presents a one-step second-derivative method with better accuracy in solving the first-order Initial Value Problems (IVPs) compared to other approaches found in the literature, which is verified by the obtained solutions. The determination of the transient solutions for Markov chains is presented using the proposed method. The results show better accuracy in solving the transient distribution in Markov chains, which implies that there is an improved assurance in adopting this approach in future studies of the Markov chain modeling process for predicting future events based on the current state of a process. Future studies on Markov chain modeling could adopt the introduced method to predict future events based on the current state of a process.