A bi-objective Markov decision process design approach to redundancy allocation with dynamic maintenance for a parallel system

• A Design and Maintenance Problem is formulated as a Bi-Objective MDP (BO-MDP) Design problem. • Heterogenous component-wise usage costs are considered. • A heuristic framework for approximately solving BO-MDP Design problems is introduced. • System designs are found via a probability-focused linearization method. • The heuristic returns optimal Pareto fronts in a fraction of the MILP solver time. The reliability of a system can be improved by the addition of redundant elements, giving rise to the well-known redundancy allocation problem (RAP). We propose a novel extension to the RAP called the bi-objective integrated design and dynamic maintenance problem (BO-IDDMP) which allows for future dynamic maintenance decisions to be incorporated. This leads to a problem with first-stage redundancy design decisions and second-stage sequential maintenance decisions under uncertainty. To the best of our knowledge, this is the first use of a continuous-time Markov Decision Process (MDP) Design framework to formulate a problem with non-trivial dynamics, as well as its first use alongside bi-objective optimization. A general heuristic optimization methodology for bi-objective MDP Design problems is developed, and then applied to the BO-IDDMP. The efficiency and accuracy of our methodology are demonstrated against an exact mixed-integer linear programming solver. The heuristic is shown to be orders of magnitude faster in the majority of cases, and in only 2 out of 84 cases produces a solution that is dominated by the exact method. The inclusion of dynamic maintenance policies is shown to yield stronger and better-populated Pareto fronts, allowing more flexibility for the decision-maker. The impacts of varying parameters unique to our problem are also investigated.