A multi-criteria process for IT project success evaluation–Addressing a critical gap in standard practices

• A novel process combining problem structuring methods and multi-criteria decision analysis is proposed for evaluating IT project success. • The process was applied in a real-world software development project for a global technology and services provider. • It effectively addresses complexity and uncertainty, enhancing clarity and consistency in project evaluation. • The approach is theoretically sound and adaptable to various evaluation contexts and criteria. • The study advocates for the creation of formal standards in IT project evaluation processes. The evaluation of project success is widely recognised as valuable for improving IT (Information Technology) project performance and impact. However, many processes fail to adequately address the requirements for a sound evaluation due to their inherent complexity or by not complying with fundamental practical and theoretical concepts. This paper presents a process that combines a problem structuring method with a multi-criteria decision analysis approach to evaluate the success of IT projects. Put into practice in the context of a software development project developed for a leading global supplier of technology and services, it offers a new way of creating a model for evaluating project success and tackling uncertainty, bringing clarity and consistency to the overall assessment process. A strong advantage of this process is that it is theoretically sound and can be easily applied to other evaluation problems involving other criteria. It also serves as a call to action for the development of formal standards in evaluation processes. Practical pathways to achieve such standardization include collaboration through industry consortia, development and adoption of ISO frameworks, and embedding evaluation processes within established maturity models. These pathways can foster consistency, comparability, and continuous improvement across organizations, paving the way for more robust and transparent evaluation practices.