A data-driven comparative analysis of Agile and Waterfall methodologies: Predicting cost and schedule variances using statistical and machine learning approaches

Project management methodologies like Agile, Waterfall, etc., play an impact- ful role in key performance indicators of the project, such as cost variance, schedule variance, etc. In this work, we deep dive into these variances with data-driven techniques and discover machine learning models for cost estima- tion. To demonstrate the efficacy of our approach, we processed a dataset with Agile and Waterfall project attributes which was collected by means of sur- vey conducted online about 100 developers from various companies. We had through categorical encoding, statistical analysis, hypothesis testing, and pre- dictive modeling to predict and compare the projects which can be successful. In the initial stages of Exploratory Data Analysis (EDA), it can be observed that the distribution of cost and schedule variance is not uniform across the wa- terfall and agile approaches, whereby the mean cost and schedule variances is 2.14 and SD is 1.32 for Agile projects and the mean cost and schedule variances for waterfall projects is higher at 3.87 with SD of 1.89. A T-test conducted to compare the methodologies results in a test statistic of -4.72 and a p-value of 0.00002, indicating a statistically significant difference in cost and schedule variances between Agile and Waterfall projects. Additionally, the use of project attributes to train a linear regression model for predicting cost variance and schedule variance for both waterfall and agile approaches achieves an average MAE of 0.98 and an average MSE of 1.54, indicating moderate predictive ac- curacy in the models. They emphasize that, on average, Agile projects have a lower cost and schedule variance than Waterfall projects and strengthen the im- pact of the project methodology on effort deviations. The study highlights the role of predictive analytics in project management and advocates the adoption of machine learning for more accurate cost estimation. The next step is to in- vestigate more advanced modeling techniques and the use of additional project parameters to improve predictive performance and project planning. • A comprehensive data-driven analysis of project management methodolo- gies (agile and waterfall) in terms of cost and schedule variances with statistical and machine learning techniques 1. • Data analysis using T-test indicates that there is a statistically significant difference between Agile projects (mean = 2.14, SD = 1.32) and Waterfall projects (mean = 3.87, SD = 1.89) in mean cost deviation (p less than 0.00002). • Cost and Schedule Variance being predicted most by the Planned Cost and Planned Duration, respectively, proved significant through ANOVA test results(p = 0.0 for both). • The predictive model estimates and errors are better when modeled using machine learning techniques (Random Forest, Gradient Boosting) com- pared to linear regression for cost and schedule variance predictions, which reveals non-linear relationships in the project data. Gradient Boosting achieved the lowest MAE for cost variance (99.64). • The Random Forest Classifier garnered high performance accuracy (98.91%) in project success prediction (success class project with cost and sched- ule variance within threshold) detection, where class imbalance effect and successful project detection issue come into play. • The risk estimation used in Monte Carlo simulation gave a pivotal value for cost and schedule variances, allowing to identify those projects that were at risk of experiencing a large exceedance. • Feature Importance analysis showed Planned Cost as the foremost con- tributor to cost variance (78.4% contribution), and Planned Duration as the most influential factor for schedule variance (63.5% contribution).