Comparative Analysis of Interpretability of Simple and Complex Machine Learning Models in Presence of Noise

This paper offers a comprehensive analysis of the interpretability of key Machine Learning models, including ElasticNet regression, Random Forest, and Neural Networks, when faced with various types of noise. Focusing on both synthetic and real-world datasets of diverse sizes (385 to 15,000 samples), the study probes the models' ability to detect hidden patterns, especially in the presence of varied noise conditions (Gaussian, Perlin, and Simplex). Through systematic evaluation using Permutation Feature Importance (PFI) and SHAP summary plots, our research reveals a strong correlation between dataset size and model robustness to noise perturbations. The results demonstrate that larger datasets consistently lead to more stable feature importance rankings and better preservation of model interpretability under noise conditions. While ElasticNet shows superior performance on larger datasets, Neural Networks prove most sensitive to noise, particularly with smaller datasets. The findings provide valuable insights for practical applications of machine learning, suggesting that emphasis should be placed on acquiring larger training datasets to ensure robust and trustworthy model interpretations in noisy environments. This work contributes to the broader understanding of ML model interpretability and provides guidance for model selection in real-world applications where data noise is inevitable.