Quantifying Explainability: Metrics and Methodologies for Dense Neural Network Interpretability

As machine learning algorithms increasingly influence decisions in our daily lives, understanding these models becomes critical. This paper explores the philosophical and practical elements of model interpretability, focusing on dense neural networks, which are often perceived as “black boxes.” LIME, SHAP, Partial Dependence Plots, Counterfactual Explanations, and Surrogate Models are applied to a dense neural network trained on the California Housing Dataset to evaluate their effectiveness in explaining the model's behavior. These methods provide insights into feature importance, global versus local model behavior, and potential decision-making logic. However, many aspects of neural networks remain unknown, such as complex feature interactions, internal decision pathways, and the specific roles of hidden layers. The paper highlights the persistent unknowns within the black box of neural networks and outlines potential avenues for future research. Ultimately, enhancing the understanding of these models is key to the trustability and usability of these large-scale machine learning models.