Reassessing Machine Learning Techniques for Electrocatalyst Design: A Call for Robust Methodologies

Gu et al. conducted a comprehensive survey on the design and application of electrocatalysts powered by machine learning techniques [1]. They presented a novel approach that utilizes Artificial Neural Networks (ANN) in conjunction with the SHAP (SHapley Additive exPlanations) method to optimize membrane electrode assemblies. The ANN model demonstrated high accuracy in predicting key performance metrics, achieving root mean square error (RMSE) values of 43.536 mW cm−2 for power density and 0.070 gPt kW−1 for platinum utilization. Additionally, the SHAP method was employed to identify the most influential features affecting the target outputs, providing valuable insights into the optimization process [1]. However, this paper raises significant theoretical and empirical concerns regarding the use of ANN in conjunction with SHAP due to the model-specific nature of these techniques, which can lead to erroneous interpretations. It appears that Gu et al. may not fully grasp the fundamental principles underlying machine learning. In supervised machine learning models like ANN, two types of accuracy are crucial: target prediction accuracy and feature importance reliability. While target prediction accuracy can be validated against known ground truth values, the derived feature importances from models lack equivalent ground truth for validation. As a result, achieving high target prediction accuracy does not ensure that the feature importances are also reliable, since there are no established ground truth values for these features. The function call “explain = SHAP(model)” further indicates that SHAP may inherit and potentially amplify any biases present in the feature importances derived from the underlying model (ANN), leading to misleading interpretations of the results [2-5]. This highlights the importance of critically evaluating both the predictions and the interpretability provided by model-agnostic methods like SHAP. In light of these concerns, the paper advocates for a more robust and multifaceted approach utilizing unsupervised machine learning techniques, such as Feature Agglomeration (FA) and Highly Variable Gene Selection (HVGS). FA is a dimensionality reduction technique that aggregates similar features, thereby simplifying the data set and reducing noise, which can enhance the interpretability of the model and the reliability of its predictions. HVGS focuses on selecting a subset of features that exhibit significant variability across samples, ensuring that only the most informative features are retained for further analysis. Following the feature selection process, the authors suggest employing nonlinear nonparametric statistical methods, such as Spearman's correlation, to assess the relationships between features and outcomes. Spearman's correlation evaluates the strength and direction of the association between ranked variables, making it particularly useful in identifying monotonic relationships that do not necessarily follow a linear pattern. The accompanying p-values provide a measure of statistical significance, offering insights into the reliability of these correlations. By leveraging these advanced methods, researchers can achieve a deeper understanding of the key factors influencing electrocatalyst performance, while also mitigating the risks associated with relying solely on model-specific interpretations. Yoshiyasu Takefuji completed this study and wrote this article. The author has nothing to report. The author has nothing to report. The author declares no conflicts of interest. The author has nothing to report.