Explainable automated wild-orchid identification combining deep neural networks and Bayesian networks

Deep learning has been shown repeatedly to be a successful method of obtaining accurate classifiers. This also applies to orchid identification from digital photographs. However, deep neural networks possess the major weakness of lack of explainability, missing the ability to explain the reasons behind a decision. Nevertheless, most current research regarding automated orchid identification applies this blackbox approach. By contrast, in this paper we propose a new method for trustworthy automated orchid identification combining two complementary methods: deep neural networks and feature-based Bayesian networks, where the Bayesian network is also utilized for providing an explanation of the generated solutions. We use other deep neural networks to extract flower characteristics, the features, from the images which are subsequently fed into the Bayesian network as uncertain evidence. When combining the deep neural network and the Bayesian network as an ensemble classifier, both reaching the same conclusion, an accuracy of 89.4% is achieved, the most trustworthy outcome. With a human-in-the-loop ensemble classifier, validation results are even better, yielding an accuracy of 98.1%. Our approach also exploits the taxonomic knowledge represented in the Bayesian network to provide an explanation of the solutions for every case, reinforcing further trust in the method. The result is an explainable user-in-the-loop ensemble classifier. Providing explainability can help build user trust in a system and may play a major role when it is used as a learning aid for new orchid enthusiasts. Finally, the proposed method may be also of value in many fields other than plant determination. • A new method for explainable automated flower recognition was developed. • The proposed method successfully balances accuracy and trust. • Deep learning predicts species and extracts morphological characteristics of flowers. • A Bayesian network acts as a classifier and explainer based on taxonomist knowledge. • Experimental results demonstrate that the proposed method works in the real world.