Enhancing hand-drawn diagram recognition through the integration of machine learning and deep learning techniques

For many generations, hand-drawn diagrams were utilized as a common graphical means of interaction in various fields, such as engineering, architecture, and education. Because human-made graphics are inherently complicated and variable, hand-drawn diagram recognition is a challenging task. Consequently, there is a growing demand for effective methods and techniques for recognizing and understanding hand-drawn diagrams. In this research, an integrated approach is being developed that combines the best features of multiple machine learning approaches to enhance overall performance and deal with the weaknesses of individual methods. Additionally, deep learning techniques, which are well known for their ability to find intricate patterns and features in data, are incorporated into the proposed system. This research proposes improvised methods such as Fossum Soergel k-means clustering, morphological Canny Bessel radial basis contour shape factor, Fisher kernel k-nearest neighbor, sing-scurve fuzzy rule generation, and wide context faster regional convolutional neural network to enhance the system's performance. The system's performance is evaluated by experimentation with benchmark datasets of hand-drawn flowcharts, finite automata, and business process models. Testing is done on the combined datasets to generalize the approach. Experimentation results are compared with the state-of-the-art methods to check the proposed model's performance. The results show that the proposed system can digitize various diagrams automatically. Future research directions on offline hand-drawn diagrams are also explored in this paper.