Local and global graph modeling with edge-weighted graph attention network for handwritten mathematical expression recognition

• We propose a novel End-to-end model with an Edge-weighted Graph Attention Mechanism (EGAT), designed for simultaneous node and edge classification tasks. • We introduce stroke-level Local Graph Modeling (LGM) and Global Graph Modeling (GGM) that applies the general EGAT model for Online Handwritten Mathematical Expression Recognition tasks. • Our system achieves improvements in symbol detection (94.12% in CROHME 2019, 93.21% in CROHME 2023), relation classification (94.49% in CROHME 2019, 93.45% in CROHME 2023), and expression-level recognition(60.72% in CROHME 2019, 55.30% in CROHME 2023) compare to our previous work [1] . In this paper, we present a novel approach to Handwritten Mathematical Expression Recognition (HMER) by leveraging graph-based modeling techniques. We introduce an end-to-end model with an Edge-weighted Graph Attention Mechanism (EGAT), designed to perform simultaneous node and edge classification. This model effectively integrates node and edge features, facilitating the prediction of symbol classes and their relationships within mathematical expressions. Additionally, we propose a stroke-level Graph Modeling method for both local (LGM) and global (GGM) information, which applies the end-to-end model to Online HMER tasks, transforming the recognition problem into node and edge labeling tasks in graph structure. By capturing both local and global graph features, our method ensures comprehensive understanding of the expression structure. Through the combination of these components, our system demonstrates superior performance in symbol detection, relation classification, and expression-level recognition.