Reconnaissance d’expressions mathématiques manuscrites à l’aide de réseaux neuronaux graphiques et de modèles linguistiques arborescents

Handwritten Mathematical Expression Recognition (HMER) aims to translate handwritten formulas into machine-readable representations, bridging visual perception and symbolic reasoning. Despite progress in deep learning, HMER remains challenging due to handwriting ambiguity, 2D spatial complexity, and intricate mathematical syntax. This thesis addresses HMER from a structural perspective by modeling mathematical expressions as graphs and trees in 2D structure. We propose an Edge-weighted Graph Attention Network (EGAT) for joint symbol and relation classification, and a Tree-Structured Transformer (TST) Decoder for hierarchical structure prediction. To facilitate evaluation, the CROHME 2023 competition was organized with newly collected handwritten and large-scale synthetic data. Contextual extensions of CROHME were further introduced to assess the role of textual and visual context using vision-language models. The proposed approaches achieve competitive performance and contribute new insights and resources toward more structured and context-aware mathematical expression recognition.