LaTeXNet: A Specialized Model for Converting Visual Tables and Equations to LaTeX Code

LaTeX provides precise representation of complex elements (i.e., tables and equations) in scientific documents. However, the automated transcription of visual representations into LaTeX code is challenging and prone to errors. This paper introduces LaTeXNet, a specialized model designed to automate the conversion of visual tables and equations into LaTeX code. First, we develop an automated annotation tool that extracts image-LaTeX pairs for tables, equations, and text paragraphs with inline equations from arXiv platform, creating the MM-LaTeX dataset with over 2.5M pairs. Moreover, we design the LaTeXNet model, trained on MM-LaTeX, which unifies the conversion of Tables, Equations, and TextEqs. Our experimental results indicate that LaTeXNet surpasses both open-source and commercial, closed-source models in Table-to-LaTeX, Equation-to-LaTeX and TextEq-to-LaTeX tasks.