FINet: Handwriting trajectory reconstruction of Chinese characters based on the font imitate network

Recovering online handwriting trajectory from Chinese handwritten images is still a difficult problem because of abundant strokes, complex structures and different writing styles of Chinese characters. To address this problem, this paper proposes a handwriting trajectory reconstruction method for Chinese characters called the Font Imitate Network. The Font Imitate Network integrates two networks, the spatial encoder uses HRNet to encode the spatial association information of each position on the offline image; the temporal decoder consists of a multilayer perceptron and a recurrent neural network. The stroke feature network is designed to help predict the next handwriting point by focusing on the stroke path in the form of heatmap. Experiments are conducted on the CASIA-OLHWDB1.1 with DTW, LDTW, RMSE and character recognition accuracy as the evaluation metrics. The experimental results show our FINet outperforms the state-of-the-art methods and the recognition accuracy of offline characters is improved by the handwriting reconstruction. • Proposing the FINet model to reconstruct complex Chinese handwriting trajectory. • Integrating spatial encoding and temporal decoding to reconstruct online trajectory. • Introducing the stroke feature to enhance the preservation of trajectory attributes. • Verifying that the FINet significantly outperforms the state-of-the-art methods. • Verifying that reconstructed trajectory improves the offline handwritten recognition.