Handwritten mathematical expression recognition

Abstract

As digital form has been used more and more frequently for text documents but typing mathematical expressions remains difficult, it is crucial to develop an effective system that can read handwritten mathematical expressions. In this project, we try to solve the problem of handwritten mathematical expression recognition by an encoder-decoder model with the help of neural networks, which can convert handwritten mathematical expressions in images to LaTeX representations. We also try to enhance the weight assignment of the attention mechanism in the decoder to improve the performance of the model on pairwise symbols. By training and testing with the CROHME 2019 dataset, the model achieves an expression recognition rate of 39.8% and our enhancement increases the expression recognition rate to 42.1%.