Mobile application on a scene text spotting

Abstract

Scene text spotting serves as an important concept in many practical applications. In particular, the applications of text spotting may include but not limited to reducing human labor of manual text extraction tasks, retrieving information from images for image context analysis, or automatically identifying human identity by reading their identification card. With many important applications of text spotting, many attempts on implementing software applications that adopted different text spotting methods have been made. While these existing applications have eased many human activities that require text extraction from natural scenes, they also experienced some limitations. Some applications still require too many manual actions from users and cannot spot text in real-time while others may suffer from accuracy and performance issues due to the obsolete text spotting methods. To resolve those issues, this project proposed the implementation of a mobile application that adopted a well-known text spotting approach known as the Adaptive Bezier Curve Network. The performance of this approach, which has been evaluated on TotalText and CTW1500 dataset, proved to achieve a state-of-the-art accuracy while having considerably high inference speed compared to the other existing state-of-the-art methods. In addition to adopting this approach, the project has successfully built an application and a text spotting server using socket programming method as well as our own defined image streaming protocol. Finally, the experiments conducted to measure the performance of the application shows that it is capable of real-time text spotting with up to eight frames per seconds on average while retaining the state-of-the-art text spotting accuracy.