Mobile media tagging and sharing

Abstract

Huge advancement in mobile and network technology has open various opportunities for development in mobile media application. One such area involves content-based visual information retrieval (CVIR), where mobile users are able to make a search using images rather than words. The long-term goal is to integrate different methods of tagging and searching of images to improve performance. However, this final year project aims to build the fundamental foundation for the long-term project first by experimenting on the existing image recognition tools and explore different techniques to achieve high performance accuracy with the shortest cost (computational time).

To narrow the scope, the area of focus is on Landmark recognition. The Singapore Landmark Database is utilized. It consists of 40 categories and 13409 images and has been reorganised with real-time application in mind. While both Matlab and OpenCV codes are used, on the other hand OpenCV is mainly utilized to allow easier system integration for future works. A Bag-of-Words framework is adopted for the image recognition tools. For feature extraction the Scale-based Invariant Feature Transform (SIFT) is used, while both the hierarchical k-means and scalable vocabulary tree are used for clustering and machine learning respectively.

Two different sampling methods are experimented on; the Dense SIFT (dense- sampling) and the Key-Point SIFT (key-point sampling). The experimental results conclude that Key-Point SIFT is better performing with an increase in recognition rate (>19%) and has a faster computational time (>20%) for landmark recognition. Two additional experiments were conducted; the first on Geometric Verification (GV) and second on Saliency Mapping integration. The results conclude that both GV and Saliency Mapping allow better performance (1~2%), but at the expense of computational time (increase in GV >2s; increase in Saliency <0.03s). Future works may include the expansion of the Singapore Landmark database and integration of GPS into the stored image content to increase performance accuracy.