Assessing the utility of synthetic images for computer vision tasks

Abstract

In computer vision, machine learning requires huge amount of training data in order to achieve a better accuracy in object recognition. However, collecting real images that covers all kinds of intra-category variations is both tedious and time-consuming. Hence, 3D CAD models is used to overcome the issue of collecting large amount of real images as it is widely available on the internet. The use of synthetic models as training models has actually been used by others in the past in training object detectors or for text recognition task. A challenge of using synthetic models for training is that it lacks photo-realism. Certain feature cues of a real object such as surface texture, lighting, background, pose are lacking in CAD models. This could result in poor accuracy of the objects detected. The objective of this project is to demonstrate the use of synthetically generated 2D images from 3D CAD models in object recognition. To solve the issue of non-photo-realistic CAD models, the synthetic models are rendered to be as close as possible to the real object by adding background and supporting surface with the CAD models. In addition, a robust pipeline is setup to extract important features from the 2D images and finally trained with a classifier. The feature representation methods used in this project are SIFT and HOG. The classifiers used are SVM and ELM. Overall, the use of CAD images as part of the training model did not produce a high accuracy. Moreover, rendering of a background with the synthetic image did not play a significant role in increasing the overall accuracy. However, after adding real images into the CAD dataset, there was a significant improvement in the result