Deep learning object detection and classification for affordance learning

Abstract

Deep learning object detection has been the hot topic among machine learning in recent years. As a result, various deep learning frameworks and models were designed for implementation of object detection in different situations. However, besides object detection, there has also been a need for smarter models and systems that are capable of more than just detection. In this case, the concept of affordance learning was introduced to utilize the visual information from detected instances and to understand other properties of the object such as if it is singular or multiple, upright or tilted, rigid or deformable, an even movable or static. In this project, a comprehensive study was performed on various feature extraction techniques, deep learning basics, as well as dataset preparation, for implementation in an object detector and image classifier. The project was targeted towards identifying cups and mugs, and the implemented affordance learning determines if the detected instance of the cup is singular or multiple. Apart from these, a study was also conducted on various deep learning frameworks to assess their strengths and best areas for implementation. Moreover, a Python script was written also to provide a one-stop solution for dataset building and management.