Exploring object detection from egocentric vision

Abstract

Egocentric vision, which involves capturing data from a first-person perspective, is crucial in many applications such as augmented reality, assistive devices, and human-computer interaction. However, it poses additional challenges compared to conventional object detection due to high-frequency camera motion, frequent occlusions, and variable environmental conditions encountered during acquisition. This dissertation systematically investigates these challenges and evaluates a range of object detection models, particularly those based on convolutional neural networks and transformer architectures. More specifically, we have put emphasis on the EgoObjects dataset where a tailored method TA-IDet was proposed to enhance detection performance by dynamically aligning features with target objects. Moreover, the approach of continual learning techniques to maintain the knowledge over time is also investigated. The findings emphasize the merits and drawbacks of contemporary methods in addressing novel items and adjusting to shifting surroundings. In particular, the work helps to better generalize the model to a wider range of datasets in consideration of object detection in egocentric vision by leveraging memory efficient gradients. Future work will expand into smarter learning strategies and multi-modal methods to better perform in real-world settings.