Multi-modality object detection in car cabin monitoring

Abstract

With the rapid advancement of autonomous driving technology, ensuring cabin safety and providing a smarter, more personalized driving experience have become critical objectives. Computer vision-based object detection has emerged as a key research topic in intelligent cabin systems. However, the constantly changing conditions within cabin environments present significant challenges to maintaining detection accuracy and reliability.

This dissertation explores an effective multi-modality object detection model tailored for complex cabin scenarios. We adopted a Cross-Modality Fusion Transformer (CFT) model, built on a dual-stream YOLOv5 feature extraction backbone, and apply it to the multimodal TICaM dataset. Comprehensive preprocessing steps, including format adjustments and data augmentation, are employed to optimize feature fusion between RGB and infrared modalities. Experimental results demonstrate that the CFT model significantly enhances accuracy and robustness compared to single-modality models.

To improve inference speed and enable real-time applications in vehicle monitoring systems, we also modified the original Cross Stage Partial Network module in YOLOv5. This attempt lays foundations for future efforts in balancing detection accuracy and inference efficiency, advancing the development of real-time intelligent vehicle monitoring systems.