Computer vision applications on the NVIDIA jetson platform

Abstract

Video stabilization, a video enhancement technique which removes unwanted shake, is becoming increasingly important with the emergence of embedded systems with cameras. The NVIDIA Jetson platforms, claimed to be the cutting-edge solutions to embedded computer vision and machine learning, have been commercially integrated into moving platforms such as drones. This project investigated and proposed a complete pipeline of video stabilization tasks, from motion estimation to video completion in order to retain the resolution. Feature-based and block-matching methods are employed in the estimation stage and Kalman filter is used to stabilize the motion. The feature-based approach relies on Shi-Tomasi corner detector and Lucas-Kanade pyramidal optical flow to estimate the motion. The block-matching method is extended with brute-force search and interpolation to estimate the angle. To achieve real-time processing, CUDA-accelerated codes are utilized for parallel computing. The result is an application capable of processing at 41fps under resolution 640x360 and robust against local motions.