Robotic arm based companion system

Abstract

This report presents the development of a 6 DOF robotic arm desktop companion that is designed to interact with the user, improving their physical health as well as their mental well-being. The robotic arm’s movements are controlled using forward and inverse kinematics, through the use of MATLAB to compute precise joint angles.

The robotic arm promotes healthy work habits by implementing work-break timers. During the work sessions, the robotic arm ensures users remain productive through the detection of drowsiness using a custom YOLOv4-Tiny model for facial recognition. It alerts the user if they are detected to be drowsy.

After finishing a work session, the break session is initiated by the robotic arm guiding users through stretching exercises. During the break session, the robotic arm can play interactive minigames with the user, including Rock-Paper-Scissors, Tic-Tac-Toe and Connect4. The Rock-Paper-Scissors game uses gesture recognition with MediaPipe, while Tic-Tac-Toe and Connect 4 implement the Minimax algorithm to make logical decisions.

Posture monitoring is achieved using several MediaPipe landmarks across the upper body, allowing the robot arm to provide real-time feedback, encouraging good sitting posture throughout the day.

This project showcases the potential of combining computer vision, algorithms, and robotics to create an interactive and intelligent desktop companion that enhances the user’s productivity as well as their well-being.