Goal-oriented grasping in clutter with human-in-the-loop implementation

Abstract

For people with motor impairment, robotic arms can act as body surrogates to aid with everyday tasks. However, teleoperation of a high-DoF robotic arm using low-DoF controllers is tedious. Even with the integration of autonomous motion planners and grasp planners, the operation is difficult in a cluttered environment. State-of-the-art works for grasping objects amidst clutter do not work well in practical settings due to errors in perception and scene understanding. In this work, we aim to address these limitations with a Human-In-The-Loop (HITL) approach. We developed an interface which allows human to interrupt and change the operation of the robot if the actions taken by the robot are incorrect. A robot arm was integrated with the interface to develop a complete system for object grasping in clutter. To evaluate the system, experiments were conducted with human subjects. Results of the experiments showed that the proposed implementation provided a greater sense of control and more robust functionalities as compared to other evaluated approaches. Our system also helped users to retrieve objects in a shorter amount of time. Discussions were made on various insights gained while developing and testing this system and ways for further improvement were suggested.