Visual servo control of robot manipulator with applications to construction automation

Abstract

The construction industry has long been a labour-intensive sector. The gap between the continuously increasing demand for housing and the shirking workforce is growing wider day-to-day. In addition, the fatal and injury rate at the workplace for the construction sector remains stubbornly high as compared to other industries. Construction companies are seeking for robotics and automation technologies to keep a balance between safety, accuracy, and efficiency. Vision system is a crucial part of the robotic system in construction automation. By deploying a vision system on a robot, we can automate the detections of construction materials, installation components, and defects. Current state-of-art models for object detection extract the image feature vectors using a deep neural network which consists of a series of convolutional layers and max-pooling layers to generate the final output. After training a deep neural network model with a suitable dataset, it can classify and localize numerous classes. Its detection performance will be fixed throughout the prediction process. Typically, detection results will include object class, confidence level, bounding box size, and bounding box coordinates. The confidence level determines how confident we are to confirm the presence of an object. It can vary dramatically due to different lighting conditions, changes in the distance and angle of the camera. This thesis aims to explore the use of robot visual servoing technique to improve detection performance during real-time inspection. The proposed method utilizes object detection information to guide the robot system for achieving a better view of the target object. A region-based visual servoing controller is developed to position the target object in the center of the field of view (FOV) while also maximize of the coverage of the object within the FOV. A case study will be performed on tile cracks inspection by using the proposed technique. The inspection process is an important step to evaluate the current stage of the construction project as well as alert the supervisors if there is an error. It is also a tedious job as it requires close observation through every wall in every room among all the units. Tile cracks are commonly occurring during the transportation or installation process and the cracks are usually tiny and therefore not easily detected by human workers. By combining the visual servo control technique with a deep-learning-based object detector, we aim to achieve a higher confidence level for the detection of the tile cracks. Experimental results are presented to illustrate the performance.