Deep learning for haptic feedback of flexible endoscopic robot without prior knowledge on sheath configuration

Abstract

Distal-end force information is usually missing in flexible endoscopic robots due to the difficulties of mounting miniature force sensors on their end-effectors. This hurdle creates big challenges in providing a sense of touch for the operating surgeons. Many existing studies have developed models to calculate the distal-end forces based on the measured proximal-end forces of Tendon-Sheath Mechanisms (TSMs), but these models assume known sheath bending configuration which is unknown during real-life surgeries. This paper presents a two-stage data-driven method that makes dynamic distal-end force prediction of a flexible endoscopic robot without this assumption. In stage one, a convolutional neural network is used to estimate the sheath cumulative bending angle based on the proximal-end force responses of the robot to a probing signal; in stage two, a combination of two long-short-term-memory models pre-trained for the bending angles nearest to the estimated angle (obtained in stage one) makes dynamic estimations of the distal-end force of the robot. The proposed approach overcomes the challenges due to unknown TSM configurations and can robustly identify the correct force hysteresis phases of TSMs. The force prediction is continuous, accurate, and has a mean RMSE of 0.1711 N. This method was validated on an actual flexible surgical robot. In addition, since the proposed approach provides an estimation of the current system cumulative bending angle, it can also be used to facilitate the mathematical modeling methods which require information on the cumulative bending angle.