Development of an augmented reality tactile system for surgical procedure

Abstract

Augmented reality (AR) technology is fast growing these days. The realistic and immersive experience it brings is very impressive. For many years, the medical training field lacked a training method that can realistically simulate operational feedback and the operating environment. The application of AR technology can provide a solution for this. The objective of this project is to develop an AR application used for surgical procedure training, which provides real-time physiological responses to the virtual patient based on the user’s operation and gives intuitive feedback to the user. The target device of the application is Magic Leap One, an AR headset, and iPad. The application is developed using Unity with LuminSDK and ARKit for Magic leap One version and iPad version respectively. The application mainly simulates the procedure of inferior epigastric artery exposure based on mesh deformation in Unity. Other auxiliary sections are added to the two versions of the application. The result of the project is the successful development of a fully functional and usable AR application in two versions. The developed application creates a practical way for medical trainees to practice their surgical skills in a simulated real-life situation.