Please use this identifier to cite or link to this item: https://hdl.handle.net/10356/148021
Title: Gesture recognition using a bioinspired learning architecture that integrates visual data with somatosensory data from stretchable sensors
Authors: Wang, Ming
Yan, Zheng
Wang, Ting
Cai, Pingqiang
Gao, Siyu
Zeng, Yi
Wan, Changjin
Wang, Hong
Pan, Liang
Yu, Jiancan
Pan, Shaowu
He, Ke
Lu, Jie
Chen, Xiaodong
Keywords: Engineering::Materials
Issue Date: 2020
Source: Wang, M., Yan, Z., Wang, T., Cai, P., Gao, S., Zeng, Y., Wan, C., Wang, H., Pan, L., Yu, J., Pan, S., He, K., Lu, J. & Chen, X. (2020). Gesture recognition using a bioinspired learning architecture that integrates visual data with somatosensory data from stretchable sensors. Nature Electronics, 3, 563-570. https://dx.doi.org/10.1038/s41928-020-0422-z
Journal: Nature Electronics
Abstract: Gesture recognition using machine learning methods is valuable in the development of advanced cybernetics, robotics, and healthcare systems, and typically relies on images or videos. To improve recognition accuracy, such visual data can be fused with data from other sensors, but this approach is limited by the quality of the sensor data and the incompatibility of the datasets. Here, we report a bioinspired data fusion architecture that can perform human gesture recognition by integrating visual data with somatosensory data from skin-like stretchable strain sensors. The learning architecture uses a convolutional neural network for visual processing, and then implements a sparse neural network for sensor data fusion and recognition. Our approach can achieve a recognition accuracy of 100%, and maintain recognition accuracy with noisy, under- or over-exposed images. We also show that our architecture can be implemented for robot navigation using hand gestures with a small error, even in the dark.
URI: https://hdl.handle.net/10356/148021
ISSN: 2520-1131
DOI: 10.1038/s41928-020-0422-z
Rights: © 2020 Macmillan Publishers Limited, part of Springer Nature. All rights reserved. This paper was published in Nature Electronics and is made available with permission of Macmillan Publishers Limited, part of Springer Nature.
Fulltext Permission: open
Fulltext Availability: With Fulltext
Appears in Collections:MSE Journal Articles

SCOPUSTM   
Citations 5

95
Updated on Jun 22, 2022

PublonsTM
Citations 5

109
Updated on Jul 7, 2022

Page view(s)

213
Updated on Aug 12, 2022

Download(s) 20

263
Updated on Aug 12, 2022

Google ScholarTM

Check

Altmetric


Plumx

Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.