TY - GEN
T1 - A robust sign language recognition system with multiple Wi-Fi devices
AU - Shang, Jiacheng
AU - Wu, Jie
N1 - Funding Information:
This research was supported in part by NSF grants CNS 1629746, CNS 1564128, CNS 1449860, CNS 1461932, CNS 1460971, CNS 1439672, CNS 1301774, ECCS 1231461, and ECCS 1231461.
Publisher Copyright:
© 2017 Association for Computing Machinery.
PY - 2017/8/11
Y1 - 2017/8/11
N2 - Sign language is important since it provides a way for us to the deaf culture and more opportunities to communicate with those who are deaf or hard of hearing. Since sign language chiefly uses body languages to convey meaning, Human Activity Recognition (HAR) techniques can be used to recognize them for some sign language translation applications. In this paper, we show for the first time that Wi-Fi signals can be used to recognize sign language. The key intuition is that different hand and arm motions introduce different multi-path distortions in Wi-Fi signals and generate different unique patterns in the time-series of Channel State Information (CSI). More specifically, we propose a Wi-Fi signal-based sign language recognition system called WiSign. Different from existing Wi-Fi signal-based human activity recognition systems, WiSign uses 3 Wi-Fi devices to improve the recognition performance. We implemented the WiSign using a TP-Link TL-WR1043ND Wi-Fi router and two Lenovo X100e laptops. The evaluation results show that our system can achieve a mean prediction accuracy of 93.8% and mean false positive of 1.55%.
AB - Sign language is important since it provides a way for us to the deaf culture and more opportunities to communicate with those who are deaf or hard of hearing. Since sign language chiefly uses body languages to convey meaning, Human Activity Recognition (HAR) techniques can be used to recognize them for some sign language translation applications. In this paper, we show for the first time that Wi-Fi signals can be used to recognize sign language. The key intuition is that different hand and arm motions introduce different multi-path distortions in Wi-Fi signals and generate different unique patterns in the time-series of Channel State Information (CSI). More specifically, we propose a Wi-Fi signal-based sign language recognition system called WiSign. Different from existing Wi-Fi signal-based human activity recognition systems, WiSign uses 3 Wi-Fi devices to improve the recognition performance. We implemented the WiSign using a TP-Link TL-WR1043ND Wi-Fi router and two Lenovo X100e laptops. The evaluation results show that our system can achieve a mean prediction accuracy of 93.8% and mean false positive of 1.55%.
KW - Human activity recognition
KW - Machine learning
KW - Signal processing
KW - Wi-Fi signals
UR - http://www.scopus.com/inward/record.url?scp=85029692119&partnerID=8YFLogxK
U2 - 10.1145/3097620.3097624
DO - 10.1145/3097620.3097624
M3 - Conference contribution
AN - SCOPUS:85029692119
T3 - MobiArch 2017 - Proceedings of the 2017 Workshop on Mobility in the Evolving Internet Architecture, Part of SIGCOMM 2017
SP - 19
EP - 24
BT - MobiArch 2017 - Proceedings of the 2017 Workshop on Mobility in the Evolving Internet Architecture, Part of SIGCOMM 2017
PB - Association for Computing Machinery, Inc
T2 - 12th ACM SIGCOMM Workshop on Mobility in the Evolving Internet Architecture, MobiArch 2017
Y2 - 25 August 2017
ER -