TY - GEN
T1 - A Robust Sign Language Recognition System with Sparsely Labeled Instances Using Wi-Fi Signals
AU - Shang, Jiacheng
AU - Wu, Jie
N1 - Publisher Copyright:
© 2017 IEEE.
PY - 2017/11/14
Y1 - 2017/11/14
N2 - Sign language is important since it permits insight into the deaf culture and allows more opportunities to communicate with those who are deaf or hard of hearing. In this paper, we show that Wi-Fi signals can be used to recognize sign language with sparsely labeled training dataset. The key intuition is that sign language introduces different multi-path distortions in Wi-Fi signals and generates different unique patterns in the time-series of Channel State Information (CSI) values. Based on these observations, we propose a sign language recognition system called WiSign. Different from existing Wi-Fi signal-based human activity recognition systems, WiSign only requires a sparsely labeled training dataset. Two solutions based on transfer learning and semi-supervised learning are proposed to reduce the number of required labeled instances. We implemented WiSign using a TP-Link TL-WR1043ND Wi-Fi router and a Lenovo X100e laptop. The evaluation results show that WiSign can achieve a mean prediction accuracy of 87.01% and 87.38% for the transfer learning-based approach and semi-supervised learning-based approach, respectively.
AB - Sign language is important since it permits insight into the deaf culture and allows more opportunities to communicate with those who are deaf or hard of hearing. In this paper, we show that Wi-Fi signals can be used to recognize sign language with sparsely labeled training dataset. The key intuition is that sign language introduces different multi-path distortions in Wi-Fi signals and generates different unique patterns in the time-series of Channel State Information (CSI) values. Based on these observations, we propose a sign language recognition system called WiSign. Different from existing Wi-Fi signal-based human activity recognition systems, WiSign only requires a sparsely labeled training dataset. Two solutions based on transfer learning and semi-supervised learning are proposed to reduce the number of required labeled instances. We implemented WiSign using a TP-Link TL-WR1043ND Wi-Fi router and a Lenovo X100e laptop. The evaluation results show that WiSign can achieve a mean prediction accuracy of 87.01% and 87.38% for the transfer learning-based approach and semi-supervised learning-based approach, respectively.
KW - human recognition systems
KW - machine learning
KW - signal processing
UR - http://www.scopus.com/inward/record.url?scp=85040547284&partnerID=8YFLogxK
U2 - 10.1109/MASS.2017.41
DO - 10.1109/MASS.2017.41
M3 - Conference contribution
AN - SCOPUS:85040547284
T3 - Proceedings - 14th IEEE International Conference on Mobile Ad Hoc and Sensor Systems, MASS 2017
SP - 99
EP - 107
BT - Proceedings - 14th IEEE International Conference on Mobile Ad Hoc and Sensor Systems, MASS 2017
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 14th IEEE International Conference on Mobile Ad Hoc and Sensor Systems, MASS 2017
Y2 - 22 October 2017 through 25 October 2017
ER -