A Robust Sign Language Recognition System with Sparsely Labeled Instances Using Wi-Fi Signals

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

21 Scopus citations

Abstract

Sign language is important since it permits insight into the deaf culture and allows more opportunities to communicate with those who are deaf or hard of hearing. In this paper, we show that Wi-Fi signals can be used to recognize sign language with sparsely labeled training dataset. The key intuition is that sign language introduces different multi-path distortions in Wi-Fi signals and generates different unique patterns in the time-series of Channel State Information (CSI) values. Based on these observations, we propose a sign language recognition system called WiSign. Different from existing Wi-Fi signal-based human activity recognition systems, WiSign only requires a sparsely labeled training dataset. Two solutions based on transfer learning and semi-supervised learning are proposed to reduce the number of required labeled instances. We implemented WiSign using a TP-Link TL-WR1043ND Wi-Fi router and a Lenovo X100e laptop. The evaluation results show that WiSign can achieve a mean prediction accuracy of 87.01% and 87.38% for the transfer learning-based approach and semi-supervised learning-based approach, respectively.

Original languageEnglish
Title of host publicationProceedings - 14th IEEE International Conference on Mobile Ad Hoc and Sensor Systems, MASS 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages99-107
Number of pages9
ISBN (Electronic)9781538623237
DOIs
StatePublished - 14 Nov 2017
Event14th IEEE International Conference on Mobile Ad Hoc and Sensor Systems, MASS 2017 - Orlando, United States
Duration: 22 Oct 201725 Oct 2017

Publication series

NameProceedings - 14th IEEE International Conference on Mobile Ad Hoc and Sensor Systems, MASS 2017

Conference

Conference14th IEEE International Conference on Mobile Ad Hoc and Sensor Systems, MASS 2017
Country/TerritoryUnited States
CityOrlando
Period22/10/1725/10/17

Keywords

  • human recognition systems
  • machine learning
  • signal processing

Fingerprint

Dive into the research topics of 'A Robust Sign Language Recognition System with Sparsely Labeled Instances Using Wi-Fi Signals'. Together they form a unique fingerprint.

Cite this