A robust sign language recognition system with multiple Wi-Fi devices

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

16 Scopus citations

Abstract

Sign language is important since it provides a way for us to the deaf culture and more opportunities to communicate with those who are deaf or hard of hearing. Since sign language chiefly uses body languages to convey meaning, Human Activity Recognition (HAR) techniques can be used to recognize them for some sign language translation applications. In this paper, we show for the first time that Wi-Fi signals can be used to recognize sign language. The key intuition is that different hand and arm motions introduce different multi-path distortions in Wi-Fi signals and generate different unique patterns in the time-series of Channel State Information (CSI). More specifically, we propose a Wi-Fi signal-based sign language recognition system called WiSign. Different from existing Wi-Fi signal-based human activity recognition systems, WiSign uses 3 Wi-Fi devices to improve the recognition performance. We implemented the WiSign using a TP-Link TL-WR1043ND Wi-Fi router and two Lenovo X100e laptops. The evaluation results show that our system can achieve a mean prediction accuracy of 93.8% and mean false positive of 1.55%.

Original languageEnglish
Title of host publicationMobiArch 2017 - Proceedings of the 2017 Workshop on Mobility in the Evolving Internet Architecture, Part of SIGCOMM 2017
PublisherAssociation for Computing Machinery, Inc
Pages19-24
Number of pages6
ISBN (Electronic)9781450350594
DOIs
StatePublished - 11 Aug 2017
Event12th ACM SIGCOMM Workshop on Mobility in the Evolving Internet Architecture, MobiArch 2017 - Los Angeles, United States
Duration: 25 Aug 2017 → …

Publication series

NameMobiArch 2017 - Proceedings of the 2017 Workshop on Mobility in the Evolving Internet Architecture, Part of SIGCOMM 2017

Conference

Conference12th ACM SIGCOMM Workshop on Mobility in the Evolving Internet Architecture, MobiArch 2017
CountryUnited States
CityLos Angeles
Period25/08/17 → …

Keywords

  • Human activity recognition
  • Machine learning
  • Signal processing
  • Wi-Fi signals

Fingerprint

Dive into the research topics of 'A robust sign language recognition system with multiple Wi-Fi devices'. Together they form a unique fingerprint.

Cite this