A robust sign language recognition system with multiple Wi-Fi devices

Jiacheng Shang; Jie Wu

doi:10.1145/3097620.3097624

A robust sign language recognition system with multiple Wi-Fi devices

Jiacheng Shang, Jie Wu

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

40 Scopus citations

Abstract

Sign language is important since it provides a way for us to the deaf culture and more opportunities to communicate with those who are deaf or hard of hearing. Since sign language chiefly uses body languages to convey meaning, Human Activity Recognition (HAR) techniques can be used to recognize them for some sign language translation applications. In this paper, we show for the first time that Wi-Fi signals can be used to recognize sign language. The key intuition is that different hand and arm motions introduce different multi-path distortions in Wi-Fi signals and generate different unique patterns in the time-series of Channel State Information (CSI). More specifically, we propose a Wi-Fi signal-based sign language recognition system called WiSign. Different from existing Wi-Fi signal-based human activity recognition systems, WiSign uses 3 Wi-Fi devices to improve the recognition performance. We implemented the WiSign using a TP-Link TL-WR1043ND Wi-Fi router and two Lenovo X100e laptops. The evaluation results show that our system can achieve a mean prediction accuracy of 93.8% and mean false positive of 1.55%.

Original language	English
Title of host publication	MobiArch 2017 - Proceedings of the 2017 Workshop on Mobility in the Evolving Internet Architecture, Part of SIGCOMM 2017
Publisher	Association for Computing Machinery, Inc
Pages	19-24
Number of pages	6
ISBN (Electronic)	9781450350594
DOIs	https://doi.org/10.1145/3097620.3097624
State	Published - 11 Aug 2017
Event	12th ACM SIGCOMM Workshop on Mobility in the Evolving Internet Architecture, MobiArch 2017 - Los Angeles, United States Duration: 25 Aug 2017 → …

Publication series

Name	MobiArch 2017 - Proceedings of the 2017 Workshop on Mobility in the Evolving Internet Architecture, Part of SIGCOMM 2017

Conference

Conference	12th ACM SIGCOMM Workshop on Mobility in the Evolving Internet Architecture, MobiArch 2017
Country/Territory	United States
City	Los Angeles
Period	25/08/17 → …

Keywords

Human activity recognition
Machine learning
Signal processing
Wi-Fi signals

Access to Document

10.1145/3097620.3097624

Cite this

Shang, J., & Wu, J. (2017). A robust sign language recognition system with multiple Wi-Fi devices. In MobiArch 2017 - Proceedings of the 2017 Workshop on Mobility in the Evolving Internet Architecture, Part of SIGCOMM 2017 (pp. 19-24). (MobiArch 2017 - Proceedings of the 2017 Workshop on Mobility in the Evolving Internet Architecture, Part of SIGCOMM 2017). Association for Computing Machinery, Inc. https://doi.org/10.1145/3097620.3097624

Shang, Jiacheng ; Wu, Jie. / A robust sign language recognition system with multiple Wi-Fi devices. MobiArch 2017 - Proceedings of the 2017 Workshop on Mobility in the Evolving Internet Architecture, Part of SIGCOMM 2017. Association for Computing Machinery, Inc, 2017. pp. 19-24 (MobiArch 2017 - Proceedings of the 2017 Workshop on Mobility in the Evolving Internet Architecture, Part of SIGCOMM 2017).

@inproceedings{d9f191514bf34b55bd108614afee1eb6,

title = "A robust sign language recognition system with multiple Wi-Fi devices",

abstract = "Sign language is important since it provides a way for us to the deaf culture and more opportunities to communicate with those who are deaf or hard of hearing. Since sign language chiefly uses body languages to convey meaning, Human Activity Recognition (HAR) techniques can be used to recognize them for some sign language translation applications. In this paper, we show for the first time that Wi-Fi signals can be used to recognize sign language. The key intuition is that different hand and arm motions introduce different multi-path distortions in Wi-Fi signals and generate different unique patterns in the time-series of Channel State Information (CSI). More specifically, we propose a Wi-Fi signal-based sign language recognition system called WiSign. Different from existing Wi-Fi signal-based human activity recognition systems, WiSign uses 3 Wi-Fi devices to improve the recognition performance. We implemented the WiSign using a TP-Link TL-WR1043ND Wi-Fi router and two Lenovo X100e laptops. The evaluation results show that our system can achieve a mean prediction accuracy of 93.8% and mean false positive of 1.55%.",

keywords = "Human activity recognition, Machine learning, Signal processing, Wi-Fi signals",

author = "Jiacheng Shang and Jie Wu",

note = "Publisher Copyright: {\textcopyright} 2017 Association for Computing Machinery.; 12th ACM SIGCOMM Workshop on Mobility in the Evolving Internet Architecture, MobiArch 2017 ; Conference date: 25-08-2017",

year = "2017",

month = aug,

day = "11",

doi = "10.1145/3097620.3097624",

language = "English",

series = "MobiArch 2017 - Proceedings of the 2017 Workshop on Mobility in the Evolving Internet Architecture, Part of SIGCOMM 2017",

publisher = "Association for Computing Machinery, Inc",

pages = "19--24",

booktitle = "MobiArch 2017 - Proceedings of the 2017 Workshop on Mobility in the Evolving Internet Architecture, Part of SIGCOMM 2017",

}

Shang, J & Wu, J 2017, A robust sign language recognition system with multiple Wi-Fi devices. in MobiArch 2017 - Proceedings of the 2017 Workshop on Mobility in the Evolving Internet Architecture, Part of SIGCOMM 2017. MobiArch 2017 - Proceedings of the 2017 Workshop on Mobility in the Evolving Internet Architecture, Part of SIGCOMM 2017, Association for Computing Machinery, Inc, pp. 19-24, 12th ACM SIGCOMM Workshop on Mobility in the Evolving Internet Architecture, MobiArch 2017, Los Angeles, United States, 25/08/17. https://doi.org/10.1145/3097620.3097624

A robust sign language recognition system with multiple Wi-Fi devices. / Shang, Jiacheng; Wu, Jie.
MobiArch 2017 - Proceedings of the 2017 Workshop on Mobility in the Evolving Internet Architecture, Part of SIGCOMM 2017. Association for Computing Machinery, Inc, 2017. p. 19-24 (MobiArch 2017 - Proceedings of the 2017 Workshop on Mobility in the Evolving Internet Architecture, Part of SIGCOMM 2017).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A robust sign language recognition system with multiple Wi-Fi devices

AU - Shang, Jiacheng

AU - Wu, Jie

PY - 2017/8/11

Y1 - 2017/8/11

N2 - Sign language is important since it provides a way for us to the deaf culture and more opportunities to communicate with those who are deaf or hard of hearing. Since sign language chiefly uses body languages to convey meaning, Human Activity Recognition (HAR) techniques can be used to recognize them for some sign language translation applications. In this paper, we show for the first time that Wi-Fi signals can be used to recognize sign language. The key intuition is that different hand and arm motions introduce different multi-path distortions in Wi-Fi signals and generate different unique patterns in the time-series of Channel State Information (CSI). More specifically, we propose a Wi-Fi signal-based sign language recognition system called WiSign. Different from existing Wi-Fi signal-based human activity recognition systems, WiSign uses 3 Wi-Fi devices to improve the recognition performance. We implemented the WiSign using a TP-Link TL-WR1043ND Wi-Fi router and two Lenovo X100e laptops. The evaluation results show that our system can achieve a mean prediction accuracy of 93.8% and mean false positive of 1.55%.

AB - Sign language is important since it provides a way for us to the deaf culture and more opportunities to communicate with those who are deaf or hard of hearing. Since sign language chiefly uses body languages to convey meaning, Human Activity Recognition (HAR) techniques can be used to recognize them for some sign language translation applications. In this paper, we show for the first time that Wi-Fi signals can be used to recognize sign language. The key intuition is that different hand and arm motions introduce different multi-path distortions in Wi-Fi signals and generate different unique patterns in the time-series of Channel State Information (CSI). More specifically, we propose a Wi-Fi signal-based sign language recognition system called WiSign. Different from existing Wi-Fi signal-based human activity recognition systems, WiSign uses 3 Wi-Fi devices to improve the recognition performance. We implemented the WiSign using a TP-Link TL-WR1043ND Wi-Fi router and two Lenovo X100e laptops. The evaluation results show that our system can achieve a mean prediction accuracy of 93.8% and mean false positive of 1.55%.

KW - Human activity recognition

KW - Machine learning

KW - Signal processing

KW - Wi-Fi signals

UR - http://www.scopus.com/inward/record.url?scp=85029692119&partnerID=8YFLogxK

U2 - 10.1145/3097620.3097624

DO - 10.1145/3097620.3097624

M3 - Conference contribution

AN - SCOPUS:85029692119

T3 - MobiArch 2017 - Proceedings of the 2017 Workshop on Mobility in the Evolving Internet Architecture, Part of SIGCOMM 2017

SP - 19

EP - 24

BT - MobiArch 2017 - Proceedings of the 2017 Workshop on Mobility in the Evolving Internet Architecture, Part of SIGCOMM 2017

PB - Association for Computing Machinery, Inc

T2 - 12th ACM SIGCOMM Workshop on Mobility in the Evolving Internet Architecture, MobiArch 2017

Y2 - 25 August 2017

ER -

Shang J, Wu J. A robust sign language recognition system with multiple Wi-Fi devices. In MobiArch 2017 - Proceedings of the 2017 Workshop on Mobility in the Evolving Internet Architecture, Part of SIGCOMM 2017. Association for Computing Machinery, Inc. 2017. p. 19-24. (MobiArch 2017 - Proceedings of the 2017 Workshop on Mobility in the Evolving Internet Architecture, Part of SIGCOMM 2017). doi: 10.1145/3097620.3097624

A robust sign language recognition system with multiple Wi-Fi devices

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this