ARSpy: Breaking Location-Based Multi-Player Augmented Reality Application for User Location Tracking

Jiacheng Shang, Si Chen, Jie Wu, Shu Yin

Research output: Contribution to journalArticlepeer-review

Abstract

Augmented reality (AR) applications that overlay the perception of the real world with digitally generated information are on the cusp of commercial viability. AR has appeared in several commercial platforms like Microsoft HoloLens and smartphones. They extend the user experience beyond two dimensions and supplement the normal 3D world of a user. A typical location-based multi-player AR application works through a three-step process, wherein the system collects sensory data from the real world, identifies objects based on their context, and finally, renders information on top of senses of a user. However, because these AR applications frequently exchange data with users, they have exposed new individual and public safety issues. In this paper, we develop ARSpy, a user location tracking system solely based on network traffic information of the user, and we test it on location-based multi-player AR applications. We demonstrate the effectiveness and efficiency of the proposed scheme via real-world experiments on 12 volunteers and show that we could obtain the geolocation of any target with high accuracy. We also propose three mitigation methods to mitigate these side channel attacks. Our results reveal a potential security threat in current location-based multi-player AR applications and serve as a critical security reminder to a vast number of AR users.

Original languageEnglish
Pages (from-to)433-447
Number of pages15
JournalIEEE Transactions on Mobile Computing
Volume21
Issue number2
DOIs
StatePublished - 1 Feb 2022

Keywords

  • attack
  • Augmented reality
  • localization

Fingerprint

Dive into the research topics of 'ARSpy: Breaking Location-Based Multi-Player Augmented Reality Application for User Location Tracking'. Together they form a unique fingerprint.

Cite this