BystandAR: Protecting Bystander Visual Data in Augmented Reality Systems

Matthew Corbett, Brendan David-John, Jiacheng Shang, Y. Charlie Hu, Bo Ji

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

Augmented Reality (AR) devices are set apart from other mobile devices by the immersive experience they offer. While the powerful suite of sensors on modern AR devices is necessary for enabling such an immersive experience, they can create unease in bystanders (i.e., those surrounding the device during its use) due to potential bystander data leaks, which is called the bystander privacy problem. In this paper, we propose BystandAR, the first practical system that can effectively protect bystander visual (camera and depth) data in real-time with only on-device processing. BystandAR builds on a key insight that the device user's eye gaze and voice are highly effective indicators for subject/bystander detection in interpersonal interaction, and leverages novel AR capabilities such as eye gaze tracking, wearer-focused microphone, and spatial awareness to achieve a usable frame rate without offloading sensitive information. Through a 16-participant user study,we show that BystandAR correctly identifies and protects 98.14% of bystanders while allowing access to 96.27% of subjects. We accomplish this with average frame rates of 52.6 frames per second without the need to offload unprotected bystander data to another device.

Original languageEnglish
Title of host publicationMobiSys 2023 - Proceedings of the 21st Annual International Conference on Mobile Systems, Applications and Services
PublisherAssociation for Computing Machinery, Inc
Pages370-382
Number of pages13
ISBN (Electronic)9798400701108
DOIs
StatePublished - 18 Jun 2023
Event21st Annual International Conference on Mobile Systems, Applications and Services, MobiSys 2023 - Helsinki, Finland
Duration: 18 Jun 202322 Jun 2023

Publication series

NameMobiSys 2023 - Proceedings of the 21st Annual International Conference on Mobile Systems, Applications and Services

Conference

Conference21st Annual International Conference on Mobile Systems, Applications and Services, MobiSys 2023
Country/TerritoryFinland
CityHelsinki
Period18/06/2322/06/23

Keywords

  • augmented reality
  • bystander privacy
  • eye tracking
  • visual data

Fingerprint

Dive into the research topics of 'BystandAR: Protecting Bystander Visual Data in Augmented Reality Systems'. Together they form a unique fingerprint.

Cite this