Augmenting the Communication Naturalness via A 3D Audio-Visual Virtual Agent for Collaborative Robots

Rui Li, Weitian Wang

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

In human-robot collaboration, current widely used human-robot communication is mainly based on audio and haptic mediums. However, this kind of communication is stiff and mechanical. Inspired by human-human communication in which vision and hearing contribute over 88% for human perception, we propose a knowledge-driven audio-visual virtual agent system, which allows collaborative robots to present its knowledge and feelings in a human-like way. During the collaboration training process, the virtual agent will build its assembly knowledge of how to work with the co-worker based on inverse reinforcement learning. To deploy a co-assembly task with its human partner, the virtual agent will also be able to produce assembly knowledge-based responses, which include knowledge-driven speech and speech synchronized facial animations. By leveraging the proposed knowledge-driven virtual agent, the collaborative robot not only can fulfill the co-assembly task but also can communicate with human partner in a more natural way.

Original languageEnglish
Title of host publicationProceedings - 2021 IEEE International Conference on Big Data, Big Data 2021
EditorsYixin Chen, Heiko Ludwig, Yicheng Tu, Usama Fayyad, Xingquan Zhu, Xiaohua Tony Hu, Suren Byna, Xiong Liu, Jianping Zhang, Shirui Pan, Vagelis Papalexakis, Jianwu Wang, Alfredo Cuzzocrea, Carlos Ordonez
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages5944-5946
Number of pages3
ISBN (Electronic)9781665439022
DOIs
StatePublished - 2021
Event2021 IEEE International Conference on Big Data, Big Data 2021 - Virtual, Online, United States
Duration: 15 Dec 202118 Dec 2021

Publication series

NameProceedings - 2021 IEEE International Conference on Big Data, Big Data 2021

Conference

Conference2021 IEEE International Conference on Big Data, Big Data 2021
Country/TerritoryUnited States
CityVirtual, Online
Period15/12/2118/12/21

Keywords

  • collaborative robots
  • communication naturalness
  • human-robot interaction
  • virtual agent

Fingerprint

Dive into the research topics of 'Augmenting the Communication Naturalness via A 3D Audio-Visual Virtual Agent for Collaborative Robots'. Together they form a unique fingerprint.

Cite this