Skip to main navigation Skip to search Skip to main content

A Speech and Facial Information based Emotion Recognition System of Collaborative Robot for Empathic Human-Robot Collaboration

  • Jianna Loor
  • , Jordan Murphy
  • , Rui Li

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

A robot's ability to effectively recognize human emotions is critical in human-robot collaboration. However, most of the current collaborative robots were designed to improve productivity. Few of these robots consider human emotions. This situation would cause humans to be unwilling to work with robots for a long time. Motivated by this gap, this research developed a human emotion recognition system for enhancing the interaction abilities of collaborative robots. In this project, both speech and facial information were analyzed for robust human emotion recognition in complex working environments like manufacturing assembly environments. In the experiment, the developed system has been tested through three different human-robot co-assembly scenarios: (1) the robot effectively assists humans in finishing the task, (2) the robot is slow in response, leading to the task failing, (3) the robot frequently picks up wrong tools leading to task failure. Experimental results have demonstrated the effectiveness of the developed system in recognizing human co-worker's emotions when the human and the robot were working in the above scenarios. It further shows the developed system has the potential to contribute to the development of an empathic collaborative robot companion in the manufacturing area.

Original languageEnglish
Title of host publication33rd IEEE International Conference on Robot and Human Interactive Communication, ROMAN 2024
PublisherIEEE Computer Society
Pages2327-2332
Number of pages6
ISBN (Electronic)9798350375022
DOIs
StatePublished - 2024
Event33rd IEEE International Conference on Robot and Human Interactive Communication, ROMAN 2024 - Pasadena, United States
Duration: 26 Aug 2024 → 30 Aug 2024

Publication series

NameIEEE International Workshop on Robot and Human Communication, RO-MAN
ISSN (Print)1944-9445
ISSN (Electronic)1944-9437

Conference

Conference33rd IEEE International Conference on Robot and Human Interactive Communication, ROMAN 2024
Country/TerritoryUnited States
CityPasadena
Period26/08/24 → 30/08/24

Fingerprint

Dive into the research topics of 'A Speech and Facial Information based Emotion Recognition System of Collaborative Robot for Empathic Human-Robot Collaboration'. Together they form a unique fingerprint.

Cite this