ERI: High-performance Human-robot Collaborative Manufacturing Enabled by Integrated Multimodal Teaching, Learning, Prediction and Interaction

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).

Robotics is one of critical technologies in advancing the manufacturing industry, because of its potential to heighten the efficiency in the productivity and part quality. Traditional industrial robots are fenced off from human workers on production lines; on the contrary, collaborative robots are not, making them capable of democratizing manufacturing industries with dynamic customer demands and high flexibility. Currently, however, most collaborative robots are programmed by conventional off-line coding for specific manufacturing applications, e.g., assembly. Then the robots collaborate with human partners via predefined workflows to perform simply repetitive tasks. Such a static human-robot collaborative manufacturing process may be degraded, if any work environment settings or tasks are ever changed. To address this challenge and advance human-robot collaborative manufacturing, this Engineering Research Initiation (ERI) award will develop a competitive solution to train robots not only to be effectively programed by learning from human demonstrations, but also to actively assist human partners to jointly accomplish manufacturing tasks. The research will contribute toward fundamental engineering research on human-robot collaboration in advanced manufacturing. This project will offer students at Montclair State University, which has a diverse student body from underrepresented groups, with the latest robotics training and research, which will diversify the future workforce and potentially benefit the US industry. In addition, this project will launch robotics workshops with cutting-edge hands-on activities for local K-12 schools, especially from underserved districts.

The goal of this project is to develop a teaching-learning-prediction-collaboration framework for robots to proactively learn from human demonstrations, predict human intentions, and collaborate with humans in collaborative manufacturing tasks. The major questions to be solved include the following: (i) how can a human-robot collaborative manufacturing process be mathematically described and can robots learn task knowledge from human demonstrations, (ii) how can robots assist human partners based on the prediction of human intentions in the collaboration process, and (iii) how can the framework be validated in human-robot collaborative manufacturing tasks? To fill the knowledge gaps, the human-robot collaboration will be parameterized through a Markov Decision Process and develop a multimodal-information-based approach for robots to learn task customization and human working preference from human partners' demonstrations in collaborative manufacturing environments. Further, computational human intention prediction and human-robot collaboration models will be developed for robots to leverage the learned strategies to proactively predict human partners' upcoming intentions and assist humans in shared tasks. Moreover, user studies will be conducted to evaluate the effectiveness of the approaches in collaboration quality improvement by applying findings to real-world human-robot collaborative tasks in advanced manufacturing contexts.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.