CobotAR: Interaction with Robots using Omnidirectionally Projected Image and DNN-based Gesture Recognition

Elena Nazarova, Oleg Sautenkov, Miguel Altamirano Cabrera, Jonathan Tirado, Valerii Serpiva, Viktor Rakhmatulin, Dzmitry Tsetserukou

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Citations (Scopus)

Abstract

Several technological solutions supported the creation of interfaces for Augmented Reality (AR) multi-user collaboration in the last years. However, these technologies require the use of wearable devices. We present CobotAR -a new AR technology to achieve the Human-Robot Interaction (HRI) by gesture recognition based on Deep Neural Network (DNN) - without an extra wearable device for the user. The system allows users to have a more intuitive experience with robotic applications using just their hands. The CobotAR system assumes the AR spatial display created by a mobile projector mounted on a 6 DoF robot. The proposed technology suggests a novel way of interaction with machines to achieve safe, intuitive, and immersive control mediated by a robotic projection system and DNN-based algorithm. We conducted the experiment with several parameters assessment during this research, which allows the users to define the positives and negatives of the new approach. The mental demand of CobotAR system is twice less than Wireless Gamepad and by 16% less than Teach Pendant.

Original languageEnglish
Title of host publication2021 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2590-2595
Number of pages6
ISBN (Electronic)9781665442077
DOIs
Publication statusPublished - 2021
Event2021 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2021 - Melbourne, Australia
Duration: 17 Oct 202120 Oct 2021

Publication series

NameConference Proceedings - IEEE International Conference on Systems, Man and Cybernetics
ISSN (Print)1062-922X

Conference

Conference2021 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2021
Country/TerritoryAustralia
CityMelbourne
Period17/10/2120/10/21

Fingerprint

Dive into the research topics of 'CobotAR: Interaction with Robots using Omnidirectionally Projected Image and DNN-based Gesture Recognition'. Together they form a unique fingerprint.

Cite this