Data-Driven Body-Machine Interface for Drone Intuitive Control through Voice and Gestures

A. Menshchikov, D. Ermilov, I. Dranitsky, L. Kupchenko, M. Panov, M. Fedorov, A. Somov

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    8 Citations (Scopus)

    Abstract

    Aerial drones can be used for a number of monitoring and control applications. Most of existing drone control platforms are quite primitive in terms of body-machine interface. They are usually a variation of a hand-held remote controller or ground control station. However, in a number of line-of-sight scenarios it would be more convenient to use the human gestures and voice for the drone control. In this work, we present an approach for instantaneous control of drones based on human voice and gestures. The proposed solution includes wearable sensors and embedded artificial intelligence. We use a microphone and an Inertial Measurement Unit (IMU) for capturing the human voice and the hand movements. Primary control is implemented by a voice recognition unit based on Recurrent Neural Network (RNN) while the secondary control is implemented by the gesture recognition system based on Convolutional Neural Network (CNN). For implementing the embedded intelligence, we use a low-power embedded system with a graphical processing unit able to run pre-trained neural networks on board of the drone. As a result, the system can perform different speech and gesture recognition tasks real-time.

    Original languageEnglish
    Title of host publicationProceedings
    Subtitle of host publicationIECON 2019 - 45th Annual Conference of the IEEE Industrial Electronics Society
    PublisherIEEE Computer Society
    Pages5602-5659
    Number of pages58
    ISBN (Electronic)9781728148786
    DOIs
    Publication statusPublished - Oct 2019
    Event45th Annual Conference of the IEEE Industrial Electronics Society, IECON 2019 - Lisbon, Portugal
    Duration: 14 Oct 201917 Oct 2019

    Publication series

    NameIECON Proceedings (Industrial Electronics Conference)
    Volume2019-October

    Conference

    Conference45th Annual Conference of the IEEE Industrial Electronics Society, IECON 2019
    Country/TerritoryPortugal
    CityLisbon
    Period14/10/1917/10/19

    Keywords

    • Body-Machine Interface
    • Embedded Systems
    • Machine Learning
    • Teleoperation
    • UAV
    • Wearable Devices

    Fingerprint

    Dive into the research topics of 'Data-Driven Body-Machine Interface for Drone Intuitive Control through Voice and Gestures'. Together they form a unique fingerprint.

    Cite this