DronePaint: Swarm Light Painting with DNN-based Gesture Recognition

Valerii Serpiva, Ekaterina Karmanova, Aleksey Fedoseev, Stepan Perminov, Dzmitry Tsetserukou

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Citations (Scopus)

Abstract

We propose a novel human-swarm interaction system, allowing the user to directly control a swarm of drones in a complex environment through trajectory drawing with a hand gesture interface based on the DNN-based gesture recognition. The developed CV-based system allows the user to control the swarm behavior without additional devices through human gestures and motions in real-time, providing convenient tools to change the swarm's shape and formation. The two types of interaction were proposed and implemented to adjust the swarm hierarchy: Trajectory drawing and free-form trajectory generation control. The experimental results revealed a high accuracy of the gesture recognition system (99.75%), allowing the user to achieve relatively high precision of the trajectory drawing (mean error of 5.6 cm in comparison to 3.1 cm by mouse drawing) over the three evaluated trajectory patterns. The proposed system can be potentially applied in complex environment exploration, spray painting using drones, and interactive drone shows, allowing users to create their own art objects by drone swarms.

Original languageEnglish
Title of host publicationSpecial Interest Group on Computer Graphics and Interactive Techniques Conference Emerging Technologies, SIGGRAPH 2021
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9781450383646
DOIs
Publication statusPublished - 5 Aug 2021
EventSpecial Interest Group on Computer Graphics and Interactive Techniques Conference: Emerging Technologies, SIGGRAPH 2021 - Virtual, Online, United States
Duration: 9 Aug 202113 Aug 2021

Publication series

NameSpecial Interest Group on Computer Graphics and Interactive Techniques Conference Emerging Technologies, SIGGRAPH 2021

Conference

ConferenceSpecial Interest Group on Computer Graphics and Interactive Techniques Conference: Emerging Technologies, SIGGRAPH 2021
Country/TerritoryUnited States
CityVirtual, Online
Period9/08/2113/08/21

Keywords

  • Deep Neural Network
  • Gesture Recognition
  • Human-Drone Interaction
  • Light Painting

Fingerprint

Dive into the research topics of 'DronePaint: Swarm Light Painting with DNN-based Gesture Recognition'. Together they form a unique fingerprint.

Cite this