An Augmented FPV Drone Racing System

In case you haven’t heard about it, augmented First Person View (FPV) Drone Racing is a system that lets spectators understand the situation of drone races by using augmented feedback techniques. From projection mapping to autonomous commentaries, a new project and study by a group of authors is pushing drone racing to a new perspective, allowing the spectators and pilots to understand the race situation in an easier way.

An augmented FPV drone race: spectators are able to understand the situation of the hybrid physical/virtual drone races by using an autonomous commentary (left), the first person view (center), and an onboard LED POV display

Promoting Drone Racing to the Masses

The aim of the paper is simple – to further promote drone racing to the masses and show why by 2020, the size of the drone market in the entertainment and media sector is expected to reach $8.8 billion in a total market cap of $130 billion.

As the authors noted in the beginning of their paper:

“Nowadays, hard-to-follow sports including FPV drone racing need technological support to be spectator friendly. FPV drones are too small and fast to identify (or match) with pilots and their FPV for the spectator.

This is one of the reasons why drone racing is hard to follow regarding the situation of the race (i.e., leaders, order of racers, and speeds). To solve these problems, we exploit interactive techniques to improve race visibility and entertainment, including projection mapping, a 3D display (LED POV display) for following each drone along with their flight paths, and sound effects/autonomous commentaries.”

Using the Drone as a Flying Display

To address these challenges, the authors are developing an augmented drone racing system which consists of drones with an on-board LED POV array, motion capture devices and projection mapping technique in order to visualize the flight path and identify the drones in an individual way. A display is also used to show the pilots’ view during the races with sound effects as well as autonomous commentaries.

System structure

By using the drone as a flying display, the authors envision a system that visually emphasizes the drone’s flight path by proper projection mapping that is based on motion capture.

“The drone’s positioning is tracked by motion detection cameras that are installed above the race course. Then the projected visual effects are shown on the screen behind the drones and trace the drone’s flight path. We exploit a POV optical illusion to indicate visual feedback (i.e. shapes or texts with full-color lighting patterns) in mid-air by utilizing a drone as a flying LED display. We have been developing a prototype which is equipped with existing LED POV components.”

As a result of this, the drones (or pilots) are not only able to indicate the status to the spectators – they can also interact with them via text messages and visual effects throughout the race.

A system like this is obviously dependent of autonomous sound effects (linked to the drone actions) and commentaries. The authors also figured that placing physical landmarks in the race course can be effective for the association of the position and identity of the drones – enhancing the sense of speed and entertainment for the first person view.

Final Words

All in all, this model aims on tackling all the hard-to-follow sports such as FPV drone racings – and can be applicable to be spectator-friendly. Since FPV drones are small and fast to identify with pilots, they come up with a plan to solve the common problems – in the form of interactive techniques that improve the race visibility and entertainment including projection mapping, 3D displays (LED POV displays) and actions for following each drone along with its flights, as well as sound effects and autonomous commentaries.

All of this can help spectators – as well as drone racers – to promote the sport further and make it easy to watch. The research in this paper, as noted, was supported by the Singapore National Research Foundation, RIEC Nation-wide Cooperative Research Projects (Tohoku University), as well as the HAYAO NAKAYAMA Foundation for Science, Technology and Culture.

Citation: Koh Sueda, Takashi Kitada, Yushin Suzuki, and Taiki Wada. 2018. Research and development of augmented FPV drone racing system. In SIGGRAPH Asia 2018 Posters (SA ’18). ACM, New York, NY, USA, Article 9, 2 pages. DOI: https://doi.org/10.1145/3283289.3283322 – https://dl.acm.org/citation.cfm?id=3283322