Autonomous Drones for Filming in Dynamic Unstructured Outdoor Environments

The recent mass commercialization of affordable UAVs or drones has altered the media production landscape and allowed easy acquisition of impressive aerial footage. The relevant applications in this manner include production of movies, television shows or even commercials. We have seen drones on TV filming outdoor events, news stories and even on the big screen, being used instead of helicopters as an affordable and more functional alternative.Autonomous UAV Filming

In a new study titled ‘Autonomous Unmanned Aerial Vehicles Filming in Dynamic Unstructured Outdoor Environments’ and published in the IEEE Signal Processing Magazine a group of authors aim to introduce and survey the emerging field of autonomous UAV filming in an attempt to familiarize the reader with the area and highlight the inherent signal processing aspects and challenges.

As the authors note in their introduction, the future of drones looks bright but there are still some challenges which need to be addressed:

“Legal restrictions typically include a requirement for a pilot maintaining direct line-of-sight with the vehicle at all times (fully autonomous civilian drones are illegal), maximum permissible flight altitude and minimum distance from human crowds. Energy consumption restrictions are also important, given the UAV continuous flight time possible with current battery technology, as well as related limitations on processing power and payload weight; the latter are factors that further reduce battery life.”

Challenges and Positives

A number of challenges arise along with the new opportunities, mainly seen in the battery autonomy limitations, legal issues, finite bandwidth in the wireless communication channels and the “dead” time intervals required for UAVs to travel from one point to another in order to shoot from a different angle.

On the good side of things, UAVs offer fast and adaptive shot setup, an ability to hover above a point of interest, access to narrow spaces and the possibility for novel aerial shot types that are not easily achievable otherwise, at a minimal cost.

A visualization of the presented technologies, clustered in three groups. Within each group, the methods currently only appearing in research settings are written in a red font, while the methods currently employed in commercial UAVs are written in a green font.

Intelligent UAV Shooting

Another major part of the study includes the intelligent UAV shooting as an emerging research area with significant industry potential – which goal is to automate as much as of the media production process as possible while ensuring adherence to artistic and cinematographic constraints.

“Although there are examples of algorithms that simply calculate the appropriate number of drones, so as to provide maximum coverage of targets from appropriate viewpoints, in general, little to no effort has been expended towards investigating automated shooting of dynamic scenes in unstructured environments using multiple cooperating UAVs,” the authors commented.

Autonomous UAV Filming

When it comes to automated UAV flight and filming, it is a process that requires a number of underlying enabling technologies to be in place, especially if they are to operate in a satisfactory manner.

However, the current processing power and energy consumption restrictions limit what is possible on a UAV in comparison to desktop computers. The main benefit of UAVs is the 2D visual target tracking which tracks a pre-specified ROI on the consecutive frames of a video sequence, taking advantage of spatio-temporal locality constraints and updating the ROI pixel coordinates at each video frame.

Conclusion

After reviewing all the autonomous features in current commercial UAVs and taking the employed algorithms in commercial drones in detail, the authors conclude that during the 21st century, drones have evolved from remotely controlled units to fully autonomous agents.

In that manner, aerial cinematography has already been transformed with the new advancements and the easy availability of advanced VTOL drones. However, there is still a lot of room for improvements in many aspects.“The currently limited UAV autonomy, the lack of commercial off-the-self cooperative UAV swarm platforms, the multitude of complications arising from legal or technological restrictions, as well as the absence of multiple-UAV cinematography expertise, are all issues prescribing directions for advancement,” according to the authors.

Citation: Mademlis, Ioannis & Nikolaidis, Nikos & Tefas, Anastasios & Pitas, Ioannis & Wagner, Tilman & Messina, Alberto. (2018). Autonomous Unmanned Aerial Vehicles Filming in Dynamic Unstructured Outdoor Environments. IEEE Signal Processing Magazine. 36. 147-153. 10.1109/MSP.2018.2875190. https://www.researchgate.net/publication/329964109_Autonomous_Unmanned_Aerial_Vehicles_Filming_in_Dynamic_Unstructured_Outdoor_Environments?enrichId=rgreq-6e26fbfda7c33a39614b1b24c0fc5c50-XXX&enrichSource=Y292ZXJQYWdlOzMyOTk2NDEwOTtBUzo3MDg3ODEyNjA0MjMxNjlAMTU0NTk5ODAyOTQwNA%3D%3D&el=1_x_3&_esc=publicationCoverPdf