A Vision-Aided Flocking System to Target Drones

Flocking systems are a hot topic nowadays in the drone world. There are many designed systems, coordinated methods as well as realistic approaches to the feature known as flocking – and the actual possibility of drones to flock like birds.

Outlining the New Day Camera and Thermal Camera Approach

A new paper submitted by seven authors in the IIEE Transactions on Industrial Electronics edition presents a new sophisticated and vision-aided flocking system that would target unmanned aerial vehicles (UAVs) and their possibility to operate in GPS-denied and unknown environments for exploring and searching missions.

The paper outlines a day camera and a thermal camera approach – both of which conducting flocking using visual sensing in different illumination and with autonomous navigation capabilities. Naturally, this system is a collective behavior which is organized in a group-like approach, exhibited by many living beings such as fish, insects and birds.

The approach of the authors here, aside from operating in these areas, is to measure the relative motion between UAVs in different lighting conditions without the use of wireless communication.

The Framework that Will Track and Learn to Detect the Drone’s Movement

As the authors state in the abstract of the paper:

“In order to realize robust vision-aided flocking, an integrated framework of tracking-learning-detection on the basis of multi-feature coded correlation filter has been developed. To achieve long-term tracking, a redetector is trained online to adaptively reinitialize target for global sensing.”

They also think that an advanced flocking strategy is specifically developed to address the cooperative flight of multi-UAVs. From light detection and ranging (LiDAR)-based navigation modules to other systems that are developed for autonomous localization, mapping and obstacle avoidance – the flocking abilities are getting more and more sophisticated.

Dividing the Current Flocking Strategies in Two Classes

There are two main classes, which according to the authors can represent the groups of flocking strategies – centralized and decentralized.

The centralized strategies would use a central unit to organize behaviors of flocking systems and may be technically unfeasible for large-scale systems. Decentralized systems, on the other hand, are being built based on the interactions between UAVs. A typical decentralized strategy would include a leader and many followers – acting as a team and simplifying the flocking control to an individual-motion planning.

To realize the flocking feature in radio-silent situations, the authors aim to use passive vision sensing and therefore achieve relative distance measurement in flocking.

A Framework of the Flocking System

There are three basic requirements through which a flock of UAVs can operate in an unknown and obstacle-rich environments, according to the paper.

As the authors wrote:

“1) each individual UAV can avoid obstacles; 2) maintain a team via vision of the follower’s camera; and 3) reach a goal by following the team leader who knows the destination.”

As such, each UAV would be an intelligent agent in an autonomous fashion for task operation. In the navigation module, the LiDAR system will help drones to sense the surrounding environment for navigation and obstacle avoidance – all thanks to on-board cameras which are installed for the followers to track their leader’s movement and mimic the trajectory reference.

Numerous Flight Experiments and Numerous Possibilities for Integration

There have been many flight experiments conducted by experts in order to verify the effectiveness of the proposed flocking systems. A quadrotor UAV has been used in most of them, with the ability to fly about 15 minutes with a 1.5 kilogram extra payload by using an 8000mAh LiPo battery.

From vision tracking to environment exploring, the flocking systems which is presented in the paper is sophisticated and can be integrated in advanced technologies such as the LiDAR-based SLAM, and a visual system for sensing day and night without continuous wireless communication and GPS signals (that are required in traditional flocking systems).

In the end, the developed flocking system can be used as a prototype of multi-UAV autonomous flights and can be applied in many cases in the future – where it needs to be fully optimized and reduced while integrating in the SLAM technology.

Citation:  “UAV Cinematography Constraints Imposed by Visual Target Tracking,” by I. Karakostas, I. Mademlis, N. Nikolaidis and I. Pitas, 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, 2018, pp. 76-80. doi: 10.1109/ICIP.2018.8451385 | URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8451385&isnumber=8451009