Examples of UAV detection in outdoor tests.
Currently, the use of multiple UAVs in drone swarms is garnering huge interest from the research community, leading to the exploration of topics such as UAV cooperation, multi-drone autonomous navigation, etc.
Researchers have been working on UAV pursuit-evasion based on two probable approaches. In this exciting new study researchers propose the use of vision-based deep learning object detection and reinforcement learning for detecting and tracking a UAV (target or leader) by another UAV (tracker or follower). The proposed framework uses vision data captured by a UAV and deep learning to detect and follow another UAV. The algorithm is divided into two parts, the detection of the target UAV and the control of UAV navigation (follower UAV). The deep reinforcement learning approach uses a deep convolutional neural network (CNN) to extract the target pose based on the previous pose and the current frame.
Proposed deep unmanned aerial vehicle (UAV) tracking framework.
The network works like a Q-learning algorithm. The output is a probabilistic distribution between a set of possible actions such as translation, resizing (e.g., when a target is moving away), or stopping. For each frame from the captured sequence, the algorithm iterates over its predictions until the network predicts a “stop” when the target is within the desired position.
Deep reinforcement learning architecture overview. Enclosed in the box is the core convolutional neural network (CNN) based on VGG-M. Dashed arrows depict state transitions.
The second approach uses deep learning object detection for the detection and tracking of a UAV in a pursuit-evasion scenario. The deep object detection approach uses images captured by a UAV to detect and follow another UAV. This approach uses historical detection data from a set of image sequences and inputs this data to a SAP algorithm in order to locate the area with a high probability UAV presence.
Ground-truth- (left) and Gaussian-noise-generated boxes (right).
The proposed framework uses images captured by a UAV and a deep learning network to detect and follow another UAV in a pursuit-evasion scenario. The position of the detected target UAV (detected bounding box) is sent to a high-level controller that decides on the controls to send to the follower UAV to keep the target close to the centre of its image frame.
Deep object detector with a search area proposal. Image sequence and detection history are used to predict the area of interest (search area).
In this work, researchers aim to develop architecture capable of tracking moving targets using predictions over time from a sequence of previously captured frames. The proposed algorithm tracks the target and moves a bounding box according to each movement prediction. The architecture is based on an ADNet (action-decision network).
Example of a prediction given by the search area proposal. The particles with higher weights (in red) are used to detect the area of interest.
Conclusions:
This work presents two approaches for UAV pursuit-evasion. The obtained results show that the proposed approach is effective in tracking moving objects in complex outdoor scenarios.
SAP improved the detection of distant UAVs. The obtained results demonstrate the efficiency of the proposed algorithms and show that both approaches are interesting for a UAV pursuit-evasion scenario.
Future developments include improving the processing time of the reinforcement learning algorithm by adding a confidence score to predictions and a scale factor to the generated actions. This scale factor will help reduce the number of actions by improving the predicted distance used to move the follower UAV close to the target. Researchers suggest that for the SAP algorithm, an end-to-end approach will be studied to integrate predictions within the deep learning architecture and improve the processing time. Combining deep object detection, SAP, and reinforcement learning will also be considered. Finally, the team says it could be interesting to evaluate the performance of other deep object detectors within the proposed framework.
Citation: Drones Chasing Drones: Reinforcement Learning and Deep Search Area Proposal, Moulay A. Akhloufi, Sebastien Arola and Alexandre Bonnet, Perception, Robotics, and Intelligent Machines (PRIME), Department of Computer Science, Université de Moncton, Canada and UPSSITECH, Toulouse, France, Drones 2019, 3(3), 58; https://doi.org/10.3390/drones3030058, https://www.mdpi.com/2504-446X/3/3/58/htm