A tiny drone that can fit in the palm of your hand has been created by researchers at ETH Zurich, Switzerland and the University of Bologna, Italy.
This in itself may not seem astonishing, as there are many ‘nano-drones’ already out there on the market, but this one has some special capabilities – artificial intelligence.
We’ve seen AI and machine learning employed in many larger commercial and industrial drones, with companies such as Microsoft and Google building software systems to create autonomous drones that can perform complex tasks tasks on a large scale such as object detection and recognition.
Such computing and imaging technology requires a great deal of power, and fitting this kind of high end technology into a small device no bigger than your hand is another thing altogether.
As the researchers stated in a paper published last week, “we provide the first (to the best of our knowledge) completely integrated hardware/software solution for autonomous navigation of nano-UAVs with completely on-board computation.”
Flying in dynamic, urban, highly-populated environments represents an open problem in robotics. State-of-the-art (SoA) autonomous Unmanned Aerial Vehicles (UAVs) employ advanced computer vision techniques based on computationally expensive algorithms, such as Simultaneous Localization and Mapping (SLAM) or Convolutional Neural Networks (CNNs) to navigate in such environments,” they stated.
With the development of the Internet of Things, these researchers consider the implementation of AI technology into nano-drones could be quite useful as mobile autonomous nodes. However considerations of maneuverability and navigation come into consideration in the context of urban environments, meaning adequate imaging and sensing technology for safe autonomous flight is imperative.
“However, autonomous flight is considered unaffordable in the context of nano-scale UAVs, where the ultra-constrained power envelopes of tiny rotor-crafts limit the on-board computational capabilities to low-power microcontrollers,” they say.
To address this issue, they’ve come up with a low-power solution that consumes a miniscule 94 milliWatts (0.094 W) of energy, built onto a 27g commercially available CrazyFlie 2.0 nano-drone. Using a GreenWaves GAP8 SoC embedded processor derived from the PULP (Parallel Ultra Low Power) open source project, the tiny drone can fly safely at speeds of 4 metres per second.
The AI technology that makes this possible is the DroNet, developed at ETH initially to improve efficiency of search and rescue processes. This convolutional neural network (CNN), designed as an efficient 8-layers residual network, producing two output images for each single input – a steering angle to navigate the drone while avoiding obstacles, and a collision probability help the drone detect risky situations and react accordingly.
While DroNet commonly processes images at 20fps, the researchers showed that by combining the GAP8 processor as an accelerator with an STM32 microcontroller, real-time computation of DroNet could be achieved with an average power cost of 94mW.
“Overall, our contribution paves the way for a huge number of advanced use-cases of autonomous nano-UAVs as IoT-connected mobile smart sensors,” the researchers said.
Speaking with The Register, co-author Antonio Loquercio said this is the first time a drone of this size has been controlled without the need for external sensing and computing.
“Computation is fully on-board, from state-estimation to navigation controls. This means, nano-drones are completely autonomous,” he said.
The power of DroNet was recently demonstrated in a Youtbe video uploaded by ICRA2018 (International Conference on Robotics and Automation), which begins next week in Brisbane, Australia.
In the abstract for the conference, the researchers state that they “propose to train a UAV from data collected by cars and bicycles, which, already integrated into urban environments, would expose other cars and pedestrians to no danger. Although trained on city streets, from the viewpoint of urban vehicles, the navigation policy learned by DroNet is highly generalizable.”
What will the future bring for autonomous nano-drones? Could we be seeing autonomous technology being applied to other pint-sized drones such as the Robofly or Walmart’s idea for farm-tending robot bee drones? For now, researchers say they intend to focus on more field tests and refining the system for better overall performance and reduced power consumption.