Drones and Neural Networking Used to Identify Citrus Trees from Above

Nowadays, remote sensing is one of the processes used in precision agriculture. The use of drones or unmanned aerial vehicles (UAVs) in this field is revolutionizing precision agriculture workflows for measurement of crop conditions as well as yields over the growing season. Aside from this, practices in the monitoring of trees for growth, individual fruit production, monitoring the weeds and others are found to be very useful for long-term farm management.

In this manner, a group of authors has published a paper in which they are detecting citrus and other crop trees from UAV imagery using a simple convolutional neural network (CNN) algorithm which is also refined using superpixels derived from a Simple Linear Iterative Clustering (SLIC) algorithm.

An Increasing Number of Algorithms Used for Feature Extraction or Segmentation from High-Resolution Imagery

As the authors note in the beginning, they are fully aware of the increased number of algorithms used for feature extraction or segmentation from high resolution imagery. They further note that the advent and use of UAVs in precision agriculture represents a milestone in the acquisition and analysis of high resolution imagery on demand.

As they stated in the paper:

“UAVs represent a low-cost method for image acquisition with successful and promising usage in tree identification analysis, particularly in agricultural settings. For example, Malek, et al. [31] identified palm trees from UAV images using an extreme learning machine classifier. Jiang, et al. [32] proposed an improved scale-space filtering algorithm in a GPU-environment for papaya tree detection, achieving high accuracy with an increase in the speed of computation.”

More importantly, deep learning (DL) algorithms are used more and more in remote sensing applications. As a subset of machine learning (ML) and artificial intelligence (AI), deep learning includes a variety of methods that automatically learn features from the large amounts of data, performing well with large number of training samples as well.

Study area location near Visalia, California (left) and a false color image as acquired by the UAV with a spatial resolution of 0.12 m and covering 64.6 ha (right).

Examples of green (a), red (b), red edge (c) and near infrared (d) bands as acquired by the senseFly eBee fixed-wing UAS using a Parrot Sequoia multi-spectral camera.

Example of training sets used. Each image is an example of one 40 × 40 pixel training for: (a–e) trees, (f–j) bare ground and (k–o) weeds.

Probability heatmap of tree presence resulted from CNN with values between 0 and 1 (black to white) (a). To reduce the effects of possible noise, the heatmap was smoothed (b) and trees were identified using a threshold of 0.5 (c). The subset depicted here demonstrates the problem of multiple crown detection for large trees (d) that was solved by iteratively segmenting the smoothed heatmap and NDVI layer using SLIC superpixels of sizes larger that 40×40 pixels (e). Multiple crown detection were identified (f) and a single hit per tree was recomputed (g). The results after the refinement (h) better matched the ground reference samples (i).

Using CNNs for the Segmentation of High Resolution Imagery

However, the authors use convolutional neural networks (CNNs) for the segmentation of high resolution imagery – as a hierarchical architecture that can be trained on large-scale datasets in order to perform object recognition and detection.

The results of their study showed that in many cases, multiple crowns were detected within single large trees. The accuracy of the results was also improved significantly, achieving great scores and showing the importance of UAV imagery collection in this field.

“UAV imagery collection for agricultural applications is increasing globally and more of these individual cases are needed to develop more standard workflows that will help field and research managers deal with large volumes of high resolution imagery.”

Even though their work highlighted one case study using a simple convolutional neural network, the authors also state that “deeper CNNs with multiple hidden layers or ability to regionalize data should be tested to understand improvements” and that some of them will be proven to be fast and efficient in detecting objects.

Individual case studies are critical in the new and growing field and integration of multiple missions over larger study areas and with different targets – providing a framework for larger projects.

Final tree detection from the test area with white crosses indicating the location of trees: (a) the southern portion of LREC; (b) medium size trees correctly classified; (c) large canopy trees with reduced effect of multiple crown detection, after the object-based refinement; (d) similar sized trees were correctly classified; and (e) heterogeneous tree canopies sizes were correctly classified

A Final Word

As the authors state, this is the first time a CNN has been used with UAV multi-spectral imagery to focus on citrus trees and more of these individual cases are needed in order to develop standard automated workflows.

The general conclusion reads the following:

“Methods to delineate, enumerate and monitor individual trees in an agricultural setting from high resolution optical imagery are required for efficient and precise crop management. Monitoring of individual trees for growth, fruit production and pest and disease occurrence remains a high research and operational priority and the delineation of each tree using automated methods as an alternative to manual delineation will be useful for future long-term crop management.”

This way, the managers in the field of agriculture can incorporate large volumes of high resolution UAV imagery into agricultural management operations.

Citation: “Identification of Citrus Trees from Unmanned Aerial Vehicle Imagery Using Convolutional Neural Networks”, Ovidiu Csillik, John Cherbini, Robert Johnson, Andy Lyons and Maggi Kelly – Drones 2018, 2(4), 39; https://doi.org/10.3390/drones2040039