Original papersAn in-field automatic wheat disease diagnosis system
Introduction
Crop disease diagnosis is of great significance to prevent the spread of diseases and maintain the sustainable development of agricultural economy. In general, the crop disease diagnosis is performed manually by visual observation or microscope techniques, which are proven to be time-consuming and have the risk of error due to subjective perception. In this context, various spectroscopic and imaging techniques have been studied for identifying crop disease symptoms (Bravo et al., 2004, Moshou et al., 2005, Chaerle et al., 2007, Belasque et al., 2008, Qin et al., 2009). Although these techniques can make a relatively rapid diagnosis for crop diseases, they can’t be separated from the support of expensive and bulky sensors.
With the development of computer vision, concerns are growing about the image-based detection technologies for crop disease (Camargo and Smith, 2009, Arivazhagan et al., 2013, Barbedo, 2014, Rastogi et al., 2015), which gets rid of the shackles of time cost and molecular analysis equipments (Martinelli et al., 2015). Instead, people just need common cameras and consumer-level electronic storage devices to perform crop disease identification. However, these methods are almost task-specific and need expert knowledge to design handcrafted feature extractors, only working for the crop images under ideal experimental environment. To make an automatic crop disease diagnosis system that can be applied to in-field images, one has to face some intractable challenges summarized in Barbedo (2016): (1) complex image backgrounds, e.g. leaves, soils, stones and even the people’s hands, (2) uncontrollable capture conditions, e.g. illumination, camera angle and image quality, (3) co-occurrence of multiple leaves or multiple disease areas in one image, (4) various characterization for different stages of disease development, (5) similarities in appearance between different disease categories. Some challenging samples, which are selected from our collected dataset of wheat diseases, are demonstrated in Fig. 1. To our best knowledge, few researches have been done to mitigate or eliminate the above challenges for crop disease diagnosis.
The overall objective of this work is to develop an automatic wheat disease diagnosis system to identify disease categories and locate corresponding disease areas simultaneously for in-field wheat images. In order to avoid the expensive and laborious manual annotation, the twofold task of identification and localization for wheat diseases is modeled as a weakly supervised learning task. As far as we know, we firstly propose to jointly handle the twofold task for wheat diseases in an in-field scenario.
In this paper, an novel in-field automatic wheat disease diagnosis system is proposed based on deep learning and multiple instance learning (MIL) (Dietterich et al., 1997), which can be deployed on mobile handsets to perform real-time diagnosis. As showed in Fig. 2, our framework is composed of mobile client and computing sever in device level. On the one hand, a fully convolutional network (FCN) is exploited to perform local feature extraction and local disease estimation for a resized image captured from mobile camera in wild conditions. Consequently, the FCN produces some spatial score maps over different disease species where each score point corresponds to a particular respective local window of raw image. Then these estimations of different local windows are fed into MIL framework to aggregate the overall evaluation for the whole image. On the other hand, the spatial score maps go through an up-sampling operation to roughly delineate the disease location, then a bounding boxes approximation (BBA) step is performed to accurately lock disease positions. The proposed model is named multiple instance learning based wheat disease diagnosis system (DMIL-WDDS). In particular, the DMIL-WDDS can achieve end-to-end training with enough prepared training data. To verify the practicability and effectiveness of our DMIL-WDDS, an in-field disease dataset Wheat Disease Database 2017 (WDD2017) is collected, which consists of 9,230 images with 7 different classes (6 for common wheat diseases, 1 for healthy wheat). The results of experiments on WDD2017 show that the proposed DMIL-WDDS outperforms conventional CNN architectures on recognition accuracy for disease categories, but also keeping precise localization for corresponding disease areas.
Our main contributions are summarized as:
- •
A weakly supervised learning framework based on DMIL is firstly exploited for wheat disease diagnosis, which has the ability to cope with intractable in-field wheat images.
- •
An integration of identification and localization for wheat diseases is achieved by proposed system, which outperforms conventional CNN-based recognition architecture under the same amount of parameters in deep model.
- •
A new in-field wheat disease dataset WDD2017 is collected to demonstrate the effectiveness of proposed system as well as building a benchmark for subsequent works.
Section snippets
Related work
Many image-based methods have been developed to handle crop disease identification. Based on the fusion of hyper-spectral and multi-spectral fluorescence measurements, Moshou et al. (2005) presented a self-organizing map (SOM) based disease classifier. Camargo and Smith (2009) proposed to identify the visual symptoms of crop diseases via color transformation and image color segmentation. In Phadikar et al. (2013), Fermi energy based segmentation method was proposed to isolate infected region
Wheat Disease Database 2017 (WDD2017)
To facilitate the researches of crop disease diagnosis, some relevant datasets have been released openly and freely, e.g. PlantVillage, which includes over 50,000 expertly annotated images on healthy and infected leaves of crops. However, it is a pity that all images in PlantVillage have been processed into an ideal state which is hard to be seen in wild conditions, namely one crop leaf in a pure background. To our best knowledge, no suitable dataset for in-field crop disease diagnosis is
Experiments
We verify the effectiveness of proposed DMIL-WDDS upon WDD2017 by performing some experiments in which several models are built to estimate the test accuracy and show the localization results. Below we give details of our experiments.
Wheat disease identification
Table 3 presents the final class-wise accuracy and total accuracy over 5-fold cross-validation for different deep model configurations. To compare our DMIL-WDDS framework with conventional CNN architectures, we firstly observe the results of VGG-CNN-S and VGG-FCN-S. Note that the VGG-FCN-S with three aggregated functions (Max-agg, Avg-agg and Soft-agg) significantly and consistently outperforms VGG-CNN-S on all classes, improving the total accuracy by 17.34%, 19.86% and 22.12% respectively.
Conclusion
In this paper we propose an novel wheat disease diagnosis framework based on deep multiple instance learning, namely DMIL-WDDS, which aims to deal with in-field wheat images without any technical preprocessing. Only with the supply of image-level labelled training data, our system realizes an integration of identification for wheat diseases and localization for disease areas. We exploit four different deep models to perform wheat disease recognition on the newly collected in-field dataset
Acknowledgments
The authors would like to thank the reviewers for their valuable suggestions on improving this paper. This work is funded by NSFC (Grant No. 61473167 and No. 61621136008) and the German Research Foundation (DFG) in Project Crossmodal Learning, DFC TRR-169.
References (33)
A review on the main challenges in automatic plant disease identification based on visible range images
Biosyst. Eng.
(2016)- et al.
An image-processing based algorithm to automatically identify plant disease visual symptoms
Biosyst. Eng.
(2009) - et al.
Multicolor fluorescence imaging for early detection of the hypersensitive reaction to tobacco mosaic virus
J. Plant Physiol.
(2007) - et al.
Solving the multiple instance problem with axis-parallel rectangles
Artif. Intell.
(1997) - et al.
Plant species classification using deep convolutional neural network
Biosyst. Eng.
(2016) - et al.
Deep learning for plant identification using vein morphological patterns
Comput. Electron. Agric.
(2016) - et al.
Plant disease detection based on data fusion of hyper-spectral and multi-spectral fluorescence imaging using kohonen maps
Real-Time Imag.
(2005) - et al.
Rice diseases classification using feature selection and rule generation techniques
Comput. Electron. Agric.
(2013) - et al.
Detection of citrus canker using hyperspectral reflectance imaging with spectral information divergence
J. Food Eng.
(2009) - et al.
An adaptive thresholding algorithm of field leaf image
Comput. Electron. Agric.
(2013)
Automatic citrus canker detection from leaf images captured in field
Pattern Recogn. Lett.
Robotized plant probing: leaf segmentation utilizing time-of-flight data
IEEE Robot. Automat. Mag.
Detection of unhealthy region of plant leaves and classification of plant leaf diseases using texture features
Agric. Eng. Int.: CIGR J.
An automatic method to detect and measure leaf disease symptoms using digital image processing
Plant Dis.
Detection of mechanical and disease stresses in citrus plants by fluorescence spectroscopy
Appl. Opt.
Cited by (281)
Detection of Fusarium head blight in wheat using UAV remote sensing based on parallel channel space attention
2024, Computers and Electronics in AgricultureLabel-efficient learning in agriculture: A comprehensive review
2023, Computers and Electronics in AgricultureIntegrated digital image processing techniques and deep learning approaches for wheat stripe rust disease detection and grading
2023, Decision Analytics JournalLeaf disease detection using machine learning and deep learning: Review and challenges
2023, Applied Soft ComputingIdentification of tomato leaf diseases based on LMBRNet
2023, Engineering Applications of Artificial Intelligence