New Methodology for Corn Stress Detection Using Remote Sensing and Vegetation Indices

Abstract

Since corn is the second most widespread crop globally and its production has an impact on all industries, from animal husbandry to sweeteners, modern agriculture meets the task of preserving yield quality and detecting corn stress. Application of remote sensing techniques enabled more efficient crop monitoring due to the ability to cover large areas and perform non-destructive and non-invasive measurements. By using vegetation indices, it is possible to effectively measure the status of surface vegetation and detect stress on the field. This study describes the methodology for corn stress detection using red-green-blue (RGB) imagery and vegetation indices. Using the Excess Green vegetation index and calculated vegetation index histogram for healthy crop, corn stress has been effectively detected. The obtained results showed higher than 89% accuracy on both experimental plots, confirming that the proposed methodology can be used for corn stress detection using images acquired only with the RGB sensor. The proposed method does not depend on the sensor used for image acquisition and vegetation index used for stress detection, so it can be used in various different setups.

1. Introduction

One of the biggest challenges facing the global population of 8 billion people is food security and preservation. Over 700 million people go hungry every day and do not have access to the basic foodstuffs needed for their existence [1]. Due to global warming, finding methods and strategies which may increase crop tolerance under different types of stress is of significance [2,3,4]. Furthermore, since arable area is limited and the need for food is growing year by year, the application of more efficient methods of food production is crucial for survival [5,6].

Corn is the second most widespread crop of all and is used in almost all areas and has an impact on all industries, from animal husbandry to sweeteners [7]. Given that corn is not a very tolerant crop to external factors, such as wet weather or temperature, especially in the early stages, and its production has an impact on the global population, one of the major tasks faced by modern agriculture is the preservation of quality yield and the detection of stress in the early stages of cultivation [8].

Starting with global producers such as America and China, which produce more than 50% of corn in the world, the challenge of producing enough food is faced by all other spheres of industrial production, including the final consumers, who either do not have food available or who cannot pay the price dictated by a small supply and high demand. Since the bacteria, viruses, weeds, and animals are the direct cause of yield loss, they are responsible for the loss of between 20 and 40% of the global yield, which in the case of corn production, would be 200–400 million metric tons [9]. It is worth noting that corn irrigation is not a common practice in developing countries; thus, rain-fed corn fields are largely affected by climate changes [10,11]. If the losses are expressed in money, the losses would be between USD 70 and 140 billion.

Stress detection in agriculture is a field that is very popular in recent times and is being encountered by various scientists and engineers [12,13]. The traditional field-based methods for crop monitoring include on-site sampling and laboratory analysis, which have disadvantages in the collection of data, since they are often labor-intensive, costly, time-consuming, and in some cases not possible because of the crop type and stage. By comparison, remote sensing and precision agriculture have found applications in plant health monitoring [14]. Precision agriculture can be used as a specific soil and crop management system that assesses variability in soil and crop properties and parameters using various tools, such as Geographical Information System (GIS), Global Positioning System (GPS), and remote sensing. Application of the remote sensing techniques enabled more efficient crop monitoring due to the ability to cover large areas rapidly and repeatedly and perform non-destructive and non-invasive measurements [15,16].

When it comes to sensors used in remote sampling methods, unmanned aerial vehicles (UAVs) have been commonly used in recent years because they enable high-resolution imaginary and their versatility, light weight, and low costs give them an advantage over other ways of remote sensing data collection. In addition, a large number of aircrafts allow the installation of various sensors that enable data collection by using hyperspectral or near-infrared sensors [17,18]. A common use of remote sensing in agriculture is the evaluation of crop condition based on canopy greenness by using vegetation indices (VIs) [19].

A vegetation index is a combination of surface reflectance at two or more wavelengths designed to highlight a particular property of vegetation [19]. It represents a simple, effective, and empirical measure of the status of surface vegetation [20]. Based on the different sensors mounted on the UAV, it can collect imagery with different spectral bands, such as visible and near-infrared (NIR); different vegetation indices can be calculated and used for studying crop condition. Accordingly, a large amount of previous research is based on the use of NDVI (Normalized Difference Vegetation Index), which is calculated as a ratio difference between measured canopy reflectance in the red and near-infrared bands, respectively [13,21]. In a study by Chen et al., an NDVI sensor and two commercial chlorophyll meters were used to detect the difference in leaf chlorophyll contents of buffaloberry under halogen light and sunlight [22]. Candiago S. et al. demonstrated the great potential of collecting multispectral images and evaluating different VIs, such as NDVI, GNDVI (The Green Normalized Difference Vegetation Index), and SAVI (The Soil-Adjusted Vegetation Index), suggesting that they can be used as a fast, reliable, and cost-effective resource in crop monitoring [23]. Furthermore, along with near-infrared imaging, thermal imaging found application in the detection of plant stress, where Gerhards et al. detected water stress in a potato plant by measuring visible, near-, and shortwave infrared reflectance and fluorescence [24].

In addition to multispectral sensors, which are expensive, RGB (red, green, blue) image sensors, which capture data across three wavelength bands in the visible spectrum, are also used for the evaluation of crop condition. Due to their lower costs and their ability to measure vegetation, vegetation indices based on the RGB sensors found their application in precision agriculture [24,25]. RGB-based vegetation indices such as Excess Green (ExG) can be used to accurately estimate canopy coverage [26]. In their research, Kim D.-W. et al. developed and validated a crop growth estimation model based on UAV RGB imagery for quantifying various biophysical parameters of Chinese cabbage and white radish [27]. They used ExG vegetation index and used the Otsu method for crop extraction from the image.

The importance of corn production stimulated many researchers to write papers related to the application of various machine learning and statistical methods and algorithms to multispectral images in order to analyze spatial variation in various biophysical factors, such as canopy cover, leaf area, and crop height, and predict corn yields along with the preservation of crop health by detecting different types of plant stress [28,29,30]. Given that the use of multispectral sensors provides more information than the standard visual sensors, which collect only red, green, and blue wavelengths, most of the previous studies were related to the use of multispectral and hyperspectral images in order to detect corn stress. In their research, Karimi et al. used the support vector machine (SVM) as a tool for classifying hyperspectral images in order to detect weed and nitrogen stress in corn, while Geol et al. applied artificial neural networks (ANNs) to accomplish stress detection on the same dataset [31,32]. Zhang et al. extracted corn canopy temperature at the late vegetative stage by using RGB and thermal imagery and applied it in water stress monitoring [33].

This paper describes the methodology for corn stress detection using UAV-RGB imagery, which is based on the corn stress detection by comparing vegetation index histograms for each region of interest of the field with the vegetation index histogram of the healthy crop. The image obtained through RGB imagery is divided into regions of interest, where red, green, and blue wavelength data are extracted and the ExG vegetation index histogram is calculated for each region. In the image processing phase, each region is compared with the healthy crop histogram previously calculated from the ground-truth data. By applying histogram overlapping of two histograms, each region is classified into a group depending on the overlap percentage. The methodology can be used with different vegetative indices and also be applied to the multispectral images.

The goal of this study is to propose a methodology for corn stress detection using RGB image and vegetation indices. Based on the challenges we encountered, the contributions of this paper are reflected in the following: (i) application of stress detection using RGB imagery for different types of plant stress—by using ground-truth for the healthy crop it is possible to detect different types of the stress, (ii) application on different crops and using different vegetation indices.

2. Materials and Methods

The methodology for corn stress detection using RGB image is based on the extraction of the ExG vegetation index histogram for each region of interest of the acquired image and its comparison with the vegetation histogram representing a healthy crop. Figure 1 shows the flow chart of the proposed methodology. In the first step of image acquisition, a UAV has been used to fly over experimental fields and collect data using an RGB sensor. Image alignment and georeferencing of the collected images have been applied in the image preprocessing step in order to convert the data into an image orthomosaic. Labelled healthy crop segments are selected and extracted for the next steps. Before the next step, the size of the region of interest, which represents the area on the field covering the maximum corn size and spacing between two crops, is defined. In the vegetation index histogram extraction step, for each pixel in the labelled healthy crop segments, the ExG value is calculated and presented on the histogram representing the standard for a healthy crop and will be used in the processing phase of the methodology. The image processing phase consists of dividing the image orthomosaic into multiple regions of interest and passing through each region to calculate the vegetation index histogram along with the overlap percentage between the selected region histogram and the histogram representing a healthy crop. Before classifying each region of interest into one of the three pre-defined clusters, cluster boundaries are determined by performing hierarchical cluster analysis on one variable using Ward’s method. The last step of the corn stress detection model is classifying each region of interest based on the calculated cluster boundaries and detecting stressed crops on the field.

2.1. Image Acquisition

Two corn fields were used as experimental regions for this study. The first one is located in Păsăreni, Romania with the area of 45.57 ha, with the corn in V4 growing stage planted (Figure 2). At the time of image acquisition and remote sensing, corn size was between 15 and 20 cm, with the crops spaced 20 cm from each other. The second field is located in Ivanovo, Serbia, with a size of 10.32 ha and the same V4 corn planted. At the time of sampling, corn size was between 10 and 20 cm, with a distance between crops of 15–20 cm.

The unmanned aerial vehicle (UAV) platform used in this study was DJI Phantom 4 Advanced (SZ DJI Technology Co., Ltd, Shenzhen, China) equipped with a 1” 20 MPX CMOS sensor and OV 84° 8.8 mm/24 mm lens. For the automatic fly-over of the experimental fields, Drone Deploy Android App 2.9 (Santa Clara, CA, USA) was used and both fields were surveyed at a height of 70 m above the ground. The fly path was generated, and the sequence of overlapped images based on the time-lapse function, which took one image every two and a half seconds, was collected on the flight mission.

In order to label field segments which represent the healthy crops, white tape was used to make the process of healthy crop segment extraction in the next step easier.

2.2. Image Preprocessing and Region of Interest

After multiple images were collected by flying over the fields, Drone Deploy web application (Santa Clara, CA, USA) was used to perform image alignment and georeferencing. After georeferencing the images, they were converted into an image orthomosaic by using image stitching functionality developed by Drone Deploy.

On the obtained georeferenced image orthomosaic, previously labelled healthy crop segments of the field were selected using the drawing tool and extracted for the next steps. For the marked area of the field representing a healthy crop, it is necessary to mark at least 10 segments with the minimum size of the region of interest to be defined.

In order to effectively perform vegetation histogram calculation and stress detection, it is necessary to select the appropriate region of interest (ROI) that represents the area on the field covering the maximum corn size and spacing between two crops. The idea of the proposed methodology is to select the region of interest which will enable the effective recognition of potential stress by measuring the difference in biophysical characteristics of the crop with the use of the vegetation index histogram. As shown in Figure 3, for the region of interest, an area of 70 × 70 cm was used, based on the geometric characteristic that included the maximum size of the crop and the distance between each crop.

2.3. Vegetation Index Histogram Extraction

For the calculation of the vegetation index histogram of the healthy crop, the vegetation index of ExG was used, since it can effectively assess canopy variation in green crop biomass based on the RGB images. The ExG was calculated using the calibrated RGB reflectance values (Equation (1)) [34]:

$$ExG=2g-r-b,$$

(1)

where r, g, and b range from 0 to 1 and were calculated as follows:

$$r=\frac{R}{R+G+B} g=\frac{G}{R+G+B} b=\frac{B}{R+G+B}$$

(2)

where R, G, and B represent reflectance values of R, G, and B bands in the original image acquired by the UAV-RGB sensor [35].

Given that the methodology is based on the comparison of vegetation index histograms and the determination of their percentage of overlap, in the first step it is necessary to calculate the histogram of a healthy crop based on the segments labelled as a healthy crop in the image acquisition phase by the agriculture expert. For each pixel in the selected segments, the ExG value is calculated and the values are presented on a histogram. In order to create a histogram which can be compared with the region of interest, the healthy crop histogram has was normalized by dividing by the total number of pixels, and the values on the y-axis were presented in relative frequencies, as in Figure 4.

Calculating the histogram in this way represents the standard for a healthy crop, and in the processing phase of the methodology, it will be compared with all regions of interest in the entire field.

2.4. Image Processing and ROI Classification

In the image processing phase, the image orthomosaic of the field is divided into multiple regions of interest previously defined. Passing through each ROI, the vegetation index histogram is calculated as mentioned in the previous section and then compared with the histogram representing a healthy crop, as calculated in the previous step.

In order to calculate the overlap percentage of two histograms, the histogram intersection algorithm was used [36]. The calculation of the overlap percentage was performed using the following formula (Equation (3)):

$${\displaystyle \sum }_{j=1}^n\min \left[I_j,M_j\right]$$

(3)

where I_j represents the histogram value of the ROI for the j-th bin, and M_j represents the histogram value of a heathy crop for the j-th bin. Both histograms ranged from 0 to 1 in steps of 0.01. In this way, the cumulative difference in relative frequencies between the two histograms is calculated. In order to compute the overlap percentage, the cumulative difference is divided by the number of relative frequencies of the M histogram of the healthy crop, as in Equation (4):

$$\frac{{{\displaystyle \sum }}_{j=1}^n\min \left[I_j,M_j\right]}{{{\displaystyle \sum }}_{j=1}^n{M}_j}$$

(4)

Figure 5 shows a graphic representation of two examples of overlapping histograms. In both histograms, the histogram M representing a healthy crop is colored in green, while the I histogram of the ROI is colored in red. In the left example, the percentage of overlap is 3.31%, while on the right, the percentage of overlap is 68.89%.

After the overlap percentages for all ROIs and the histogram of the healthy crop have been calculated, the next step is to perform their classification and detect regions of interest with corn stress. Each ROI will be classified in one of the three pre-defined clusters: healthy crop, potential stress, and plant stress. In order to determine cluster boundaries for each of the clusters, hierarchical cluster analysis (HCA) on one variable using Ward’s method was performed [37]. When the cluster boundaries were defined, each ROI was classified into a cluster depending on the calculated percentage of overlap. ROIs with the lowest overlap percentage were classified into a plant stress cluster, while the ROIs with the highest overlap percentage were classified into a healthy plant cluster. Finally, in order to enable visual representation of the obtained results, a corn stress map was generated using ArcGIS 10.0.2 (Esri, Redlands, CA, USA), where each ROI contains georeferenced data and is marked with the appropriate cluster color.

3. Results

The data collected by the UAV-RGB sensor were uploaded into the Drone Deploy software package and georeferenced. After applying image stitching functionality, an image orthomosaic was obtained for both test corn V4 fields. Before flying over the fields, field segments representing healthy crops were marked and will be used in the next step for vegetation index calculation. Experimental field data as well as the number of segments labelled as healthy crops and their minimum area are provided in

Table 1. Experimental field data including total field area, number of labelled segments representing healthy crop along with the total labelled area, and minimum segment size sampled.

Test Field	Area	Labelled Segments	Total Labelled Area	Min. Segment Size
Păsăreni (Romania)	45.57 ha	8	245 m2	20 m2
Ivanovo (Serbia)	10.32 ha	11	261 m2	18 m2

.

The segments representing the healthy crop were labeled using Drone Deploy software and extracted so that they could be used for the next phase of histogram calculation. For each test field, a histogram of the ExG vegetation index was calculated and then standardized, as explained in the previous section, in order to display relative frequencies on the y-axis. Calculated healthy crop histograms for both test fields are shown in Figure 6.

On the obtained histograms of a healthy crop, 95% of the values for the Păsăreni field are located between 0.17 and 0.51, with a median value of 0.29. On the other hand, a lower median of 0.11 was obtained for the Ivanovo field, where 95% of the values are located between 0.01 and 0.20. The lower values of the ExG index for the Ivanovo field can be influenced by the brighter image, because during the flyover period, the sun fell directly on the ground, and there was also a slightly larger distance between crops.

In the next step of the methodology for corn stress detection, the complete image was divided into regions of interest. For the ROI size, an area of 70 × 70 cm was selected as mentioned in the previous section, in order to cover the maximum width of the crop and the distance between each crop. Passing through each ROI, the vegetation index histogram was calculated. For each ROI histogram, the overlap percentage with the healthy crop histogram was calculated using the histogram intersection algorithm.

Hierarchical cluster analysis on one variable using Ward’s method was performed in order to obtain overlap percentage boundaries for each of the three pre-defined clusters: healthy plant, potential stress, and plant stress. The potential stress cluster is defined as a group where corn stress cannot be confirmed, but the region of interest is not a healthy crop. The results of the application of hierarchical cluster analysis with the obtained cluster boundaries are shown in

Table 2. Results of hierarchical cluster analysis (HCA) using Ward’s method, displaying cluster ranges in percentage of histogram overlap for each test field.

Cluster Range	Healthy Plant	Potential Stress	Plant Stress
Păsăreni	(64.32–100]	(32.78–64.32]	[0–32.78)
Ivanovo	(59.46–100]	(23.54–59.46]	[0–23.54)

. Based on the cluster boundaries, all ROIs were classified into a cluster depending on the percentage of overlap.

Georeferenced data for each ROI were obtained by taking the center point for each 70 × 70 cm segment, and a georeferenced stress map was generated using the ArcGIS 10.0.2 software package. The stress map was generated in order to visually represent the ROIs marked as potential stress or plant stress (Figure 7), where the green color represents the healthy plant, the yellow color represents potential stress, and plant stress is displayed with the red color. In this way, it is possible to combine georeferenced data with other methods of precision agriculture in order to monitor plant health and apply necessary measures on the selected region of interest in order to preserve the crops.

Validation of the presented methodology for stress detection in corn was performed using isolated segments that represent a healthy crop and were used to calculate the healthy crop histogram, as well using field segments that represent the ground-truth of a stressed plant. Samples with a total of 192 ROIs were allocated in the Păsăreni field, while in the Ivanovo field, 112 ROIs were allocated. In order to validate the proposed method, a confusion matrix was calculated where ground-truth data were used as the sample. The confusion matrix is displayed in

Table 3. Results of confusion matrix for the two test fields, presenting difference between actual and predicted ROI classes.

			Predicted
Test Field			Healthy Plant	Potential Stress	Plant Stress	Total	Error %
Păsăreni	Actual	Healthy Plant	834	49	22	905	7.84
		Plant Stress	2	12	178	192	7.29
Ivanovo	Actual	Healthy Plant	739	54	31	824	10.31
		Plant Stress	1	10	101	112	9.82

.

As shown in Table 3, when it comes to validating the ground-truth, the data used for the healthy crop gave good results, with an accuracy of 92.16% and 89.69% for both fields, respectively. For the ground-truth data representing stressed crops, the accuracy percentage is slightly higher, so the model error is 7.29% for the Păsăreni field and 9.82% for the Ivanovo field. The precision percentage of the model is 89%, with a recall of 92.71%, for Păsăreni, and the precision percentage is 76.52% along with a recall of 90.18% for the Ivanovo field. Within the misclassified ROIs, most of them are classified as potential stress, which tells us that this cluster needs to be additionally processed in detail in order to reduce the model error [38].

4. Discussion

Vegetation indices are an effective way to separate crops from the background, that is, to identify all differences in soil that can be characterized as stress. When it comes to images collected using only RGB bands, Excess Green (ExG) gives excellent results for healthy crop identification. In a study by Kim et al., ExG was applied for automatic crop segmentation with the Otsu threshold [27]. In our study, a histogram of vegetation index ExG was calculated based on the defined region of interest covering the maximum width of the corn in V4 phase and the spacing between two crops. By applying the defined methodology, each ROI was classified into one of the previously defined groups. Based on the confusion matrix used for validation of the results, the model showed errors ranging from 7.84% to 10.31% when it comes to cross-validation of ground-truth data of a healthy crop. For the ground-truth data of a stressed crop, the error ranged from 7.29% to 9.82%, which implies that the applied methodology gives acceptable results, with >90% success. Kim et al. also obtained significant results with >80% success with their method [27]. The downside of this methodology is that a large number of ROIs are classified into the potential stress cluster, which represents the sensitivity zone of this method, and thus these data should be further processed in the future in order to increase the efficiency of the given methodology.

In previous research related to the stress detection in crops and corn itself, most researchers have used vegetation indices calculated using multispectral and hyperspectral sensors with a big application of thermal cameras, which provide much more information than an image collected using only the visible RGB band [33,39,40]. In our research, UAV-RGB imaging was used, as it does not require expensive equipment and can be used with different vegetation indices. Furthermore, the presented methodology is applicable to different types of corn stress, considering that the histogram of a healthy crop is generated based on the ground-truth data, while in most research in this field, the focus is on a certain type of corn stress, such as water, nitrogen drought, or weed stress [41,42].

5. Conclusions

This paper presents a methodology for corn stress detection based on a UAV-RGB image obtained by remote sensing, as corn stress represents one of the biggest challenges faced by most farmers. Using the ExG vegetation index, the histogram of a healthy crop was calculated based on the marked segments from the field. The calculated histogram of a healthy crop was then compared with vegetation index histograms of all defined regions of interest on the field, and the percentage of overlap was calculated. Based on the percentage of overlap, all regions of interest were classified into one of the three pre-defined groups in order to detect corn stress. The advantage of this methodology is reflected in the fact that it is not dependent on the sensor used for image acquisition and can be applied with multispectral sensors in order to use different vegetation indices for stress detection. Moreover, the methodology detects general stress in crops, so it can be applied on different types of stress in corn or another crops. However, the drawback of this method is the potential stress cluster, where regions of interest classified in this group should be further analyzed in detail. Future research should include additional analysis of the potential stress cluster, as well as the application of this methodology on different plant cultures and stages of growth. Additionally, the application of different vegetation indices can be used in order to compare the results and define which vegetation index gives the best results.