Classification of crops and weeds extracted by active shape models

Active shape models (ASMs) for the extraction and classification of crops using real field images were investigated. Three sets of images of crop rows with sugar beet plants around the first true leaf stage were used. The data sets contained 276, 322 and 534 samples, equally distributed over crops and weeds. The weed populations varied between the data sets resulting in from 19% to 53% of occluded crops. Three ASMs were constructed using different training images and different description levels. The models managed to correctly extract up to 83% of the crop pixels and remove up to 83% of the occluding weed pixels. Classification features were calculated from the shapes of extracted crops and weeds and presented to a k-NN classifier. The classification results for the ASM-extracted plants were compared to classification results for manually extracted plants. It was judged that 81–87% of all plants extracted by ASM were classified correctly. This corresponded with 85–92% for manually extracted plants.

Introduction

Conventional farming today uses herbicides to control weeds, while organic farming depends on alternative methods such as flaming, steaming and cultivation. Most of these alternative methods treat only the area between the crop rows, leaving the area within the rows requiring hand weeding. To reduce chemical use and the time and cost of hand weeding, alternative approaches to control weeds must be developed.

One approach is to use computer-based signal processing and sensor technology such as computer vision to recognise and localise crops and weeds (Åstrand, 2005). The system can control either precision sprayers to apply herbicide only to the weeds, or guide mechanical weeding tools to physically remove the weeds. In an on-line system, such as used with an autonomous robot, the vision system needs to automatically determine where the crops and the weeds are. A classifier can determine the plant type by examining the shape and colour of features extracted from the plant.

The major problems that can occur when using computer vision are occlusion and variations in plant appearance. Occlusion is where leaves from crops and weeds partially overlap, resulting in two or more plants in the image being interpreted as one plant (Fig. 1).

The variations in plant appearance are variations between different samples of the same plant species. The leaves do not face the same direction and are different in size. Furthermore, variations in the soil conditions and the amount of sunlight can result in colour variations and varying differences in size between crops and weeds.

Moreover, when the image is segmented into plants and soil, the resulting plant objects can be single leaf objects or multiple leaf objects (Fig. 2). The multi-leaf objects can contain leaves from the same plant, but it is also very likely that leaves from different plants are included (occlusion). When the objects are merged into plants it can be difficult to discern which objects contain leaves belonging to the same plant.

Extraction of the plants can be carried out in different ways and the choice of the extraction method depends on the application and the plant species. Woebbecke et al. (1995) used morphological shape features to identify plants from different weed species. Onyango and Marchant (2003) used colour and the planting grid to extract cauliflower plants from weeds and background. They found that for cauliflower 82–96% of pixels were classified correctly and for weeds this range was 68–92%. The planting grid was also used in Åstrand (2005) where the plant material in colour images of sugar beets was extracted from the soil using a linear discriminant in the normalised RGB (red, green, blue) colour space. To distinguish between sugar beets and weeds, features including colour, area and shape were calculated from the extracted plants and combined with a model of the distance between the sown crops. The total classification rate was 90–98% when occluded plants were excluded with the tested data sets containing 5–24% occluded plants. Alchanatis et al. (2005) extracted plant material from the soil using the difference in multi-spectral images at 660 nm and 800 nm. The cotton plant was extracted from the weed using the local inhomogenity of the pixel values. For cotton 86% of the pixels were classified correctly and the same was true for the weed pixels. Tang et al. (2000) used a genetic algorithm to find colour boundaries for extracting plant material from soil in the HIS (hue, intensity, saturation) colour space. The algorithm did not separate crop pixels from weed pixels. Approximately 90% of the plant and background pixels were classified correctly. These methods include colour or spectral information but when colour or spectral information is used for plant extraction the difficulty is that conditions may change from one field to another depending on crop health and soil conditions.

Instead of looking at its colour, the size-independent shape of the plant can be used. Franz et al. (1991) used Fourier descriptors to describe the leaf contour of soybean, velvet leaf, ivy-leaf morning glory and giant foxtail. Depending on the amount of occlusion, 74.5–100% of the plants were correctly identified. Mahn et al. (2001) used a deformable template based on the parametric formula of a skeleton and envelope functions. The image was searched for pointy leaf tips; for each leaf tip, the model was placed over it and deformed to match the whole leaf. Out of 600 green foxtail leaves the model extracted 84% correctly. These methods analysed only one leaf at a time; if the plant consisted of several leaves it was necessary to combine them afterwards. Instead of analysing the leaves separately, Søgaard (2005) considered the whole plant by using active shape models (ASMs) (Cootes and Taylor, 1992; Cootes et al., 1995, Cootes et al., 1994), to extract and distinguish between different weed species. Three weed species were tested using three different models, one model per species. Each plant was tested with all three models, and the model that had the best fit was assumed to be the correct one. The classification rates for the three species were 77%, 65% and 93%. The plants in the test were seedlings with up to two true leaves and no mutual overlapping.