Semantic region of interest and species classification in the deep neural network feature domain

Highlights

• Detecting and classifying of semantic animal region in camera-trap images.

• Generating candidate animal regions by k-means and graph cut semantic segmentation.

• Accurate classification is performed by semantic region.

Abstract

In this paper, we focus on animal object detection and species classification in camera-trap images collected in highly cluttered natural scenes. Using a deep neural network (DNN) model training for animal- background image classification, we analyze the input camera-trap images to generate a multi-level visual representation of the input image. We detect semantic regions of interest for animals from this representation using k-mean clustering and graph cut in the DNN feature domain. These animal regions are then classified into animal species using multi-class deep neural network model. According the experimental results, our method achieves 99.75% accuracy for classifying animals and background and 90.89% accuracy for classifying 26 animal species on the Snapshot Serengeti dataset, outperforming existing image classification methods.

Introduction

Wildlife monitoring with camera-traps allows us to collect data at large scales in space and time to study the impact of climate changes, land-use, and human actions on wildlife population dynamics, and biodiversity (Kays et al., 2011). Camera-traps are stationary camera-sensor systems attached to trees in the field. Triggered by animal motion with on-board infrared motion sensors, they capture short image sequences of the animal appearance and activities associated with other sensor data, such as light level, moisture, temperature, and GPS sensor data. Camera traps are now being extensively used in wildlife monitoring due to their relatively low cost, rapid deployment, and easy maintenance (He et al., 2016). From camera-trap images, we can extract useful information such as animal species, motion, appearance, and biometric features (Kays et al., 2014). They are widely used in ecological research to track animal movements (Silveira et al., 2003), study habitat use (Bowkett et al., 2008), assess species behaviors and population dynamics (Karanth et al., 2006), and identify new species (Rovero and Menegon, 2008).

In this work, we focus on automatic animal species recognition in highly cluttered camera-trap images using deep learning methods. Fig. 1 shows some samples of camera-trap images with animals often appearing in a relatively small region of the highly cluttered wooded scene. We recognize that it is not efficient to detect and classify animal species directly on the whole image using image classification or object detection approaches. Instead, it is highly desirable to first locate the animal region in the large image and then perform animal species classification on this smaller image region. In this way, we expect that the animal classification will be more accurate and robust since the interference from the cluttered background is suppressed.

Deep neural networks (DNN) have emerged as a powerful method for image representation in various computer vision and machine learning tasks, such as object detection and classification (Oquab et al., 2014; Razavian et al., 2014). They provide a rich hierarchical set of learned visual features, from low-level pixel statistics to high-level semantic features. In this paper, we propose to explore how this DNN visual representation could be used for semantic animal region detection and species classification in challenging natural scenes. Specifically, we first design and train a DNN for animal-background object classification, which is used to analyze the input image to generate multi-layer feature maps, representing the responses of different image regions to the animal-background classifier. In this DNN feature domain, we perform clustering and graph cut to construct the semantic regions of animals. We then perform animal species classification on these semantic regions. Our experimental results demonstrate that the proposed method significantly outperforms existing classification and detection methods.

The main contributions of this work include: (1) we have developed a new approach for representing camera-trap image using semantic regions and detecting semantic regions of interest for animals in highly cluttered natural scenes in the learned DNN feature domain. (2) We have proposed a method to identify semantic regions of interest for more accurate image classification. (3) We have achieved accurate classification of animal species on a challenging dataset, outperforming existing methods.

The rest of paper is organized as follows. Section 2 reviews the related work. Section 3 presents our animal species classification method using semantic region of interest. Experimental results are presented in Section 4. Further discussions are provided in Section 5. Section 6 concludes the paper.