Keywords

1 Introduction

The video supervision system represents a new paradigm for reliable surrounding information collection. The applications widely covered from agriculture [1, 2] to monitoring wild animal movement and distribution [3, 4]. Nowadays, the video supervision systems are widely deployed in modern zoos. These supervision systems provide animal’s raw video streams for attracting online visitors. In this paper, we introduce a system which can estimate animal’s movement and evaluate their behavior for digital zoo visitors and local zoo keepers. In BearWatcher system, zoo’s visitors can get more meaningful message with the animals’ activity and set up friend-to-friend connection between each other. Meanwhile, we explore the possibility of analyzing animal’s behavior and their surroundings to significantly assist zoology.

The BearWatcher concept is based on the Animal Sensor Networks project [5] that uses wireless sensor networks to capture information about the animals and their surroundings. The project is part of the Botnia-Atlantica cross-border cooperation program including Sweden, Finland, and Norway.

The BearWatcher application is implemented for zoology and tourism in Lycksele Zoo in Sweden and Predator Center in Kuusamo, Finland. The Predator Center, which is located in north east of Finland near the town of Kuusamo, is best known for all their bears. Tourism, arranging photo shoots of the animals for TV programs and animal photographers are its main attractions. The Lycksele Zoo in Sweden, which is one of the biggest attractions in northern Sweden, has been test bed of Animal Sensor Networks and Digital Djurpark projects [6]. The BearWatcher system has been running successfully in bear area of Lycksele Zoo for 1 year.

2 System Architecture

Figure 1 shows the overview of the system architecture. Thanks to the employment of sensor networks, rich content information about the animals and their surroundings, such as video, picture, sound, and temperature are captured. Computer vision system manages, analyses and extracts information from sensors. To present the semantic information, BearWatcher website and smartphone applications are designed. The feedback conveys to animals through the welfare.

Fig. 1.
figure 1

Overview of the system architecture

Full size image

2.1 Sensor Networks Setup

The deployment in the Lycksele Zoo consists of 18 video transmitting nodes distributed over the whole zoo. The sensors connect themselves in a wireless ad-hoc network. In the bear area, we set up temperature sensor, three pan-tilt-zoom cameras for outdoor, and two cameras including one IR camera for bear den where bear hibernates and gives birth to cubs during winter. Various data as video, picture, sound, and temperature provide rich content information about the animals and their surroundings.

2.2 Computer Vision System

In the computer vision system, the major efforts are focused on providing the effective information of interest from the sensor’s raw data. In this case, capturing the bear’s activity is essential to understand bear’s behavior. To achieve this task, four steps are carried out:

  • Background/foreground estimation: Separate the foreground from the background.

  • Motion detection: Detect the motion of the animals.

  • Motion score: Evaluate the motion. The motion data combining time and position information are prepared for estimating animal’s behavior.

  • Store and manage activity data: Generate daily/weekly/monthly activity report, and automatically publish images with high activity.

In the real environment of the zoo, there is usually a complex background appearing in the video stream, for example a background that contains shadows, moving objects, or illumination changing. In the bear den, the light condition is poor, and ventilation systems cause noisy pictures. To solve these challenges, the background and foreground estimation from a real-time video steam containing complex background is designed based on Bayes decision theory [7].

First, the non-change pixels in the video sequence are filtered out by using simple background and temporal differences. The changes in the video sequence are separated as pixels belonging to stationary and moving objects according to inter-frame changes. Then the pixels associated with still or moving labels are further classified as background or foreground based on the learned statistics of colors and color co-occurrences respectively by using the Bayes decision rule. Then foreground objects are segmented by combining the classification results from both stationary and moving parts. At last, background models are updated. Both gradual and Once-off learning strategies are utilized to learn the statistics of feature vectors. Meanwhile, a reference background image is maintained to make the background difference accurate and adaptive to the background changing [8]. Figure 2 shows the result of background/foreground estimation.

Fig. 2.
figure 2

Result of background/foreground estimation

Full size image

The purpose of using motion detection is to find the video which contains animal activity. Motion detection works by comparing the intensity of pixels between new video sequence image and reference image. The connected component analysis uses the foreground pixels result to detect the close moving object. The motion score is calculated as the size of moving object to the scene. To reduce the noise, the moving object should be continuous moving for 10 frames.

By setting the threshold of the motion score, the system will record the bear’s high activity data, and plot them into statistical graphics in web application. In the meantime, the pictures with large movement will be saved and pushed to the web application. Figure 3 shows bears’ activity for the whole day in the bear den.

Fig. 3.
figure 3

28 h bear activity in the bear den

Full size image

In traditional zoo management way, when bears hibernate, they will sleep in their cave for the whole winter without disturbance, which means the zoo keepers will not know when the bear gives birth to cubs, neither how many kids she has until the next spring. By analyzing the long-term activity data, bear’s behavior can be predicted. During winter 2013/2014, BearWatcher recorded the activity data during bear hibernation and giving birth to cubs. Figure 4 shows bear activity changes during this period. From the 1 month activity data we can see the week before labor, bear mother started to move double amount than normal hebetation days during sleep. The activity kept increasing. Bear started to wake up and walked inside the cave more and more. While the motion scores of the labor day (marked as red) is 5 times more than the score from normal hibernation days. With BearWatcher monitoring the animal’s movement, zoo keeper can get reliable, accurate help to predict and understand animal’s behavior.

Fig. 4.
figure 4

30 days bear’s activity in the bear den during hibernation and giving birth period. Labor day’s data is marked as red. Bear mother started to move double amount than normal hebetation days from 4 days before labor, on the labor day is 5 times more than normal (Color figure online).

Full size image

Motion estimation is performed for automatically capturing the bear’s activity from all cameras. Activity data will be recorded. System will automatically push pictures with large movement to web application, which means visitors can receive the zoo’s fun moments, instead of sleepy animals. In the meantime, bear’s activity data will be stored and managed as activity daily/weekly/monthly report and presented to zoo keepers.

2.3 Interaction Design for BearWatcher

In traditional zoos, information flow from animal to visitors is passive. Based on the sensor networks setup and computer vision system analyzed bears’ activity data, BearWatcher is designed that give the users a more interactive visiting experience. Three components are concerned by BearWatcher to enhance the interactive experience: Information Component, Web Application Component, Interactor Component. These components take charge of their own function, give different outputs. In the meantime, they relate to each other, use each other’s output as input, update and communicate with each other. Figure 5 shows the relationship of three components:

Fig. 5.
figure 5

3 interactive components in BearWatcher

Full size image

  • Information Component contains basic information which shows the knowledge and events of the zoo: maps, directions, animal’s fact, etc. Sensor networks capture live stream and their surrounding features. Activity report and bear’s photos with high activity are pushed by computer vision system. The design of this component focuses on meaningful context used to present to the Interactor Component. The information will be automatically updated, and always in the background making Interactors, if only subconsciously, aware of the state of on-going processes.

  • Interactor Component refers all the people, both visitors and zoo keepers, involved as active users of the information. They make interaction through the Web Application Component with both animals and other people; we give them a new name, Interactors [9].

  • Web Application Component concerns how to present the Information Component to the Interactor Component. BearWatcher developed two web applications: bear broadcast for tourism and bear activity report for zoo management. They both use website and smartphone application as interface.

Bear Broadcast for Tourism. In the traditional zoo visiting experience, information flow is passive. Visitors walk around the zoo, look at animals, and in some zoos they can pet animals. Every so often, animals are sleeping in the cave, or hiding in the bush. By estimating activity and motion of the animals in the digital zoo, BearWatcher enhances the visiting experience with information containing richer content. The purpose of this application is to provide visitor, as Interactor, more opportunities to get direct way following the animals and get to know animals a little better in personal way.

In the BearWatcher website, visitors can select different live streams of the bears; browse the gallery of bear’s activity images. Moreover, to help the bears involving in the visitor’s life, social media such as Twitter is used. Computer Vision System pushes all the photos with high activity to web application. The photos combine the position information and time information can generate the meaningful message. For instance, high activity at feeding place around 5 p.m. indicates bears are eating. A Twitter message says “Dinner time!” with a photo will be generated and broadcast by system through phone application and Twitter. The message communicates as the same way as friend to friend in real life, the visitors are engaged in real-time. They can comment directly on the website or retweet the message to response or share the event. As the administrator in management, zoo staff can manage all the photos, upload some extra behind scenes photos, generate new broadcast, and respond the visitor’s comments. In this way, the two-way dialogue is created. Figure 6 is the activity function for Bear watcher smartphone application.

Fig. 6.
figure 6

Activity function for BearWatcher smartphone application

Full size image

With the help of bear broadcast application, the zoos can keep connection with their visitors even after their attendees leave. A broadcast with a series of bear’s activity posts, can surface their story to engage their global audiences. One of the usage scenarios was when the Lycksele zoo got the news of baby bears will be born at the zoo in the winter. Bear Mama’s expecting cubs photo started the story, followed with she was walking around the cave which means babies were on their way, to the first look of the small cubs gathered together without clear number of the babies, until finally clear photos of three cubs.

In this way, the relationship between the visitors and bears is more interactive. System gives the visitors of the zoo a more digital, more available, more attractive experience.

Bear Activity Report for Zoo Management. In most traditional zoos, zoo keepers need to write welfare diary about the animal’s behavior such as feeding, sleeping and abnormal behaviors for every animal that he, or she, takes care of everyday. The content of the welfare diary is based on the observations and staff’s experience. Bear activity report application for zoo management aiming for helping with zoo keepers develop a more efficient, more accurate and easier way to take care of animals.

In this application, interaction design focuses on how to present the information organized, and easy to handle by the Interactor, which means something that is easy and appealing to use by non-technical users. BearWatcher present all the information to zoo’s staff by using the same web application system just with log-in function. After log-in as zoo keeper account, the administration organized into three catalogues: Activity report, Tweet and Control Room. In Activity Report page, by selecting different cameras, detail of daily/weekly/monthly individual animal’s activity line chart is shown; zoo keepers can directly export the graph to their welfare diary. In Tweet page, an easy to use Tweet function is designed for generate quick Twitter message. Bear’s photos during the whole day with high activity are list on the right side of the page, user can select the fun picture from them, drag and drop to the Tweet box, send it with short description. User can edit Twitter Messages that generated automatically by computer vision system. Control Room page lists recent posts, latest 24 h activity chart, recent comments from visitors, and recent activity photos. Figure 7 shows the control page for the zoo staff. Figure 8 is the twitter update page.

Fig. 7.
figure 7

Control page for the zoo staff.

Full size image
Fig. 8.
figure 8

Twitter update page using easy drag and drop function

Full size image

3 Discussion

BearWatcher is a system sensing animal’s movement, analyzing their activity, and present them to website and smartphone application. The purpose of the system is using computer vision and media technology to improve interactive experience of the zoo and help animal’s welfare. From a marketing aspect, modern zoo representatives increase in value of having contact with their visitors [10]. This shows that modern zoos combining local zoo physical environment, live streaming digital world, animal welfare information, and social media interaction can create tourist attractions. In the meantime, using digital world’s record to analyze and help physical animal’s welfare can generate more accurate and cost-effective method to take care of the animals.

The whole system is created to enhance user experience. The system is designed to meet the user experience hierarchy of needs: functional, reliable, usable, convenient, pleasurable and meaningful [11]. The sensor networks and computer vision system provide the basic functional and reliable result, which have been running successfully in bear area of Lycksele Zoo for 1 year. The challenge of the system in the future work focuses on truly creating a convenient and meaningful product. In the further work, the system needs more participants for the user experience evaluations.

4 Conclusion

In this paper, we introduce an application based on animal motion estimation in the zoo. BearWatcher is a system sensing animal’s movement, analyzing their activity, and presenting them to website and smartphone applications. With the interactive experience design, the system bridges the gap between digital zoo and their visitors. The system gives the zoo visitors more active, more involved experience. Meanwhile it provides more accurate and cost-effective welfare conveys to animals.