Elsevier

Ecosystem Services

Volume 31, Part C, June 2018, Pages 318-325
Ecosystem Services

Using image recognition to automate assessment of cultural ecosystem services from social media photographs

https://doi.org/10.1016/j.ecoser.2017.09.004Get rights and content

Highlights

  • Cultural use of ecosystems can be inferred from photographs.

  • Machine learning image recognition was used to classify photographs rapidly.

  • Over 20% of photographs taken in Singapore were of nature.

  • Automated classification of cultural services allows rapid mapping over large areas.

Abstract

Quantifying and mapping cultural ecosystem services is complex because of their intangibility. Data from social media, such as geo-tagged photographs, have been proposed for mapping cultural use or appreciation of ecosystems. However, manual content analysis and classification of large numbers of photographs is time consuming. This study develops a novel method for automating content analysis of social media photographs for ecosystem services assessment. The approach applies an online machine learning algorithm – Google Cloud Vision – to analyse over 20,000 photographs from Singapore, and uses hierarchical clustering to group these photographs. The accuracy of the classification was assessed by comparison with manual classification. Over 20% of photographs were taken of nature, being of animals or plants. The distribution of nature photographs was concentrated around particular natural attractions, and nature photographs were more likely to occur in parks and areas of high vegetation cover. The approach developed for clustering photographs was accurate and saved approximately 170 h of manual work. The method provides an indicator of cultural ecosystem services that can be applied rapidly over large areas. Automated assessment and mapping of cultural ecosystem services could be used to inform urban planning.

Introduction

Ecosystem services are the benefits that nature provides to people, including provisioning, regulating and cultural services (Millenium Ecosystem Assessment, 2003). To integrate ecosystem services within environmental decision making, we require information on a broad range of these services, so as to identify trade-offs among different objectives (Fish, 2011). Cultural ecosystem services are the non-material benefits that nature can provide, including recreational, spiritual and heritage values (Hernández-Morcillo et al., 2013). Quantifying cultural ecosystem services has traditionally been a time-consuming process, involving interviews (Plieninger et al., 2013), focus groups (Norton et al., 2012), or social surveys (Pleasant et al., 2014). More recently, data extracted from social media, such as geo-tagged photographs, have been used as indicators of recreational and aesthetic cultural ecosystem service value (Casalegno et al., 2013, Gliozzo et al., 2016, Keeler et al., 2015, Nahuelhual et al., 2013, Richards and Friess, 2015, Tenerelli et al., 2016, van Zanten et al., 2016, Wood et al., 2013). Data from social media offer great potential for quantifying ecosystem services rapidly and over large areas, though for media such as photographs this still involves human supervision, making it a time-consuming process (Richards and Friess, 2015). This study presents an approach for automating the content analysis of social media photographs to facilitate rapid and large-scale ecosystem service mapping.

Previous studies that have used social media data in ecosystem service mapping have mapped the density of geo-tagged photographs as a proxy for public interest in an area (Casalegno et al., 2013, Keeler et al., 2015, van Zanten et al., 2016, Wood et al., 2013). Such studies have demonstrated that social-media data can be valuable in helping to map cultural ecosystem services over large areas, but there is a wealth of additional information in social media that is currently under-used. To extract more from the resource provided by social media, it is necessary to analyse the content of photographs, firstly to ensure that the photographs are relevant to the natural environment, and secondly to understand what aspects of the environment are of most interest to people in a particular area (Richards and Friess, 2015).

The density of photographs in a location corresponds closely to its popularity with visitors (Wood et al., 2013), but does not necessarily relate to public interest in the environment. The presence of a photograph does not tell us why people visited a location; for example, there may be a high density of photographs because of high public interest in a natural feature, or due to interest in a nearby popular restaurant. Photograph density as an index of cultural ecosystem service quality may be particularly confounded in urban areas, because recreational green spaces are found in close proximity to built infrastructure with high recreational potential, such as malls, cinemas, and restaurants. The risk of overestimating public interest in ecosystems can be reduced by discounting photographs that fall over non-natural areas (Gliozzo et al., 2016). However, this approach requires a complete knowledge of where those areas are, which may be difficult to obtain in heterogeneous urban-rural landscapes. In urban areas, many human-nature interactions occur in very small spaces, including community or private gardens (Marco et al., 2010, Seik, 2000). Such spaces may be hidden within larger areas that are assumed to be urban, and excluding all photographs from these areas could undervalue the importance of nature. The risk of overestimating public interest in the environment can also be limited by applying filters to the textual content (“tags”) associated with each image (van Zanten et al., 2016). However, photograph tagging relies on classification of the images by users, some of whom are less diligent in recording a broad range of tags, or may not record any tags at all.

There is a wealth of information held within photographs of the environment that can be analysed alongside social surveys and interviews to infer how, and why, people interact with nature (Dorwart et al., 2009). The density of nature photographs in a place can be considered as an indicator of public interest in nature in that area, which to some extent relates to recreational and cultural heritage ecosystem services (Richards and Friess, 2015). However, interpreting the information held within photographs can be a challenge for research into cultural uses of the environment, because the choice of what to take a photograph of is subjective on behalf of the photographer, and the motivation for taking the photograph may be unclear without additional contextual information (Scott and Canter, 1996). Despite the caveats in analysing the content of photographs, images from social media websites have previously been analysed by manual classification, to assess the relative importance of different cultural and recreational uses (Richards and Friess, 2015, Thiagarajah et al., 2015). Manual analysis of photographs can be consistent between assessors, and can be relatively rapid at small spatial scales; indeed, a sufficient analysis of one natural site can be completed in around 30 min (Richards and Friess, 2015). However, manual classification of photograph content does not scale up easily, as the time investment required to compare a large number of sites would be substantial. To allow rapid assessment of cultural ecosystem services over large areas, we require automated content analysis of photographs from social media.

The capability of image recognition software has improved rapidly over the past few years, spurred by the availability of large image datasets and high-power cloud computing (Agrawal et al., 2015, Kwak and An, 2016). Generally-applicable image recognition algorithms are now accessible online, such as Google’s Cloud Vision, and Microsoft’s Computer Vision (Google Cloud Vision, 2017, Microsoft Computer Vision, 2017). Google Cloud Vision, launched in 2016, provides an Application Programming Interface (API) for image recognition, allowing users to analyse individual images for their content, which is described in terms of keywords (Google Cloud Vision, 2017). Generalised image recognition APIs are now being applied to analyse the content of large number of images for research purposes, for example to categorise the content and gender balance of images in the global news (Kwak and An, 2016).

Automated content analysis of photographs from social media could help to ensure that only relevant photographs are included in indices of cultural ecosystem service quality, and may also provide more information, by distinguishing between different cultural ecosystem services, or aspects of the environment that are of interest (Di Minin et al., 2015, Richards and Friess, 2015). This study demonstrates a novel method for analysing photographs taken from social media with the Google Cloud Vision image recognition algorithm, and applies this approach across the city-state of Singapore. The objectives of the study were (1) to quantify the occurrence of photographs relating to nature, (2) to analyse the drivers of spatial variation in nature photographs, and (3) to assess the accuracy of the automated classification by comparison with a manual classification conducted by a human.

Section snippets

Cultural ecosystem services in Singapore

Singapore is an island city-state of 5.6 million people, located in Southeast Asia (Department of Statistics, 2016). The population is relatively wealthy and there is a high rate of mobile phone ownership (IDA, 2014), making it a suitable country for analyses of social media data (Richards and Friess, 2015). While largely urban, around 56% of the land area is covered in managed and spontaneous vegetation, including public parks, gardens and nature reserves (Yee et al., 2011). Recreation is an

Extraction and clustering of photographs

The process extracted 130,115 unique photographs from Flickr, taken by 4174 photographers. The median number of photographs taken by each photographer was 3, and 90% of photographers uploaded fewer than 20 photographs. The maximum number of photographs uploaded by a single photographer was 739. A random subset of 25,000 images were assigned 2200 unique keywords by the Google Cloud Vision algorithm. 1952 images were not assigned any keywords and were excluded from further analysis, leaving

Occurrence of nature in social media photographs in Singapore

Of the 23,048 photographs that were classified by Google Cloud Vision, over 20% were of animals or plants. Despite the high population and built density, there is an active and vocal nature conservation community present in Singapore (Kong et al., 1996, Wee and Hale, 2008), and nature tourism is popular at particular sites, including Sungei Buloh and the Central Catchment (Henderson, 2000). Nature photographs tended to focus on particular organismal groups, such as flowers, trees, and birds (

Conclusions

This study presents a method for using a publicly-available image recognition algorithm to interpret the content of a large number of images taken from social media, in order to quantify and map interest in nature. Such an approach can be used to map the predominate areas for nature photography, and distinguish between different types of nature photographs, as shown through the case study of Singapore. The approach developed for clustering photographs based on their content was accurate, and

Acknowledgement

The research was conducted at the Future Cities Laboratory at the Singapore-ETH Centre, which was established collaboratively between ETH Zurich and Singapore's National Research Foundation (FI 370074016) under its Campus for Research Excellence and Technological Enterprise programme. The authors would like to thank Prof. Peter Edwards and two anonymous reviewers for their helpful additions to the manuscript.

References (56)

  • Agrawal, H., Mathialagan, C.S., Goyal, Y., Chavali, N., Banik, P., Mohapatra, A., Osman, A., Batra, D., 2015. Cloudcv:...
  • J. Burger et al.

    The effect of human activities on migrant shorebirds:successful adaptive management

    Environ. Conserv.

    (2004)
  • S. Casalegno et al.

    Spatial covariance between aesthetic value & other ecosystem services

    PLoS One

    (2013)
  • B. Clucas et al.

    Flagship species on covers of US conservation and nature magazines

    Biodivers. Conserv.

    (2008)
  • M. Davey

    Gardens by the bay: Ecologically reflective design

    Archit. Des.

    (2011)
  • Department of Statistics, S., 2016. Population Trends, 2016...
  • E. Di Minin et al.

    Prospects and challenges for social media data in conservation science

    Front. Environ. Sci.

    (2015)
  • C. Dobbs et al.

    Global drivers and tradeoffs of three urban vegetation ecosystem services

    PLoS One

    (2014)
  • C.E. Dorwart et al.

    Visitors’ perceptions of a trail environment and effects on experiences: a model for nature-based recreation experiences

    Leis. Sci.

    (2009)
  • J. Elith et al.

    A statistical explanation of MaxEnt for ecologists

    Divers. Distrib.

    (2011)
  • R.D. Fish

    Environmental decision making and an ecosystems approach: some challenges from the perspective of social science

    Prog. Phys. Geogr.

    (2011)
  • Gliozzo, G., Pettorelli, N., Haklay, M., 2016. Using crowdsourced imagery to detect cultural ecosystem services: a case...
  • Google Cloud Vision, 2017. Documentation for the Google Cloud Vision API. Available online:...
  • J.C. Henderson

    Urban parks and green spaces in Singapore

    Manag. Leis.

    (2013)
  • J.C. Henderson

    Managing tourism in small islands: the case of Pulau Ubin

    Singapore J. Sustain. Tour.

    (2000)
  • L. Hollenstein et al.

    Exploring place through user-generated content: Using Flickr to describe city cores

    Jsis

    (2010)
  • IDA, 2014. Statistics on Telecom Services for 2012 (July–December). Infocomm Development Authority of Singapore, URL...
  • B.L. Keeler et al.

    Recreational demand for clean water: evidence from geotagged photographs by visitors to lakes

    Front. Ecol. Environ.

    (2015)
  • Cited by (0)

    View full text