Skip to main content
SpringerLink
Log in

Performance Evaluation of the SLEUTH Model in the Shenyang Metropolitan Area of Northeastern China

  • Published:
Environmental Modeling & Assessment Aims and scope Submit manuscript

Abstract

Performance evaluation is crucial for the development and improvement of an urban cellular automata model, such as SLEUTH. In this paper, we employed multiple methods for map comparison and model validation to evaluate the simulation performance of the SLEUTH urban growth model in the Shenyang metropolitan area of China. These multiple methods included the relative operating characteristic (ROC) curve statistic, multiple-resolutions error budget, and landscape metrics. They were used to quantitatively examine model performance in terms of the amount and spatial location of urban development, urban spatial pattern and prediction ability. The assessment results showed that SLEUTH performed well in the way of the quantitative simulation of urban growth for this case study. Similar to other urban growth models, however, the simulation accuracy for spatial location of new development at the pixel scale and urban spatial pattern still needs to be improved greatly. These inaccuracies might be attributed to the structure and nature of SLEUTH, local urban development characteristics, and the temporal and spatial scale of its application. Finally, many valuable suggestions had been put forward to improve simulation performance of SLEUTH model for spatial location of urban development in the Shenyang metropolitan area.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Alberti, M., & Waddell, P. (2000). An integrated urban development and ecological simulation model. Integrated Assessment, 1, 215–227. doi:10.1023/A:1019140101212.

    Article  Google Scholar 

  2. Batty, M., & Xie, Y. (1994). From cells to cities. Environment and Planning B: Planning and Design, 21, 31–48. doi:10.1068/b21s031.

    Article  Google Scholar 

  3. Candau, J. C. (2002). Temporal Calibration Sensitivity of the SLEUTH Urban Growth Model. M.A. Thesis, University of California, Santa Barbara, USA

  4. Carletta, J. (1996). Assessing agreement on classification tasks: the kappa statistic. Computational Linguistics, 22(2), 249–254.

    Google Scholar 

  5. Claggett, P. R., Jantz, C. A., Goetz, S. J., & Bisland, C. (2004). Assessing development pressure in the Chesapeake Bay watershed: an evaluation of two land use change models. Environmental Monitoring and Assessment, 94, 129–146. doi:10.1023/B:EMAS.0000016884.96098.77.

    Article  Google Scholar 

  6. Clarke, K. C., Hoppen, S., & Gaydos, L. (1997). A self-modifying cellular automaton model of historical urbanization in the San Francisco Bay Area. Environment and Planning B: Planning and Design, 24, 247–261. doi:10.1068/b240247.

    Article  Google Scholar 

  7. Clarke, K. C., & Dietzel, C. (2004). A Decade of SLEUTHing: Lessons learned from applications of a cellular automaton land use change model. Paper presented at the Joint International Workshop on Integrated Assessment of the Land System: The Future of Land Use. Institute for Environmental Studies, Amsterdam.

  8. Dietzel, C. (2004). Spatio-temporal difference in model outputs and parameter space as determined by calibration extent. In P. M. Atkinson, G. M. Foody, S. E. Darby, & F. Wu (Eds.), Geodynamics pp. 233–247. Boca Raton: CRC.

    Google Scholar 

  9. Dietzel, C., & Clarke, K. C. (2004). Spatial differences in multi-resolution urban automata modeling. Transactions in GIS, 8(4), 479–492. doi:10.1111/j.1467-9671.2004.00197.x.

    Article  Google Scholar 

  10. Dietzel, C., & Clarke, K. C. (2006). The effect of disaggregating land use categories in cellular automata during model calibration and forecasting. Computers, Environment and Urban Systems, 30, 78–101. doi:10.1016/j.compenvurbsys.2005.04.001.

    Article  Google Scholar 

  11. Goldstein, N. C., Candau, J. T., & Clarke, K. C. (2004). Approaches to simulating the “March of Bricks and Mortar”. Computers, Environment and Urban Systems, 28, 125–147. doi:10.1016/S0198-9715(02)00046-7.

    Article  Google Scholar 

  12. Gustafson, E. J. (1998). Quantifying landscape spatial pattern: what is the state of the art? Ecosystems, 1, 143–156. doi:10.1007/s100219900011.

    Article  Google Scholar 

  13. Hagen, A. (2002). Multi-method assessment of map similarity. Paper presented at the 5th AGILE Conference on Geographic Information Science, Palma, Spain.

  14. Hagen-Zanker, A. (2006). Map comparison methods that simultaneously address overlap and structure. Journal of Geographical Systems, 8(2), 165–185. doi:10.1007/s10109-006-0024-y.

    Article  Google Scholar 

  15. Herold, M., Goldstein, N. C., & Clarke, K. C. (2003). The spatiotemporal form of urban growth: measurement, analysis and modeling. Remote Sensing of Environment, 86, 286–302. doi:10.1016/S0034-4257(03)00075-0.

    Article  Google Scholar 

  16. Herold, M., Couclelis, H., & Clarke, K. C. (2005). The role of spatial metrics in the analysis and modeling of urban land use change. Computers, Environment and Urban Systems, 29, 369–399. doi:10.1016/j.compenvurbsys.2003.12.001.

    Article  Google Scholar 

  17. Jantz, C. A., Goetz, S. J., & Shelley, M. K. (2003). Using the SLEUTH urban growth model to simulate the impacts of future policy scenarios on urban land use in the Baltimore–Washington metropolitan area. Environment and Planning B: Planning and Design, 31, 251–271. doi:10.1068/b2983.

    Article  Google Scholar 

  18. Jantz, C. A., & Goetz, S. J. (2005). Analysis of scale dependencies in an urban land use change model. International Journal of Geographical Information Science, 19(2), 217–241. doi:10.1080/13658810410001713425.

    Article  Google Scholar 

  19. Klostermann, R. E. (1999). The what if? Collaborative planning support system. Environment and Planning B: Planning and Design, 26, 393–408. doi:10.1068/b260393.

    Article  Google Scholar 

  20. Li, X., & Yeh, A. (2000). Modeling sustainable urban development by the integration of constrained cellular automata and GIS. International Journal of Geographical Information Science, 14(2), 131–152. doi:10.1080/136588100240886.

    Article  Google Scholar 

  21. Li, X., & Yeh, A. (2004). Analyzing spatial restructuring of land use patterns in a fast growing region using remote sensing and GIS. Landscape and Urban Planning, 69, 335–354. doi:10.1016/j.landurbplan.2003.10.033.

    Article  Google Scholar 

  22. McGarigal, K., & Marks, B. J. (1995). FRAGSTATS: spatial pattern analysis program for quantifying landscape structure. Gen. Tech. Rep. PNW-GTR-351. Portland, OR: U.S. Department of Agriculture, Forest Service, Pacific Northwest Research Station. 122 p.

  23. Pontius Jr., R. G. (2000). Quantification error versus location error in comparison of categorical maps. Photogrammetric Engineering and Remote Sensing, 66(8), 1011–1016.

    Google Scholar 

  24. Pontiusx Jr., R. G., & Schneider, L. C. (2001). Land-cover change model validation by an ROC method for the Ipswich watershed, Massachusetts, USA. Agriculture, Ecosystems & Environment, 85, 239–248. doi:10.1016/S0167-8809(01)00187-6.

    Article  Google Scholar 

  25. Pontius Jr., R. G. (2002). Statistical methods to partition effects of quantity and location during comparison of categorical maps at multiple resolutions. Photogrammetric Engineering and Remote Sensing, 68(10), 1041–1049.

    Google Scholar 

  26. Pontius Jr., R. G., & Batchu, K. (2003). Using the relative operating characteristic to quantify certainty in prediction of location of land cover change in India. Transactions in GIS, 7(4), 467–484. doi:10.1111/1467-9671.00159.

    Article  Google Scholar 

  27. Pontius Jr., R. G., & Suedmeyer, B. (2004a). Components of agreement between categorical maps at multiple resolutions. In R. S. Lunetta, & J. G. Lyon (Eds.), Remote Sensing and GIS Accuracy Assessment (pp. 233–251). Boca Raton: CRC.

    Google Scholar 

  28. Pontius Jr., R. G., Huffaker, D., & Denman, K. (2004b). Useful techniques of validation for spatially explicit land-change models. Ecological Modelling, 179(4), 445–461. doi:10.1016/j.ecolmodel.2004.05.010.

    Article  Google Scholar 

  29. Project Gigalopolis: Urban and Land Cover Modeling. URL: www.ncgia.ucsb.edu/projects/gig/project_gig.htm.

  30. Silva, E. A., & Clarke, K. C. (2002). Calibration of the SLEUTH urban growth model for Lisbon and Porto, Portugal. Computers, Environment and Urban Systems, 26, 525–552. doi:10.1016/S0198-9715(01)00014-X.

    Article  Google Scholar 

  31. Sui, D. Z., & Zeng, H. (2001). Modeling the dynamics of landscape structure in Asia's emerging desakota regions: a case study in Shenzhen. Landscape and Urban Planning, 53, 37–52. doi:10.1016/S0169-2046(00)00136-5.

    Article  Google Scholar 

  32. Swets, J. A. (1986). Indices of discrimination or diagnostic accuracy: their ROC’s and implied models. Psychological Bulletin, 99, 100–17. doi:10.1037/0033-2909.99.1.100.

    Article  CAS  Google Scholar 

  33. Torrens, P. M., & O’Sullivan, D. (2001). Cellular automata and urban simulation: where do we go from here. Environment and Planning B: Planning and Design, 28, 163–168. doi:10.1068/b2802ed.

    Article  Google Scholar 

  34. Verburg, P. H., de Nijs, T. C. M., Ritsema van Eck, J. R., Visser, H., & de Jong, K. (2004). A method to analyse neighborhood characteristics of land use patterns. Computers, Environment and Urban Systems, 28(6), 667–690. doi:10.1016/j.compenvurbsys.2003.07.001.

    Article  Google Scholar 

  35. White, R., & Engelen, G. (1993). Cellular automata and fractal urban form: a cellular modeling approach to the evolution of urban land use patterns. Environment and Planning A, 25, 1175–1199. doi:10.1068/a251175.

    Article  Google Scholar 

  36. Wu, F. (1998). Simulating urban encroachment on rural land with fuzzy logic controlled cellular automata in a geographical information system. Journal of Environmental Management, 53, 293–308. doi:10.1006/jema.1998.0195.

    Article  Google Scholar 

  37. Xian, G., Crane, M., & Steinwand, D. (2005). Dynamic modeling of Tampa Bay urban development using parallel computing. Computers & Geosciences, 31(7), 920–928. doi:10.1016/j.cageo.2005.03.006.

    Article  Google Scholar 

  38. Yang, X., & Lo, C. P. (2003). Modelling urban growth and landscape change in the Atlanta metropolitan area. International Journal of Geographical Information Science, 17(5), 463–488. doi:10.1080/1365881031000086965.

    Article  Google Scholar 

Download references

Acknowledgments

We thank Professor Keith C. Clarke, University of California Santa-Barbara, for providing helpful suggestions. This work was jointly sponsored by the Knowledge Innovation Programs of the Chinese Academy of Sciences (KSCX2-SW-133) and the Knowledge Innovation Programs of Institute of Applied Ecology, Chinese Academy of Sciences (O6LYQY1001). We are grateful to the anonymous reviewers for their constructive suggestions and comments.

Author information

Authors and Affiliations

  1. Institute of Applied Ecology, Chinese Academy of Sciences, 72, Wenhua Road, Shenhe District, Shenyang, 110016, China

    Xiaoqing Wu, Yuanman Hu, Hong S. He, Rencang Bu & Fengming Xi

  2. Department of International Relations and Geography, Florida International University, 11200 S.W. 8th Street, Miami, FL, 33199, USA

    Jeff Onsted

Authors
  1. Xiaoqing Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuanman Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hong S. He
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rencang Bu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jeff Onsted
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Fengming Xi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuanman Hu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wu, X., Hu, Y., He, H.S. et al. Performance Evaluation of the SLEUTH Model in the Shenyang Metropolitan Area of Northeastern China. Environ Model Assess 14, 221–230 (2009). https://doi.org/10.1007/s10666-008-9154-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10666-008-9154-6

Keywords

Search

Navigation