Abstract
Context
Validation of models of Land Use and Cover Change often involves comparing maps of simulated and reference change. The interpretation of differences between simulated and reference change depends on the characteristics of the process being studied. Our paper focuses on validation of models of one-way land change processes that spread in space.
Objectives
Our objective is to develop a method for validation of one-way land change models, such that the method provides objective information about the spatial distribution of errors.
Methods
Using distance analysis on reference data, we build a baseline model for comparison with simulations. We then simultaneously compare the four maps of reference at initial time, reference at final time, simulation at final time, and baseline at final time. We also use Total Operating Characteristic curves and multiple-resolution map comparison. We illustrate the methods with a simulation of forest insect infestations.
Results
The methods give insights concerning the reference data and the spatial distribution of misses, hits, and false alarms with respect to initial points of infestations. The new methods reveal that the simulations underestimated change near initial points of spread.
Conclusions
The spatial distribution of errors is a topic of land change models that deserves attention. For models of one-way, geographically-spreading processes, we recommend that validation should distinguish between near and far allocation errors with respect to initial points of spread.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The datasets generated and/or analysed are available in the Open Science Framework repository, via https://osf.io/d5em3/.
References
Batty M, Torrens PM (2005) Modelling and prediction in a complex world. Futures 37:745–766
Brown DG, Walker R, Manson S, Seto K (2012) Modeling land use and land cover change. In: Gutman G, Janetos AC, Justice CO et al (eds) Land change science. Springer, Dordrecht, pp 395–409
Brown DG, Verburg PH, Pontius RG Jr, Lange MD (2013) Opportunities to improve impact, integration, and evaluation of land change models. Curr Opin Environ Sustain 5:452–457
Chen H, Pontius RG Jr (2010) Diagnostic tools to evaluate a spatial land change projection along a gradient of an explanatory variable. Landsc Ecol 25:1319–1331
Congalton RG (2004) Putting the map back in map accuracy assessment. In: Lunetta RS, Lyon JG (eds) Remote sensing and GIS accuracy assessment. CRC Press, Boca Raton, pp 1–11
Cushman SA, Macdonald EA, Landguth EL et al (2017) Multiple-scale prediction of forest loss risk across Borneo. Landsc Ecol 32:1581–1598
de Sousa-Neto ER, Gomes L, Nascimento N et al (2018) Land use and land cover transition in Brazil and their effects on greenhouse gas emissions. Soil management and climate change. Academic Press, Cambridge, pp 309–321
Di Gregorio S, Serra R, Villani M (1997) A cellular automata model of soil bioremediation. Complex Syst 11:31–54
ESRI (2015) ArcGIS 10.4.1 for desktop
Foody GM (2004) Thematic map comparison: evaluating the statistical significance of differences in classification accuracy. Photogramm Eng Remote Sens 70:627–633
Gaudreau J, Perez L, Drapeau P (2016) BorealFireSim: a GIS-based cellular automata model of wildfires for the boreal forest of Quebec in a climate change paradigm. Ecol Inform 32:12–27
Hagen-Zanker A (2006) Map comparison methods that simultaneously address overlap and structure. J Geogr Syst 8:165–185
Harati S, Perez L, Molowny-Horas R (2020) Integrating neighborhood effect and supervised machine learning techniques to model and simulate forest insect outbreaks in British Columbia, Canada. Forests 11:1–23
Hermoso V, Morán-Ordóñez A, Brotons L (2018) Assessing the role of Natura 2000 at maintaining dynamic landscapes in Europe over the last two decades: implications for conservation. Landsc Ecol 33:1447–1460
Hijmans RJ (2019) raster: geographic data analysis and modeling. R package version 2.9–5
Lambin EF, Geist H, Rindfuss RR (2006) Introduction: local processes with global impacts. In: Lambin EF, Geist H (eds) Land-use and land-cover change. Springer, Berlin, pp 1–8
Liu Z (2020) TOC Curve Generator. https://lazygis.github.io/projects/TOCCurveGenerator
Li Z, Huffman T, Zhang A et al (2012) Spatially locating soil classes within complex soil polygons – Mapping soil capability for agriculture in Saskatchewan Canada. Agric Ecosyst Environ 152:59–67
Moulds S, Buytaert W, Mijic A (2015) An open and extensible framework for spatially explicit land use change modelling: the lulcc R package. Geosci Model Dev 8:3215–3229
National Research Council (2014) Advancing land change modeling: opportunities and research requirements. National Academies Press, Washington, D.C
Natural Resources Canada (2019) Mountain pine beetle. https://www.nrcan.gc.ca/our-natural-resources/forests-forestry/wildland-fires-insects-disturban/top-forest-insects-diseases-cana/mountain-pine-beetle/13381. Accessed 2 June 2020
Paudel S, Yuan F (2012) Assessing landscape changes and dynamics using patch analysis and GIS modeling. Int J Appl Earth Obs Geoinf 16:66–76
Pérez L, Dragićević S, White R (2013) Model testing and assessment: perspectives from a swarm intelligence, agent-based model of forest insect infestations. Comput Environ Urban Syst 39:121–135
Perez L, Molowny-Horas R, Harati S (2016) Modelling forest insect outbreaks: efforts towards an inverse approach to model calibration. In: Sauvage S, Sánchez-Pérez JM, Rizzoli AE (eds) Proceddings of the 8th International Congress on Environmental Modelling and Software (iEMSs). Toulouse, France, p 688
Pijanowski BC, Pithadia S, Shellito BA, Alexandridis K (2005) Calibrating a neural network-based urban change model for two metropolitan areas of the Upper Midwest of the United States. Int J Geogr Inf Sci 19:197–215
Pontius RG Jr (2000) Quantification error versus location error in comparison of categorical maps. Photogramm Eng Remote Sens 66:1011–1016
Pontius RG Jr (2002) Statistical methods to partition effects of quantity and location during comparison of categorical maps at multiple resolutions. Photogramm Eng Remote Sensing 68:1041–1050
Pontius RG Jr (2018) Criteria to confirm models that simulate deforestation and carbon disturbance. Land 7:1–14
Pontius RG Jr, Millones M (2011) Death to Kappa: birth of quantity disagreement and allocation disagreement for accuracy assessment. Int J Remote Sens 32:4407–4429
Pontius RG Jr, Parmentier B (2014) Recommendations for using the relative operating characteristic (ROC). Landsc Ecol 29:367–382
Pontius RG Jr, Si K (2014) The total operating characteristic to measure diagnostic ability for multiple thresholds. Int J Geogr Inf Sci 28:570–583
Pontius RG Jr, Huffaker D, Denman K (2004) Useful techniques of validation for spatially explicit land-change models. Ecol Modell 179:445–461
Pontius RG Jr, Boersma W, Castella JC et al (2008) Comparing the input, output, and validation maps for several models of land change. Ann Reg Sci 42:11–37
Pontius RG Jr, Peethambaram S, Castella JC (2011) Comparison of three maps at multiple resolutions: a case study of land change simulation in Cho Don district, Vietnam. Ann Assoc Am Geogr 101:45–62
Pontius RG Jr, Santacruz A, Tayyebi A, et al (2015) TOC: total operating characteristic curve and ROC curve. R package version 0.0–4 https://cran.r-project.org/web/packages/TOC/index.html
Pontius RG Jr, Castella J-C, de Nijs T et al (2018) Lessons and challenges in land change modeling derived from synthesis of cross-case comparisons. In: Behnisch M, Meinel G (eds) Possible urban futures: the impact of planners and developers on urban dynamics. Springer International Publishing, Cham, pp 143–164
Pontius RG Jr, Walker R, Yao-kumah R et al (2007) Accuracy assessment for a simulation model of Amazonian deforestation. Ann Assoc Am Geogr 97:677–695
Province of British Columbia (2015) BC MPB observed cumulative Kill - Vol.12
Province of British Columbia (2020) Aerial Overview Survey Methods. https://www2.gov.bc.ca/gov/content/industry/forestry/managing-our-forest-resources/forest-health/aerial-overview-surveys/methods. Accessed 2 June 2020
R Core Team (2019) R: a language and environment for statistical computing
Rollins MG, Keane RE, Parsons RA (2004) Mapping fuels and fire regimes using remote sensing, ecosystem simulation, and gradient modeling. Ecol Appl 14:75–95
Rykiel EJ (1996) Testing ecological models: the meaning of validation. Ecol Modell 90:229–244. https://doi.org/10.1016/0304-3800(95)00152-2
Tobler WR (1970) A computer movie simulating urban growth in the detroit region. Econ Geogr 46:234
van Vliet J, Bregt AK, Hagen-Zanker A (2011) Revisiting Kappa to account for change in the accuracy assessment of land-use change models. Ecol Modell 222:1367–1375
van Vliet J, Bregt AK, Brown DG et al (2016) A review of current calibration and validation practices in land-change modeling. Environ Model Softw 82:174–182
Verburg PH, Kok K, Pontius RG Jr, Veldkamp A (2006) Modeling land-use and land-cover change. In: Lambin EF, Geist H (eds) Land-use and land-cover change. Springer, Berlin, pp 117–135
White R (2006) Pattern based map comparisons. J Geogr Syst 8:145–164
Acknowledgements
We are thankful to the Natural Sciences and Engineering Research Council (NSERC) of Canada for partial support of this study under the Discovery Grant Program awarded to LP, and to the Université de Montréal’s International Affairs Office (IAO) for their financial support through the International Partnership Development program, which allowed the collaboration between researchers from UdeM and CREAF. RMH received financial support from the NEWFOREST (PIRSES-GA-2013-612645) program of the European Union’s Seventh Framework Programme. The United States National Science Foundation supported RGP through its Long Term Ecological Research Network via grant OCE-1637630 for Plum Island Ecosystems. We thank four anonymous reviewers who provided constructive comments on this paper
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Harati, S., Perez, L., Molowny-Horas, R. et al. Validating models of one-way land change: an example case of forest insect disturbance. Landscape Ecol 36, 2919–2935 (2021). https://doi.org/10.1007/s10980-021-01272-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10980-021-01272-0