Next Article in Journal
Influence of Ecological Factors on Estimation of Impervious Surface Area Using Landsat 8 Imagery
Next Article in Special Issue
Linear Multi-Task Learning for Predicting Soil Properties Using Field Spectroscopy
Previous Article in Journal
Aerosol Optical Properties and Associated Direct Radiative Forcing over the Yangtze River Basin during 2001–2015
Previous Article in Special Issue
Effect of Solar-Cloud-Satellite Geometry on Land Surface Shortwave Radiation Derived from Remotely Sensed Data
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Remote Sensing
Volume 9
Issue 7
10.3390/rs9070743
remotesensing-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Evaluation of the U.S. Geological Survey Landsat Burned Area Essential Climate Variable across the Conterminous U.S. Using Commercial High-Resolution Imagery

by
Melanie K. Vanderhoof
*,
Nicole Brunner
,
Yen-Ju G. Beal
and
Todd J. Hawbaker
U.S. Geological Survey, Geosciences and Environmental Change Science Center, P.O. Box 25046, DFC, MS980, Lakewood, CO 80225, USA
*
Author to whom correspondence should be addressed.
Remote Sens. 2017, 9(7), 743; https://doi.org/10.3390/rs9070743
Submission received: 8 May 2017 / Revised: 10 July 2017 / Accepted: 17 July 2017 / Published: 20 July 2017
(This article belongs to the Special Issue Quantitative Remote Sensing of Land Surface Variables)
Browse Figures
Versions Notes

Abstract

:
The U.S. Geological Survey has produced the Landsat Burned Area Essential Climate Variable (BAECV) product for the conterminous United States (CONUS), which provides wall-to-wall annual maps of burned area at 30 m resolution (1984–2015). Validation is a critical component in the generation of such remotely sensed products. Previous efforts to validate the BAECV relied on a reference dataset derived from Landsat, which was effective in evaluating the product across its timespan but did not allow for consideration of inaccuracies imposed by the Landsat sensor itself. In this effort, the BAECV was validated using 286 high-resolution images, collected from GeoEye-1, QuickBird-2, Worldview-2 and RapidEye satellites. A disproportionate sampling strategy was utilized to ensure enough burned area pixels were collected. Errors of omission and commission for burned area averaged 22 ± 4% and 48 ± 3%, respectively, across CONUS. Errors were lowest across the western U.S. The elevated error of commission relative to omission was largely driven by patterns in the Great Plains which saw low errors of omission (13 ± 13%) but high errors of commission (70 ± 5%) and potentially a region-growing function included in the BAECV algorithm. While the BAECV reliably detected agricultural fires in the Great Plains, it frequently mapped tilled areas or areas with low vegetation as burned. Landscape metrics were calculated for individual fire events to assess the influence of image resolution (2 m, 30 m and 500 m) on mapping fire heterogeneity. As the spatial detail of imagery increased, fire events were mapped in a patchier manner with greater patch and edge densities, and shape complexity, which can influence estimates of total greenhouse gas emissions and rates of vegetation recovery. The increasing number of satellites collecting high-resolution imagery and rapid improvements in the frequency with which imagery is being collected means greater opportunities to utilize these sources of imagery for Landsat product validation.

Graphical Abstract

1. Introduction

The Global Climate Observing System (GCOS) program identifies fire disturbance as one of the high priority Essential Climate Variables (ECV) [1]. Fire is a critical source of disturbance influencing vegetation dynamics [2,3], carbon and nutrient cycling [4,5], greenhouse gas emissions [6,7], and landscape heterogeneity [8,9]. The U.S. Geological Survey (USGS) has developed a Burned Area Essential Climate Variable (BAECV) product for the conterminous United States (CONUS) which capitalizes on the continuous record and moderate spatial resolution provided by the Landsat satellites to provide wall-to-wall maps of burned areas across CONUS (1984–2015) at 30 m resolution [10]. This product is meant to complement existing global burned area products (300 m to 1 km in resolution), such as NASA’s Moderate Resolution Imaging Spectrometer (MODIS) burned area products (2000–present), the MCD45 [11] and the MCD64 [12]. Providing a burned area record for longer time spans at finer spatial resolutions can improve our ability to discern temporal trends [13], relationships with climate and other drivers [14,15,16], as well as detect smaller fires and map fire heterogeneity [17]. The BAECV was initially validated using an independent dataset also derived from Landsat [18]. This initial validation approach allowed for the BAECV to be validated across a large temporal and spatial extent, which provides valuable information on the algorithms performance and stability. However, this type of validation does not allow for consideration of inaccuracies imposed by the Landsat sensor itself, such as mixed pixels (burned and unburned cover types within a single pixel) [19] or errors in the manual interpretation of the imagery due to its moderate spatial resolution. The current study uses high-resolution commercial imagery collected across the CONUS to validate the USGS’s Landsat BAECV (1984–2015).
Validation is a critical component in the generation of remotely sensed products that allows users to decide when and how to utilize datasets, correctly interpret results, and provide feedback to improve products [20,21]. The Committee on Earth Observation Satellites (CEOS), Land Product Validation Subgroup (LPVS), which formed in 2000, has specified that validation should follow internationally agreed upon validation best practices to measure accuracy and precision (standard error of accuracy estimates) at comprehensive spatial and temporal scales [21]. Accuracy of existing global burned area products is relatively low with documented errors of omission and commission for burned areas ranging 51% to 93%, and 42% to 94%, respectively [22,23,24]. This is due to the tremendous amount of spectral diversity introduced when attempting to map burned areas across diverse vegetation types, fire combustion levels (e.g., ash, char, soot), and burn severities (e.g., ground vs. crown fires). Coarse resolution (300 m to 1 km) burned area products are most commonly validated with Landsat imagery [22,23,24,25]. Similarly, efforts to map forest disturbance or burned area using Landsat at a regional scale, also typically rely on Landsat imagery for validation and use manual interpretation of the validation imagery to improve the quality of the reference dataset [26,27,28]. This is largely because to date it has been considered challenging, and potentially prohibitively costly, to acquire enough incidentally collected high-resolution imagery covering burned areas, across adequate numbers of spatial locations and years to sufficiently validate a large disturbance dataset [26,27]. Efforts to map a single or several fire events with Landsat often rely on fire perimeter datasets for validation (e.g., GPS delineated perimeters) [29,30,31], but high-resolution imagery has also been used for validation [32,33]. In addition, high-resolution imagery (e.g., IKONOS and GeoEye-1) has been used to directly map burned area and burn severity [34,35,36,37,38] or been paired with Landsat imagery to improve efforts to map burned area and burn severity [39,40].
Validation efforts often focus on pixel-based approaches, as they can provide easily interpretable errors of omission and commission to summarize the accuracy of a dataset [21]. However, landscape metrics have also begun to be reported for burned area products [24], and patch scale metrics, such as fire size distribution, have been calculated for global burned area products [41,42]. A number of studies have shown that measurements informing the spatial pattern of fires can both complement our understanding of fire regimes [8,43,44] and inform post-fire dynamics [45,46,47]. At broad spatial and temporal scales, the patterns and abundance of fires are controlled by continental-scale variation in climate and long-term patterns in vegetation type and successional stage [48,49,50]. However, at finer spatial and temporal scales, spatial heterogeneity in fire occurrence, extent or severity can be attributed to variation in fuels, ignition sources, topography, weather, and barriers to fire spread [44,51,52]. For example, fire orientation has been successfully related to locally dominant wind directions, which can inform efforts to place fuel breaks on the landscape [8,53,54]. An understanding of the controls on burned area extent across space and time is critical to anticipating shifts in fire regimes under climate change [55]. Detecting these finer-scale relationships, however, requires burned area inputs with adequate detail regarding within fire heterogeneity.
This study sought to validate the USGS’s Landsat BAECV (1984–2015) using an independent dataset derived from close to 300 commercial high-resolution images, collected from QuickBird-2, Worldview-2, GeoEye-1 and RapidEye across CONUS, between 2003 and 2015. Our research questions included:
(1)
How accurate is USGS’s BAECV across diverse ecoregions of the CONUS?
(2)
How does the spatial resolution of imagery influence the spatial characteristics and the within fire heterogeneity of mapped fires?
(3)
How does a validation using commercial high-resolution imagery compare with and complement a Landsat-based validation of the BAECV?

2. Methods

2.1. BAECV Product Algorithm

The BAECV algorithm is explained in detail in Hawbaker et al. [10]. The product is provided as an annual composite of all burned area identified across CONUS at 30 m resolution, and a minimum fire size of 4.05 ha (45 pixels) (https://doi.org/10.5066/F73B5X76). The algorithm used a supervised machine-learning approach and relied on the Monitoring Trends in Burn Severity (MTBS) data as the primary source of observed burned area to train the algorithm. Of the 32 years of Landsat 4, 5, and 7 data (1984–2015), corresponding MTBS data were available for all except the two most recent years (1984–2013) at the time of production (30 years). Training data points were derived from 24 of the years, while the remaining 6 years (1988, 1993, 1998, 2003, 2008 and 2013) were retained for testing and validation. The BAECV algorithm used a suite of predictor variables calculated from a dense time series of Landsat data including both single-scene, pre-fire surface conditions (e.g., 3-year lagged means and standard deviations), and change from pre-fire surface conditions. These variables were used as the inputs to train a gradient-boosted regression model [56] that uses a sequence of classification and regression tree models [57] to predict the probability that a pixel has burned in any given Landsat image. Burn classification images were generated by applying thresholds and a region-growing method to the burn probability images so that both pixels with very high burn probability (≥98%) and adjacent pixels with high burn probability (≥95%) were classified as burned.

2.2. High-Resolution Imagery Sampling Design

Burned area is a spatially rare and temporary cover type. The BAECV classified 0.4% of the CONUS as burned each year (1984–2013) [10]. Relying on high-resolution imagery to validate a rare cover type presents a unique sampling challenge. A high-resolution image (~256 to 324 km2) is <1% as large as a Landsat image (~33,300 km2 or 185 km by 180 km). A common approach is to use stratified random sampling to select image locations informed by ancillary data, such as fire density and biome, but without prior confirmation regarding fire presence or absence [22]. Although this approach may be adequate when using Landsat imagery, with high-resolution imagery it is problematic because: (1) there is no guarantee that a high-resolution image has been collected for a specific location and date; and (2) we would be unlikely to identify enough burned area without using an exceedingly large number of images. Given the labor-intensive nature of processing and manually editing classifications from high-resolution imagery, this is not feasible. Because of the shortcomings of random sampling for validating rare events, we employed disproportionate sampling, which increases the proportion of samples collected within the rare change, in this case unburned to burned [19,58]. Disproportionate sampling can be achieved by focusing on areas known to be experiencing high rates of fire events via domain knowledge [59]. The advantage of this approach is increased precision in estimating burned area accuracy and more efficient resource utilization in deriving reference data [19].
High-resolution imagery containing burned patches across CONUS were found primarily using DigitalGlobe’s (https://www.digitalglobe.com/) imagery database and secondarily Planet’s (https://www.planet.com/) imagery database, both of which are searchable by image location or time-frame, but not by features found within the imagery. We used MTBS and MODIS fire point datasets (MCD14) to identify “hot spots” of burned areas, or regions that frequently burned, and manually searched through imagery collected over these locations looking for imagery containing recently (same-year) burned patches. We selected additional search locations to stratify images geographically and across ecoregions to the best of our ability. This approach allowed us to collect enough samples of true burned area to: (1) test for errors of omission in the BAECV; and (2) maximize our precision in estimating burned area accuracy. To ensure that we also adequately documented the potential for errors of commission, or the BAECV mapping fires where the high-resolution images did not, we paired each high-resolution image containing a burned area with a nearby image, collected on a similar date, with similar vegetation types, containing no burned area. In cases in which such imagery was not available, an image with similar vegetation types, but a different date or year was used instead. We recognize that there may be inherent bias in where high-resolution images were collected over burned areas. For instance, commercial enterprises may be more likely to collect imagery near developed areas as opposed to highly remote locations. There is also inherent bias in the probability of us finding burned area imagery. For instance, larger burned areas are more easily recognized when searching through imagery databases than small burned patches.
We processed a total of 286 high-resolution images stratified across four major ecoregions within CONUS including the Arid West (45 images containing a burn, 45 with no burn), the Mountain West (33 images containing a burn, 33 with no burn), the Great Plains (34 images containing a burn, 34 with no burn), and the East (31 images containing a burn, 31 with no burn) (Figure 1). The images were distributed between GeoEye-1 (n = 9), QuickBird-2 (n = 172), Worldview-2 (n = 73) and RapidEye (n = 37). The images were also distributed between 2003 and 2015 (Table 1). For images in which the burn occurred or the dominant land cover type, as defined by the 2011 National Land Cover Database (NLCD) [60], was grassland, agriculture or hay/pasture, the gap between the high-resolution image and Landsat was limited to 5 days or less. For other cover types, we can expect evidence of the burn to persist longer; in these cases the closest cloud-free Landsat date to the date of the high-resolution image was used. The mean date gap between the high-resolution and Landsat image dates across CONUS was 8 days (Table 2). A complete list of the high-resolution and Landsat image pairs is shown in Table A1 in Appendix A.

2.3. High-Resolution Reference Dataset

The QuickBird-2, GeoEye-1 and Worldview-2 images were resampled and delivered by DigitalGlobe at 2 m resolution. The RapidEye imagery was resampled and delivered by Planet at 5 m resolution. Details regarding differences in the satellite characteristics, relative to Landsat, are shown in Table 3. All images were re-projected to a consistent projection prior to analysis and comparison. In addition, each image pair was checked for potential georeferencing-related errors prior to comparison. The high-resolution images were georeferenced to the Landsat image as necessary. The high-resolution imagery was processed in both PCI Geomatica and ENVI. Two programs were used due to changes in the availability of the atmospheric correction modules, mid-project. In PCI Geomatica, the imagery was atmospherically corrected and converted to ground reflectance using ATCOR (the Atmospheric Correction module) [61]. In ENVI imagery was atmospherically corrected and converted to ground reflectance using FLAASH [62,63]. In both programs, burned area was identified using maximum likelihood supervised classification, in which each image was trained on manually selected “burned” and “unburned” polygons. A sieve filter, in PCI Geomatica, and a low pass filter, in ENVI, were applied to reduce noise in the output images with the window size and pixel aggregation varying based on the amount of noise produced by each classification. Most images were filtered using a 5 × 5 window size. Burned area outputs were then manually edited using visual interpretation of the high-resolution imagery as well as ancillary datasets (e.g., MTBS, MODIS active fire detections) to improve confidence in edits, as relevant.
Commercial high-resolution satellites lack a short-wave infrared (SWIR) band, which has been found to be useful in detecting burned areas [64]. However, we found that by training and classifying images individually we were able to successfully distinguish burned area from non-burned area using both four-band satellites, such as GeoEye-1 and QuickBird-2, five-band satellites, such as RapidEye, as well as eight-band satellites, such as Worldview-2. Variation in biomass combustion levels within a given image (e.g., ash, char, soot) can result in spectral diversity across areas classified as burned. This potential source of error was minimized by classifying multiple classes of burned area as needed based on combustion levels, then combining these classes into a single burned area class. Another challenge regarding using commercial high-resolution satellites is that it is often not feasible or possible to calculate burned area from pre- and post-fire image pairs. For most fires burned area extent from a single post-fire image was visually and spectrally interpretable, however for low-severity surface fires it is possible that a pre-fire image would have improved the identification of these areas. The largest source of error in mapping burned area with high-resolution images was in mapping low severity or ground fires in which tree crowns remained green. Because of the fine spatial resolution, individual trees or small clusters of trees were often classified as unburned, artificially elevating within fire heterogeneity. This source of error was reduced by filtering and manual editing where feasible. This approach removed single trees classified as unburned within a continuous burned area, but left larger, more continuous patches of green trees classified as unburned. This approach retained unburned patches, but may have underestimated low, severity surface fires.

2.4. Pixel-Based Validation of the BAECV

The annual BAECV product represents a temporal aggregation of individual Landsat images. If an individual pixel was classified as burned in any of the Landsat images from that year, it was classified as burned within the annual BAECV product. The high-resolution images, in contrast, represented burned area extent at a specific point in time and were therefore not directly comparable to the annual BAECV product. Because of this, the high-resolution images were compared to the closest Landsat image processed by the BAECV. This date was sometimes prior to and sometimes following the high-resolution image collection date. In a few cases Landsat did not map the burned area in any of the nearby dates, but the fire was mapped in the annual burned area raster. In these cases, the annual burned area raster was compared to the classified high-resolution image. The BAECV (30 m resolution) was up-sampled to match the resolution of the reference dataset (2 m resolution) for a pixel to pixel comparison [22]. This approach allowed for no loss of information provided by the finer-resolution reference imagery. The images classified from RapidEye (5 m resolution) were also up-sampled to 2 m resolution for consistency purposes. Accuracy metrics calculated included overall accuracy, omission error, commission error, Dice coefficient, and relative bias. Omission and commission errors were calculated for the category “burned” [22,25]. The Dice coefficient is the conditional probability, presented as a percentage, that if one classifier (product or reference data) identifies a pixel as burned, the other one will as well, and therefore integrates omission and commission errors [65,66]. The relative bias provides the proportion, presented as a percentage, that burned area is under or overestimated relative to the total burned area of the reference product [22].
Spatial autocorrelation in accuracy assessments is introduced when reference datasets are created over continuous units in space (e.g., an image). Reporting standard errors from pixel-counts can therefore underrepresent error because of the influence of spatial autocorrelation on error [67]. To account for the influence of stratification and clustering, the pixel-level accuracy metrics were calculated for each high-resolution image, individually. Standard errors, reported for accuracy metrics by ecoregion and CONUS, were then estimated to account for the stratified sampling design. The general estimator for each accuracy metric ( R ^ ) was defined as the stratified combined ratio estimator [68]:
R ^ = ∑ h = 1 H K h y h ¯ ∑ h = 1 H K h x h ¯
where H is the number of strata, Kh is the size of stratum h, y h ¯ and x h ¯ are the sample means of yt and xt of stratum h, and yt and xt are the numerator and denominator of each accuracy metric equation, respectively [22]. The estimated variance of R ^ was in turn defined as:
V ^ ( R ^ ) = 1 X 2 ^ ^ ∑ h = 1 H K h 2 k h ( k h − 1 ) ∑ t ∈ h d t 2
where kh is the number of images sampled in stratum h and X ^ and dt are defined as:
X ^ = ∑ h = 1 H K h x h ¯
d t = ( y t − y h ¯ ) −   R ^ ( x t − x h ¯ )
Lastly, the standard error was calculated as:
S E =   V ^ ( y ¯ s t )
Accuracy metrics were presented by ecoregion the distribution of which is shown in Figure 1. Ecoregions were based on a modified version of the U.S. Environmental Protection Agency’s Level I ecoregions and included the Mountain West, Arid West, Great Plains and East [69] (Figure 1). The relative abundance of NLCD land cover types within each of the ecoregions are shown in Table 4. The Arid West, in the southwestern United States, is dominated by shrub/scrub, while the Mountain West, in the northwestern United States, is dominated by evergreen forest and shrub/scrub. The Great Plains, in the central United States, is dominated by grasslands and cultivated crops, and the East, extending across the eastern part of the United Sates, is the most diverse, dominated by deciduous forest, evergreen forest, pasture/hay, and woody wetlands (Table 4). Accuracy metrics were scaled from ecoregions to CONUS using our current understanding regarding the relative distribution of burned area by ecoregion. This distribution was averaged from three independent sources for burned area: (1) the USGS Geospatial Multi-Agency Coordination (GeoMAC) (2000–2015); (2) MTBS (1984–2014); and (3) MODIS MCD45 (2000–2015) (Table 5). GeoMAC is a mapping application designed to allow fire managers to access online maps of current fire locations and perimeters across the United States. The perimeters represent verified fires where U.S. federal resources or agencies were involved, but under-map fires that occurred on private land (https://www.geomac.gov/). The averaged proportion of burned area by ecoregion was used to weight the ecoregion accuracy metrics to calculate CONUS-based accuracy metrics. Linear regressions were also performed, comparing the amount of burned area mapped by the reference dataset to the amount of burned area mapped by the BAECV product (ha per image).

2.5. Patch-Based Evaluation of the BAECV

Landscape metrics were calculated for each individual fire event that were mapped by both the high-resolution imagery and the BAECV (n = 91). This sample size was smaller than the number of images containing a burn (n = 143) because some fire events extended across multiple high-resolution images and agricultural burns were excluded as these tend to be rectangular and uniform in shape. For fire events that were also mapped by MODIS (MCD45), landscape metrics were calculated for the MODIS version of the fire event as well. The inclusion of MODIS was meant to help contextualize potential observed differences between the BAECV and the high-resolution burned area. The burned area as mapped by the high-resolution imagery, Landsat and MODIS were clipped to the high-resolution image extent. Burned areas classified from RapidEye satellites were up-sampled to 2 m, for consistency purposes, while Landsat and MODIS (MCD45) were run at 30 m and 500 m, respectively. Landscape metrics were calculated using FRAGSTATS [70]. Landscape metrics were selected that have been used in fire-related studies and that characterize the areal distribution, configuration and arrangement of burned patches for a given fire event. These metrics included:
  • Patch Density [9,71]
  • Area-Weighted Mean Patch Size [9,43,52]
  • Area-Weighted Mean Perimeter-Area Ratio [24]
  • Edge Density [71]
  • Landscape Shape Index [8,9,71].
Patch Density (PD) equals the number of burned area patches divided by the total landscape area. PD is maximized when every cell is a separate patch, but since it accounts for landscape area, is ultimately constrained by the pixel size [70]. Area-Weighted Mean Patch Size (AM) is the sum of the area of each burned patch multiplied by the proportional abundance of the patch, so that larger patches are weighted more heavily. The perimeter-area ratio is a simple measure of shape complexity. The Area-Weighted Perimeter-Area Ratio (PARA) is calculated the same as AM, except that instead of patch area, the ratio of the patch perimeter to patch area is used. Edge Density (ED) equals the total length of edge (or perimeter) of burned patches divided by the total landscape area and reflects both the relative abundance of burned area as well as the aggregation of burned patches. The Landscape Shape Index (LSI) provides a measure of burn patch aggregation and equals the total perimeter of all burned area patches, divided by the minimum total perimeter for a maximally aggregated class, so that an LSI of 1 occurs when a landscape consists of a single square patch [70]. Paired t-tests were used to evaluate if the landscape metrics of the fire events varied significantly between the high-resolution imagery, Landsat and MODIS. Data were normalized using the log function and found to comply with normal data distributions using the Shapiro–Wilks test.

3. Results

3.1. Pixel-Level Validation of the BAECV

Overall accuracy was high (≥97%) across all four ecoregions due to the relative ease of identifying unburned as unburned and the relatively rare occurrence of burned area, even when disproportionately sampling images containing a burned area (Table 2). Errors of omission and commission for burned area when averaged across CONUS were 22% and 48%, respectively. Errors were unevenly distributed across the ecoregions. Errors were relatively low in the Arid West (14% and 36% omission and commission error, respectively) and Mountain West (22% and 39% omission and commission error, respectively), while errors of commission were highest in the Great Plains (70% commission error) and high but balanced in the East (46% and 48% errors of omission and commission, respectively) (Table 2). Despite the frequency of small grassland and agricultural fires in the Great Plains, errors of omission were very low (13% error of omission). Most fires in the East occur in forest, but fires are also common within agricultural and wetland areas, two cover types which are particularly difficult to map fire within. Wetland areas are difficult because depending on the water level, a burn event can result in a change from vegetated to open water, while agricultural areas experience frequent shifts between vegetated and non-vegetated. The BAECV had a consistent bias in over-estimating burned area extent in the other three ecoregions (Great Plains, Arid West and Mountain West). Across the Arid West and Mountain West the BAECV moderately overestimated burned area (relative bias of 34% and 27%, respectively), while in the Great Plains the BAECV greatly overestimated burned area (194% relative bias). Despite a consistent bias in the BAECV over-estimating burned area extent, relative to the high-resolution imagery used as reference data, the correlation between burned area identified by the reference dataset and the BAECV was strong (r2 = 0.83, p < 0.01) (Figure 2).

3.2. Landscape Metrics Comparison

For non-agricultural fires, the BAECV mapped 91 of 93 fire events identified within the high-resolution imagery, meaning that much of the reported error was due to disagreement regarding the mapping of within-fire heterogeneity. An example of one such fire event is shown in Figure 3. For each fire event we calculated the ratio of the total area mapped as burned by the high-resolution imagery compared to the total area mapped as burned by the BAECV. Ratios ranged from 0.005 (large commission by BAECV) to 22.5 (large omission by BAECV) with a median ratio value of 0.79, meaning that the BAECV on average, mapped a burned area 21% larger than the high-resolution imagery did. Several examples comparing the reference burned area and BAECV are shown in Figure 4. In each of the examples, the disagreement was regarding mapping within-fire heterogeneity. The high-resolution imagery tended to be more conservative, mapping fires in a more “patchy” manner relative to the BAECV. These examples were selected to provide visual examples of what CONUS-scale error rates look like for individual fire events (Figure 4).
Using FRAGSTATS, we observed significant differences in landscape metrics when fire events were mapped at each of the three spatial resolutions (2 m, 30 m and 500 m) (Table 5). As the resolution became finer, patch and edge density increased significantly and the area-weighted mean patch size decreased significantly as a fire event was mapped using more patches and became less aggregated. The shape complexity, evaluated by the Landscape Shape Index, increased significantly, and we saw more edge and less internal area, or more disaggregation or dispersal, as indicated by higher area-weighted perimeter-area ratios (Table 6).

4. Discussion

Emerging Landsat science products are challenging to validate as no finer resolution source of imagery has been regularly collected over the entire Landsat 4, 5, and 7 record (1984 to present). However, commercial, high-resolution imagery has now been collected frequently enough for 12 of the 32 years of the BAECV record (2003–2015) to begin allowing for validation of Landsat products with finer spatial resolution reference imagery. Although QuickBird-2 was launched in 2001, imagery collection appears limited until 2003 onward. Validating with high-resolution imagery, however, faces several challenges. A traditional stratified sampling strategy is not feasible because of the: (1) non-regular spatial and temporal data collection of high-resolution imagery; (2) small image extent relative to Landsat (<1% the size); and (3) burned area occurring as a rare cover type. To overcome these challenges, we used a disproportionate sampling strategy to ensure that enough burned pixels were collected. We also integrated images lacking burned areas to ensure our ability to test for errors of commission. Using this approach allowed for a validation assessment of the BAECV, particularly when complemented by a Landsat-based validation that can provide a validation over greater temporal and spatial areas relative to high-resolution imagery [18].
The accuracy of burned area maps and products often depends on the size and diversity of the area mapped. When burned area algorithms are optimized for site level performance, the accuracy can exceed 95% [29,72]. As the target area expands, accuracy often begins to decrease (e.g., 15% to 30% error for burned area) due to variance imposed by local factors [33,73,74]. Disturbances (fires, clearcut and insect mortality) have been mapped with Landsat imagery across forested portions of CONUS. The corresponding validations relied on manual interpretation of Landsat images and calculated errors of omission and commission averaging 40–45% and 20–30%, respectively [26,27]. Validation for these efforts was supplemented using Forest Service Inventory and Analysis (FIA) plots and showed errors of omission and commission averaging 36% (0 to 100%) and 64% (26 to 100%), respectively. Boschetti et al. (2015) mapped burned areas from Landsat for a single year across much of the western United States. Relative to MTBS perimeters, the burned area extent showed 51% omission error and 14% commission error. The spectral diversity within burned patches increases as the vegetation and cover type diversity increases, as can be expected for national and global scale efforts. This makes achieving high levels of accuracy at these larger spatial extents considerably more challenging. At the global scale, accuracy of existing global burned area products, derived from coarser resolution imagery, show relatively weak accuracy. Errors of omission for such products have been documented to range from 51% to 93% for omission error and from 36% to 94% for commission error [23,24]. Error is even higher for specific vegetation types. For example, although the MODIS burned area product (MCD45) performed well relative to other global burned area products, it performed poorly in temperate forest (99% omission, and 95% commission), which is a major biome across CONUS (Figure 1) [23].
Validating the BAECV with high-resolution imagery, we documented errors of omission and commission that averaged 22% and 48%, respectively, across CONUS. These error rates are comparable to a previous BAECV validation effort using manually edited Landsat images, which documented errors of omission and commission that averaged 42% and 33%, respectively, across CONUS [18]. The error rates for the Landsat BAECV were lower than most accuracy statistics reported for coarser resolution global burned area products and showed lower rates of omission but higher rates of commission than mapping efforts using Landsat imagery but constrained to forested cover types. In the Landsat-based validation, the BAECV errors of omission exceeded errors of commission in all four ecoregions [18]. In this validation effort, however, errors of commission exceeded errors of omission in all four ecoregions. The difference in error distribution can be potentially attributed to: (1) region-growing functions included in generating both the BAECV [10] and Landsat reference dataset (using the program Burned Area Mapping Software (BAMS)) [73]; and (2) an intrinsic ability to map more fire heterogeneity as the spatial resolution of the input imagery becomes finer. A potential source of this increased patchiness or fire heterogeneity mapped by the high-resolution imagery (e.g., Figure 4) could have been live crowns mapped incorrectly as unburned in areas with surface burns, which would have incorrectly elevated errors of commission for the BAECV; however, much of the increased heterogeneity can be attributed to the high-resolution images correctly mapping bare soil, rock and unburned patches of vegetation within the burn perimeter as unburned. Many of these patches within the BAECV were classified as burned when a region-growing function was applied to areas within and near the burned area [10].
One of the potential factors influencing the quality of the high-resolution burned area extent was mapping burned area extent from single images instead of pre- and post-fire image pairs, as is common when mapping burned area with Landsat imagery. Some sources of high-resolution imagery have begun to collect imagery at regular intervals using networks of micro-satellites (e.g., Dove and RapidEye). This trend will make it easier, in the future, to map burned area extent from pre- and post-image pairs. However, an added complication is the need for pre- and post-fire image pairs to show consistent and highly accurate positional accuracy in order to ensure that the same pixels are being subtracted from one another. While this requirement also holds for Landsat imagery, the resolution of Landsat (30 m) relative to Worldview-2 (2 m) for example, necessitates a much higher positional accuracy for high-resolution imagery relative to Landsat imagery. In working extensively with multiple sources of high-resolution imagery we have found that this area of image quality needs improvement. For example, DigitalGlobe’s Standard Level 2A imagery products use a 90 m DEM in georeferencing, which resulted in large inaccuracies in mountainous areas and required extensive manual georeferencing on our end so that the high-resolution image could be compared to the BAECV. A potential work-around is to acquire Level 1 imagery products (prior to georeferencing) and apply a 10 m DEM; however even this DEM resolution may not be adequate to compare two 2 m resolution images. This source of error will need to be reduced before pre- and post-fire image pairs can be used to map burned area from high-resolution imagery.
One of the most prominent differences between the previous validation effort [18] and the high-resolution based validation effort was in the Great Plains. In the previous effort we saw high errors of both omission error and commission error for burned area (63% and 59%, respectively); in contrast, in this analysis we observed high rates of commission error for burned area (70%), but very low rates of omission error for burned area (13%). In a visual assessment of product-reference image pairs within the region, we found that the BAECV reliably detected and mapped small agricultural fires, but frequently mapped tilled areas or areas with low vegetation as burned. This is shown in Figure 5, where an area with little live vegetation but no clear burn signal (purple circle) was classified as burned by the BAECV but not by the RapidEye-3 image. This error was common and resulted in an over-estimation of burned area across the Great Plains. However, we note that the Landsat collection interval (8–16 days) relative to the persistence of a burn signal following a grassland or agricultural fire event also likely resulted in substantial omission errors for the BAECV across the Great Plains, meaning the true bias of the BAECV total burned area across the Great Plains remains unknown.
One of the greatest challenges of mapping agricultural fires is the lack of training data and our limited ability to visually distinguish burned from bare soil. MODIS active fire points, when present, can help confirm that fire occurred in the area but the spatial uncertainty due to the spatial resolution of MODIS makes it challenging to confidently attribute active fire points to a specific field. The equal distribution of error between omission and commission when comparing the BAECV and the Landsat reference dataset suggested that both the BAECV and the reference dataset struggled to appropriately distinguish burned from tilled or low vegetation soil. The difficulty for the BAECV and Landsat reference dataset may have been exacerbated by change-detection approaches. Given the frequent changes in site condition (green vegetation, non-photosynthetic vegetation, burned, tilled) areas can show spectral change not related to a fire event. Even visually interpreting high-resolution imagery was challenging. In Figure 6, we followed a small sub-section of an agricultural area from 5 March to 13 April 2016 using the RapidEye satellites. Over the 39 days, we can see several fields change visually, and it was not always clear, given a single image, as to the current state or trajectory of a given field.
Another major challenge in validating with high-resolution imagery was the variable date gap between the date the high-resolution image was collected and the date the Landsat image was collected. A close date match between the high-resolution images and the Landsat images was critical to minimize false error due to changes in land condition (e.g., new fires occurring, vegetation recovery) between the product and reference dataset. The mean image date gap across comparisons was eight days (Table 2), but in some cases the gap was larger. It was time-consuming to find enough high-resolution images with burned areas visible within the image. Although Landsat images were typically available every eight days for 2003 to 2011, as Landsat TM data collection ended in 2011, only BAECV images processed from Landsat ETM+ with scan line off were available for 2012–2015 (every 16 days). The BAECV has not yet incorporated Landsat-8 imagery. Cloud cover also frequently increased the time before an image was available. Figure 7 shows two fire events mapped using Worldview-2 imagery in areas dominated by a mix of forest, scrub and grassland relative to the two closest Landsat date matches (−38 and −35 days, respectively, and +10 and +13 days, respectively). In both cases, the burned area extent as mapped by BAECV shrank between the two dates, likely due to vegetation regrowth (e.g., grasses and herbaceous plants), while the Worldview-2 burned area extent represented a mid-point both in time as well as burned area extent between the two BAECV dates (Figure 7). These examples demonstrate potential false error due to a time gap between the product and reference datasets. In the Arid West and Mountain West, we can expect that the primary source of false error will be due to vegetation recovery; however, in the Great Plains, it is common for multiple small agricultural and grassland fire events to occur within a given high-resolution image. This meant that new fires occurring could also present false errors. In Figure 5, a yellow circle shows an agricultural fire that occurred after the Landsat image was collected (DOY 120), but before the RapidEye-3 image was collected (DOY 122), so that the burn was classified as a false error of omission in the BAECV. To balance having enough image comparisons with the need to minimize false sources of error, we applied a maximum date gap of five days to images which were dominated by grassland or agricultural land cover types; however, false error due to date gaps remains a challenge for using high-resolution imagery to validate a Landsat science product.
This analysis also raised the question, what is the ideal resolution to map (and validate) fires? Traditionally, products are validated using a source of imagery finer in resolution than the product [19]; however, it is also critical to consider the size and characteristics of the object being mapped, in this case burned areas. The high-resolution imagery had difficulty, for example, with unburned tree crowns within a continuously burned area. Alternatively, the BAECV would commonly map areas within a fire that showed little to no evidence of being burned in the high-resolution imagery. These were often bare soil patches or patches with little to no vegetation. The landscape metrics comparison demonstrated that the spatial resolution can be expected to influence within-fire heterogeneity. The BAECV mapped significantly more fire heterogeneity relative to MODIS, which suggests an improvement in data quality as products become finer in resolution. However, the high-resolution imagery also mapped significantly more fire heterogeneity relative to the BAECV. The region-growing function built into the BAECV to reduce noise, combined with the spatial resolution of the reference data, may explain the switch from a lower commission error, relative to omission error, in the first validation effort [18] to a lower omission error, relative to the commission error, in this high-resolution based effort to validate the BAECV. The consistent differences observed in fire heterogeneity and shape aggregation/disaggregation based on the spatial resolution of the imagery may be more or less relevant depending on the application of the burned area dataset. Differences in total area burned can influence estimates of total greenhouse gas emissions [6,7], while differences in distance to nearest unburned patch will influence predictions of recovery rate due to seed dispersal [75,76,77] and could be relevant to guiding ground-based recovery measures. Alternatively, the additional detail may be less relevant in deriving relationships between burned area and climate, as most climate and other geographical data layers are rarely offered at a resolution finer than 30 m.

5. Conclusions

The validation of Landsat science products, such as the USGS’s BAECV is an essential component of product generation, but challenging to accomplish given the long time span and moderate spatial resolution over which these products are generated. We used commercial high-resolution imagery to validate the BAECV. Although we were unable to validate the first portion of the product (1984–2002) using this source of imagery, by utilizing a disproportionate sampling strategy and close to 300 high-resolution images across CONUS (2003–2015) we were able to generate accuracy statistics with which to evaluate the BAECV. The BAECV performed well across the western United States (Arid West and Mountain West), but saw large errors of commission in the Midwest or Great Plains, where areas with low vegetation were frequently mapped as burned. Accuracy was also poor in the eastern United States, where difficulty in accurately mapping agricultural and wetland fires may have contributed. The BAECV showed a strong linear correlation in magnitude burned relative to high-resolution images but tended to overestimate burned area extent. This source of commission was likely due to the increase in fire heterogeneity and burned area disaggregation observed in mapping fires at a finer spatial resolution. Further, region-growing methods incorporated into the BAECV algorithm can help reduce within-fire noise [10], but may have exacerbated errors of commission when using high-resolution imagery to validate. Commercial high-resolution imagery has been commonly used to validate maps of individual fire events [32,33], but has not been widely utilized for national-scale products. However, the increasing number of satellites collecting high-resolution imagery and rapid improvements in the frequency with which imagery is being collected means increased opportunity to utilize these sources of imagery in validation efforts.

Acknowledgments

We thank the USGS, Climate and Land Use Mission Area, Land Remote Sensing Program for providing funding to support this research. We are grateful to Clifton Burt and Marena Gilbert for their assistance in processing data, and to Zhuoting Wu, Eugene Schweig and anonymous reviewers for providing comments on the manuscript. Any use of trade, firm, or product names is for descriptive purposes only and does not imply endorsement by the U.S. Government.

Author Contributions

All authors made major and unique contributions. Melanie K. Vanderhoof is the primary author who designed the study, led the acquisition and processing of the high-resolution imagery and wrote the manuscript. Nicole Brunner and Todd J. Hawbaker assisted with the study design and data interpretation. Yen-Ju G. Beal processed the BAECV comparison images and was key in the data management process.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Table
Table A1. All high-resolution images and the corresponding Landsat images used to validate the USGS Burned Area Essential Climate Variable (BAECV). W wetland: woody wetland; EH wetland: emergent herbaceous wetland; TM: Thematic Mapper; ETM+ Enhanced Thematic Mapper Plus; Mtn: mountain.
Table A1. All high-resolution images and the corresponding Landsat images used to validate the USGS Burned Area Essential Climate Variable (BAECV). W wetland: woody wetland; EH wetland: emergent herbaceous wetland; TM: Thematic Mapper; ETM+ Enhanced Thematic Mapper Plus; Mtn: mountain.
RegionStateSensorImages (Burned and Not Burned)YearImage DOY (High-Resolution)Image DOY (Landsat)Gap (Days)Landsat SensorLandsat Path/RowDominant Landcover
Arid WestAZWorldview-22201116417410ETM+p35r37Evergreen
Arid WestAZQuickBird-2220111781824TMp35r37Evergreen
Arid WestAZWorldview-2120132102111ETM+p35r37Evergreen, shrub/scrub
Arid WestAZWorldview-2520141521502ETM+p35r37Evergreen, shrub/scrub
Arid WestAZWorldview-2320121741751ETM+p37r36Evergreen
Arid WestAZ/NMWorldview-2620122752732ETM+p35r37Evergreen, shrub/scrub
Arid WestCAGeoEye-1120092692645TMp43r35Shrub/scrub, grassland
Arid WestCAGeoEye-1120092582646TMp43r35Shrub/scrub, evergreen
Arid WestCAWorldView-2320141991990ETM+p42r35Agriculture
Arid WestCAQuickBird-23200635132922TMp42r36Shrub/scrub
Arid WestCAQuickBird-2420063333294TMp42r36Shrub/scrub
Arid WestCAQuickBird-23200923024111TMp42r36Shrub/scrub
Arid WestCAQuickBird-23200831829424TMp43r35Shrub/scrub, evergreen
Arid WestCAQuickBird-29200832330914ETM+p44r33Shrub/scrub
Arid WestNMQuickBird-2120031621697TMp34r37Evergreen forest
Arid WestNMQuickBird-2420102011965ETM+p34r37Shrub/scrub, evergreen
Arid WestNMQuickBird-2120122212276ETM+p33r35Shrub/scrub, evergreen
Arid WestNMWorldview-21201123824911ETM+p32r35Shrub/scrub, evergreen
Arid WestNVQuickBird-21020052432412TMp39r34Shrub/scrub, grassland
Arid WestNVQuickBird-2220052152096TMp39r34/p39r35Shrub/scrub
Arid WestNVQuickBird-2120113253294ETM+p43r33Developed
Arid WestORQuickBird-2320112482524TMp45r29Shrub/scrub
Arid WestWAQuickBird-211200327124625TMp45r28Shrub/scrub
Arid WestWAQuickBird-2320032452461TMp45r28Shrub/scrub
Arid WestWAGeoEye-1220092612547ETM+p45r28Shrub/scrub, grassland
Arid WestWAQuickBird-21200326824622TMp45r27Shrub/scrub
Arid WestWAQuickBird-2420092352383ETM+p45r28Shrub/scrub, agriculture
Mtn WestCAWorldView-2320132362288ETM+p42r35Mixed forest
Mtn WestCAWorldView-22201226828113ETM+p43r33Evergreen, shrub/scrub
Mtn WestCAQuickBird-23201024327431TMp44r33Evergreen, grassland
Mtn WestCAWorldView-2220102122111ETM+p43r33Evergreen, shrub/scrub
Mtn WestCAQuickBird-21200928927118TMp44r33Shrub/scrub
Mtn WestCAQuickBird-21200718419511TMp43r33Shrub/scrub, evergreen
Mtn WestCAQuickBird-23200925823919TMp44r34Evergreen
Mtn WestCOWorldview-2120121861959ETM+p33r33Evergreen
Mtn WestCOWorldView-22201216921849ETM+p34r32Evergreen
Mtn WestCOQuickBird-2120121271292ETM+p35r32Shrub/scrub
Mtn WestCOQuickBird-22201332429331ETM+p33r33Evergreen
Mtn WestCOQuickBird-2220101421486ETM+p34r32Evergreen, shrub/scrub
Mtn WestCOWorldview-2220042821831ETM+p35r32Shrub/scrub
Mtn WestCOWorldView-21201123824810TMp33r34Shrub/scrub
Mtn WestCOWorldview-2120102552605ETM+p34r32Evergreen
Mtn WestIDWorldView-2220112752723TMp41r28Evergreen
Mtn WestIDQuickBird-23200625627418TMp41r29Evergreen
Mtn WestIDQuickBird-22200730024555TMp41r29Evergreen
Mtn WestIDQuickBird-2220032252187TMp41r30Shrub/scrub, evergreen
Mtn WestMTQuickBird-22200628727413TMp41r28Shrub/scrub, evergreen
Mtn WestMTQuickBird-2220062742740TMp41r28Evergreen, grassland
Mtn WestMTQuickBird-2120062562506ETM+p41r28Evergreen
Mtn WestMTWorldView-25201129127219TMp41r28Evergreen
Mtn WestNMQuickBird-21200621924223ETM+p33r34Shrub/scrub, grassland
Mtn WestNMWorldView-2220103042011 annual-TM/ETM+p33r35/p34r35Evergreen
Mtn WestNMQuickBird-25201027626115TMp33r34Evergreen, grassland
Mtn WestORWorldView-24201126625214TMp45r28/p45r29Shrub/scrub
Mtn WestWAQuickBird-2620032402466TMp45r28Evergreen
Mtn WestWYGeoEye-11201218520217ETM+p34r31Shrub/scrub, evergreen
Mtn WestWYQuickBird-2120052572543TMp34r31Shrub/scrub, evergreen
Great PlainsKSQuickBird-22200691883ETM+p27r33Agriculture, grassland
Great PlainsKSQuickBird-23200691883ETM+p27r33Agriculture, grassland
Great PlainsKSQuickBird-2420041071070TMp27r33/p27r34Agriculture, grassland
Great PlainsKSQuickBird-22200471674ETM+p27r34Agriculture
Great PlainsKSWorldview-21201077825TMp28r33Grassland
Great PlainsKSWorldview-2120102942904TMp28r33Grassland, agriculture
Great PlainsTXQuickBird-24201176679TMp30r36Shrub/scrub, grassland
Great PlainsSDQuickBird-2220111511521TMp33r30Evergreen, grassland
Great PlainsMNQuickBird-2620031011021TMp29r27Agriculture
Great PlainsMNWorldview-2120121071103ETM+p30r26Agriculture
Great PlainsMNWorldview-2620151461433ETM+p29r26/p29r27Grassland
Great PlainsMNWorldview-2320152002077ETM+p29r27Agriculture
Great PlainsKSRapidEye-2620091731741ETM+p29r33Agriculture
Great PlainsSDRapidEye-2220141552014 annual-ETM+p33r30Evergreen
Great PlainsKSRapidEye-2920131371370ETM+p29r33Agriculture, grassland
Great PlainsKSRapidEye-2132014991012ETM+p28r33Grassland, agriculture
Great PlainsKSRapidEye-2320151221202ETM+p28r34Grassland, agriculture
EastFLWorldview-2120112582535ETM+p15r42EH wetlands
EastFLQuickBird-21201133431420TMp15r41W wetland, agriculture
EastFLQuickBird-2620053273292TMp15r42Agriculture, EH wetlands
EastFLQuickBird-2320073383435ETM+p15r42EH wetlands
EastFLQuickBird-2620121451472ETM+p17r39W wetland
EastFL/GAQuickBird-21200432234927ETM+p17r39Evergreen, W wetland
EastFL/GAWorldview-29201432419ETM+p17r39W wetland, evergreen
EastGAQuickBird-23200711914122TMp17r38/p17r39Shrub/scrub, W wetland
EastLAGeoEye-1120091751794TMp24r39EH wetlands
EastLAGeoEye-1320112162171TMp24r39EH wetlands
EastLARapidEye-2420142772814ETM+p24r39EH wetlands
EastSCQuickBird-21320091111074TMp16r37Agriculture, W wetlands
EastSCQuickBird-2320081121219TMp16r37W wetland
EastSCQuickBird-2420081531530TMp16r37W wetland
EastWVQuickBird-2420071291323TMp18r33Mixed forest

References

  1. Global Climate Observing System. Implementation Plan for the Global Observing System for Climate in Support of the UNFCCC (GCOS-92 (ES), WMO/TD No. 1244). Available online: https://www.wmo.int/pages/prog/gcos/Publications/gcos-92_GIP_ES.pdf (accessed on 3 October 2016).
  2. Bond, W.J.; Woodward, F.I.; Midgley, G.F. The global distribution of ecosystems in a world without fire. New Phytol. 2005, 165, 525–538. [Google Scholar] [CrossRef] [PubMed]
  3. Goetz, S.J.; Bunn, A.G.; Fiske, G.J.; Houghton, R.A. Satellite-observed photosynthetic trends across boreal North America associated with climate and fire disturbance. Proc. Natl. Acad. Sci. USA 2005, 102, 13521–13525. [Google Scholar] [CrossRef] [PubMed]
  4. Conard, S.G.; Sukhinin, A.I.; Stocks, B.J.; Cahoon, D.R.; Davidenko, E.P.; Ivanova, G.A. Determining effects of area burned and fire severity on carbon cycling and emissions in Siberia. Clim. Chang. 2002, 55, 197–211. [Google Scholar] [CrossRef]
  5. Bond-Lamberty, B.; Peckham, S.D.; Ahl, D.E.; Gower, S.T. Fire as the dominant driver of central Canadian boreal forest carbon balance. Nature 2007, 450, 89–92. [Google Scholar] [CrossRef] [PubMed]
  6. Palacios-Orueta, A.; Chuvieco, E.; Parra, A.; Carmona-Moreno, C. Biomass burning emissions: A review of models using remote-sensing data. Environ. Monit. Assess. 2005, 104, 189–209. [Google Scholar] [CrossRef] [PubMed]
  7. Randerson, J.T.; van der Werf, G.R.; Collatz, G.J.; Giglio, L.; Still, C.J.; Kasibhatla, P.; Miller, J.B.; White, J.W.C.; DeFries, R.S.; Kasischke, E.S. Fire emissions from C3 and C4 vegetation and their influence on interannual variability of atmospheric CO2 and δ13CO2. Glob. Biogeochem. Cycles 2005, 19, 1–13. [Google Scholar] [CrossRef]
  8. Parisien, M.; Peters, V.; Wang, Y.; Little, J.; Bosch, E.; Stocks, B. Spatial patterns of forest fires in Canada, 1980–1999. Int. J. Wildland Fire 2006, 15, 361–374. [Google Scholar] [CrossRef]
  9. Hayes, J.J.; Robeson, S.M. Relationships between fire severity and post-fire landscape pattern following a large mixed-severity fire in the Valle Vidal, New Mexico, USA. For. Ecol. Manag. 2011, 261, 1392–1400. [Google Scholar] [CrossRef]
  10. Hawbaker, T.J.; Vanderhoof, M.K.; Beal, Y-J.; Takacs, J.D.; Schmidt, G.; Falgout, J.; Brunner, N.; Caldwell, M.; Dwyer, J. An automated approach to identify burned areas in Landsat images. Remote Sens. Environ. 2017, 198, 504–522. [Google Scholar] [CrossRef]
  11. Roy, D.P.; Boschetti, L.; Justice, C.O.; Ju, J. The collection 5 MODIS burned area product—Global evaluation by comparison with the MODIS active fire product. Remote Sens. Environ. 2008, 112, 3690–3707. [Google Scholar] [CrossRef]
  12. Giglio, L.; Lobada, T.; Roy, D.P.; Quayle, B.; Justice, C.O. An active-fire based burned area mapping algorithm for the MODIS sensor. Remote Sens. Environ. 2009, 113, 408–420. [Google Scholar] [CrossRef]
  13. Mouillot, F.; Schultz, M.G.; Yue, C.; Cadule, P.; Tansey, K.; Ciais, P.; Chuvieco, E. Ten years of global burned area products from spaceborne remote sensing—A review: Analysis of user needs and recommendations for future developments. Int. J. Appl. Earth Obs. Geoinf. 2014, 26, 64–79. [Google Scholar] [CrossRef]
  14. Podur, J.; Martell, D.L.; Csillag, F. Spatial patterns of lightning-caused forest fires in Ontario, 1976–1998. Ecol. Modell. 2002, 164, 1–20. [Google Scholar] [CrossRef]
  15. Miller, J.D.; Safford, H.D.; Crimmins, M.; Thode, A.E. Quantitative evidence for increasing forest fire severity in the Sierra Nevada and Southern Cascade Mountains, California and Nevada, USA. Ecosystems 2009, 12, 16–32. [Google Scholar] [CrossRef]
  16. Whitman, E.; Battlori, E.; Parisien, M.A.; Miller, C.; Coop, J.D.; Krawchuk, M.A.; Chong, G.W.; Haire, S.L. The climate space of fire regimes in north-western North America. J. Biogeogr. 2015, 42, 1736–1749. [Google Scholar] [CrossRef]
  17. Stroppiana, D.; Bordogna, G.; Carrara, P.; Boschetti, M.; Boschetti, L.; Brivio, P.A. A method for extracting burned areas from Landsat TM/ETM+ images by soft aggregation of multiple Spectral Indices and a region growing algorithm. ISPRS J. Photogramm. Remote Sens. 2012, 69, 88–102. [Google Scholar] [CrossRef]
  18. Vanderhoof, M.K.; Brunner, N.; Beal, Y.J.G.; Hawbaker, T.J. Validation of the USGS Landsat burned area essential climate variable (BAECV) across the conterminous United States. Remote Sens. Environ. 2017, 198, 393–406. [Google Scholar] [CrossRef]
  19. Strahler, A.H.; Boschetti, L.; Foody, G.M.; Friedl, M.A.; Hansen, M.C.; Herold, M.; Mayaux, P.; Morisette, J.T.; Stehman, S.V.; Woodcock, C.E. Global Land Cover Validation: Recommendations for Evaluation and Accuracy Assessment of Global Land Cover Maps; European Commission Joint Research Centre EUR: Ispra, Italy, 2006; p. 22156. [Google Scholar]
  20. Roy, D.P.; Frost, P.G.H.; Justice, C.O.; Landmann, T.; Le Roux, J.L.; Gumbo, K.; Makungwa, S.; Dunham, K.; Du Toit, R.; Mhwandagara, K.; et al. The Southern Africa Fire Network (SAFNet) regional burned-area product-validation protocol. Int. J. Remote Sens. 2005, 26, 4265–4292. [Google Scholar] [CrossRef]
  21. Morisette, J.T.; Baret, F.; Liang, S. Special issue on global land product validation. IEEE Trans. Geosci. Remote Sens. 2006, 44, 1695–1697. [Google Scholar] [CrossRef]
  22. Padilla, M.; Stehman, S.V.; Chuvieco, E. Validation of the 2008 MODIS-MCD45 global burned area product using stratified random sampling. Remote Sens. Environ. 2014, 144, 187–196. [Google Scholar] [CrossRef]
  23. Padilla, M.; Stehman, S.V.; Hantson, S.; Oliva, P.; Alonso-Canas, I.; Bradley, A.; Tansey, K.; Mota, B.; Pereira, J.M.; Chuvieco, E. Comparing the accuracies of remote sensing global burned area products using stratified random sampling and estimation. Remote Sens. Environ. 2015, 160, 114–121. [Google Scholar] [CrossRef]
  24. Chuvieco, E.; Yue, C.; Heil, A.; Mouillot, F.; Alonso-Canas, I.; Padilla, M.; Pereira, J.M.; Oom, D.; Tansey, K. A new global burned area product for climate assessment of fire impacts. Glob. Ecol. Biogeogr. 2016, 25, 619–629. [Google Scholar] [CrossRef]
  25. Roy, D.P.; Boschetti, L. Southern Africa validation of the MODIS, L3JRC, and GlobCarbon burned-area products. IEEE Trans. Geosci. Remote Sens. 2009, 47, 1032–1044. [Google Scholar] [CrossRef]
  26. Masek, J.G.; Huang, C.; Wolfe, R.E.; Cohen, W.; Hall, F.; Kutler, J.; Nelson, P. North American forest disturbance from a decadal Landsat record. Remote Sens. Environ. 2008, 112, 2914–2926. [Google Scholar] [CrossRef]
  27. Thomas, N.E.; Huang, C.; Goward, S.N.; Powell, S.; Rishmawi, K.; Schleeweis, K.; Hinds, A. Validation of North American forest disturbance dynamics derived from Landsat time series stacks. Remote Sens. Environ. 2011, 115, 19–32. [Google Scholar] [CrossRef]
  28. Boschetti, L.; Roy, D.P.; Justice, C.O.; Humber, M.L. MODIS-Landsat fusion for large area 30 m burned area mapping. Remote Sens. Environ. 2015, 161, 27–42. [Google Scholar] [CrossRef]
  29. Mitri, G.H.; Gitas, I.Z. A semi-automated object-oriented model for burned area mapping in the Mediterranean region using Landsat-TM imagery. Int. J. Wildland Fire 2004, 13, 367–376. [Google Scholar] [CrossRef]
  30. Henry, M.C. Comparison of single- and multi-date Landsat data for mapping wildfire scares in Ocala National Forest, Florida. Photogramm. Eng. Remote Sens. 2008, 74, 881–891. [Google Scholar] [CrossRef]
  31. Bastarrika, A.; Chuvieco, E.; Martin, M.P. Mapping burned areas from Landsat TM/ETM+ data with a two-phase algorithm: Balancing omission and commission errors. Remote Sens. Environ. 2011, 115, 1003–1012. [Google Scholar] [CrossRef]
  32. Smith, A.M.S.; Drake, N.A.; Wooster, M.J.; Hudak, A.T.; Holden, Z.A.; Gibbons, C.J. Production of Landsat ETM+ reference imagery of burned areas within Southern African savannahs: Comparison of methods and application to MODIS. Int. J. Remote Sens. 2007, 12, 2753–2775. [Google Scholar] [CrossRef]
  33. Mallinis, G.; Koutsias, N. Comparing ten classification methods for burned area mapping in a Mediterranean environment using Landsat TM satellite data. Int. J. Remote Sens. 2012, 33, 4408–4433. [Google Scholar] [CrossRef]
  34. Mitri, G.H.; Gitas, I.Z. Fire type mapping using object-based classification of Ikonos imagery. Int. J. Wildland Fire 2006, 15, 457–462. [Google Scholar] [CrossRef]
  35. Mitri, G.H.; Gitas, I.Z. Mapping the severity of fire using object-based classification of IKONOS imagery. Int. J. Wildland Fire 2008, 17, 431–442. [Google Scholar] [CrossRef]
  36. Holden, Z.A.; Morgan, P.; Smith, A.M.S.; Vierling, L. Beyond Landsat: A comparison of four satellite sensors for detecting burn severity in ponderosa pine forests of the Gila Wilderness, NM, USA. Int. J. Wildland Fire 2010, 19, 449–458. [Google Scholar] [CrossRef]
  37. Dragozi, E.; Gitas, I.Z.; Stavrakoudis, D.G.; Theocharis, J.B. Burned area mapping using support vector machines and the FuzCoC feature selection method on VHR IKONOS imagery. Remote Sens. 2014, 6, 12005–12036. [Google Scholar] [CrossRef]
  38. Dragozi, E.; Gitas, I.Z.; Bajocco, S.; Stavrakoudis, D.G. Exploring the relationship between burn severity field data and very high resolution GeoEye images: The case of the 2011 Evros wildfire in Greece. Remote Sens. 2016, 8, 566. [Google Scholar] [CrossRef]
  39. Chen, G.; Metz, M.R.; Rizzo, D.M.; Meentemeyer, R.K. Mapping burn severity in a disease-impacted forest landscape using Landsat and MASTER imagery. Int. J. Appl. Earth Obs. Geoinf. 2015, 40, 91–99. [Google Scholar] [CrossRef]
  40. Wu, Z.; Middleton, B.; Hetler, R.; Vogel, J.; Dye, D. Vegetation burn severity mapping using Landsat-8 and Worldview-2. Photogramm. Eng. Remote Sens. 2015, 2, 143–154. [Google Scholar] [CrossRef]
  41. Archibald, S.; Scholes, R.; Roy, D.; Roberts, G.; Boschetti, L. Southern African fire regimes as revealed by remote sensing. Int. J. Wildland Fire 2010, 19, 861–878. [Google Scholar] [CrossRef]
  42. Hantson, S.; Pueyo, S.; Chuvieco, E. Global fire size distribution is driven by human impact and climate. Glob. Ecol. Biogeogr. 2015, 24, 77–86. [Google Scholar] [CrossRef]
  43. Cansler, C.A.; McKenzie, D. Climate, fire size, and biophysical setting control fire severity and spatial pattern in the northern Cascade Range, USA. Ecol. Appl. 2014, 24, 1037–1056. [Google Scholar] [CrossRef] [PubMed]
  44. Mansuy, N.; Boulanger, Y.; Terrier, A.; Gauthier, S.; Robitaille, A.; Bergeron, Y. Spatial attributes of fire regime in eastern Canada: Influences of regional landscape physiography and climate. Landsc. Ecol. 2014, 29, 1157–1170. [Google Scholar] [CrossRef]
  45. Lertzman, K.; Fall, J.; Dorner, B. Three kinds of heterogeneity in fire regimes: At the crossroads of fire history and landscape ecology. Northwest Sci. 1998, 72, 4–23. [Google Scholar]
  46. Turner, M.G.; Baker, W.L.; Peterson, C.J.; Peet, R.K. Factors influencing succession: Lessons from large, infrequent natural disturbances. Ecosystems 1998, 1, 511–523. [Google Scholar] [CrossRef]
  47. Madoui, A.; Leduc, A.; Gauthier, S.; Bergeron, Y. Spatial pattern analyses of post-fire residual stands in the black spruce boreal forest of western Quebec. Int. J. Wildland Fire 2010, 19, 1110–1126. [Google Scholar] [CrossRef]
  48. Morgan, P.; Hardy, C.C.; Swetnam, T.W.; Rollins, M.G.; Long, D.G. Mapping fire regimes across time and space: Understanding coarse and fine-scale fire patterns. Int. J. Wildland Fire 2001, 10, 329–342. [Google Scholar] [CrossRef]
  49. Whitlock, C.; Moreno, P.; Bartlein, P. Climatic controls of Holocene fire patterns in southern South America. Quat. Res. 2007, 68, 28–36. [Google Scholar] [CrossRef]
  50. Flannigan, M.D.; Krawchuk, M.A.; de Groot, W.J.; Wotton, B.M.; Gowman, L.M. Implications of changing climate for global wildland fire. Int. J. Wildland Fire 2009, 18, 483–507. [Google Scholar] [CrossRef]
  51. Heyerdahl, E.K.; Brubaker, L.B.; Agee, J.K. Spatial controls of historical fire regimes: A multiscale example for the Interior West, USA. Ecology 2001, 82, 660–678. [Google Scholar] [CrossRef]
  52. Collins, B.M.; Kelly, N.M.; Wagtendonk, J.W.; Stephens, S.L. Spatial patterns of large natural fires in Sierra Nevada wilderness areas. Landsc. Ecol. 2007, 22, 545–557. [Google Scholar] [CrossRef]
  53. Bergeron, Y.; Gauthier, S.; Flannigan, M.; Kafka, V. Fire regimes at the transition between mixedwood and coniferous boreal forest in Northwestern Quebec. Ecology 2004, 85, 1916–1932. [Google Scholar] [CrossRef]
  54. Barros, A.; Pereira, J.; Lund, U. Identifying geographical patterns of wildfire orientation: A watershed-based analysis. For. Ecol. Manag. 2012, 264, 98–107. [Google Scholar] [CrossRef]
  55. Falk, D.A.; Heyerdahl, E.K.; Brown, P.M.; Farris, C.; Fulé, P.Z.; McKenzie, D.; Swetnam, T.W.; Taylor, A.H.; Van Horne, M.L. Multi-scale controls of historical forest-fire regimes: New insights from fire-scar networks. Front. Ecol. Environ. 2011, 9, 446–454. [Google Scholar] [CrossRef]
  56. Hastie, T.; Tibshirani, R.; Friedman, J. The Elements of Statistical Learning; Data Mining, Inference, and Prediction, 2nd ed.; Springer: New York, NY, USA, 2009. [Google Scholar]
  57. Breiman, L.; Freiedman, J.H.; Olshen, R.A.; Stone, C.J. Classification and Regression Trees, 1st ed.; Chapman and Hall: New York, NY, USA, 1984. [Google Scholar]
  58. Biging, G.S.; Colby, D.R.; Congalton, R.G. Sampling systems for change detection accuracy assessment, remote sensing change detection. In Environmental Monitoring Methods and Applications; Lunetta, R.S., Elvidge, C.D., Eds.; Ann Arbor Press: Chelsea, MI, USA, 1998; pp. 281–308. [Google Scholar]
  59. Khorram, S. (Ed.) Accuracy Assessment of Remote Sensing-Derived Change Detection; American Society for Photogrammetry and Remote Sensing: Bethesda, MD, USA, 1999. [Google Scholar]
  60. Homer, C.; Dewitz, J.; Yang, L.; Jin, S.; Danielson, P.; Xian, G.; Coulston, J.; Herold, N.; Wickham, J.; Megown, K. Completion of the 2011 National Land Cover Database for the conterminous United States—Representing a decade of land cover change information. Photogramm. Eng. Remote Sens. 2015, 81, 345–354. [Google Scholar]
  61. Richter, R.; Schläpfer, D. Atmospheric/Topographic Correction for Satellite Imagery; ATCOR-2/3 User Guide, Version 9.0.2; ReSe Applications: Langeggweg, Switzerland, 2016; Available online: http://www.rese.ch/pdf/atcor3_manual.pdf (accessed on 8 February 2017).
  62. Adler-Golden, S.M.; Berk, A.; Bernstein, L.S.; Richtsmeier, S.; Acharyal, P.K.; Matthew, M.W.; Anderson, G.P.; Allred, C.L.; Jeong, L.S.; Chetwynd, J.H. FLAASH, a MODTRAN4 atmospheric correction package for hyperspectral data retrievals and simulations. In Proceedings of the Summaries of the Seventh JPL Airborne Earth Science Workshop, Pasadena, CA, USA, 12–16 January 1998; pp. 1–6. [Google Scholar]
  63. Adler-Golden, S.M.; Matthew, M.W.; Bernstein, L.S.; Levine, R.Y.; Berk, A.; Richtsmeier, S.C.; Acharya, P.K.; Anderson, G.P.; Felde, J.W.; Gardner, J.A.; et al. Atmospheric correction for shortwave spectral imagery based on MODTRAN4. SPIE Proc. Imaging Spectrom. V 1999, 3753, 61–69. [Google Scholar]
  64. Chuvieco, E. (Ed.) Remote Sensing of Large Wildfires in the European Mediterranean Basin; Springer: Berlin, Germany, 1997. [Google Scholar]
  65. Fleiss, J.L. Statistical Methods for Rates and Proportions, 2nd ed.; John Wiley & Sons: New York, NY, USA, 1981. [Google Scholar]
  66. Forbes, A.D. Classification-algorithm evaluation: Five performance measures based on confusion matrices. J. Clin. Monit. 1995, 11, 189–206. [Google Scholar] [CrossRef] [PubMed]
  67. Stehman, S.V. Estimating standard errors of accuracy assessment statistics under cluster sampling. Remote Sens. Environ. 1997, 60, 258–269. [Google Scholar] [CrossRef]
  68. Cochran, W.G. Sampling Techniques, 3rd ed.; John Wiley & Sons: New York, NY, USA, 1977. [Google Scholar]
  69. Omernik, J.M.; Griffith, G.E. Ecoregions of the conterminous United States: Evolution of a hierarchical spatial framework. Environ. Manag. 2014, 54, 1249–1266. [Google Scholar] [CrossRef] [PubMed]
  70. McGarigal, K.; Cushman, S.A.; Ene, E. FRAGSTATS v4: Spatial Pattern Analysis Program for Categorical and Continuous Maps; University of Massachusetts: Amherst, MA, USA, 2012; Available online: http://www.umass.edu/landeco/research/fragstats/fragstats.html (accessed on 22 September 2016).
  71. Keane, R.E.; Agee, J.K.; Fulé, P.; Keeley, J.E.; Key, C.; Kitchen, S.G.; Miller, R.; Schulte, L.A. Ecological effects of large fires on US landscapes: Benefit or catastrophe? Int. J. Wildland Fire 2009, 17, 696–712. [Google Scholar] [CrossRef]
  72. Petropoulos, G.P.; Kontoes, C.; Keramitsoglou, I. Burnt area delineation from a uni-temporal perspective based on Landsat TM imagery classification using Support Vector Machines. Int. J. Appl. Earth Obs. Geoinf. 2011, 13, 70–80. [Google Scholar] [CrossRef]
  73. Bastarrika, A.; Alvarado, M.; Artano, K.; Martinez, M.P.; Mesanza, A.; Torre, L.; Ramo, R.; Chuvieco, E. BAMS: A tool for supervised burned area mapping using Landsat data. Remote Sens. 2014, 6, 12360–12380. [Google Scholar] [CrossRef]
  74. Goodwin, N.R.; Collett, L.J. Development of an automated method for mapping fire history captured in Landsat TM and ETM+ time series across Queensland, Australia. Remote Sens. Environ. 2014, 148, 206–221. [Google Scholar] [CrossRef]
  75. Scheller, R.M.; Domingo, J.B.; Sturtevant, B.R.; Williams, J.S.; Rudy, A.; Gustafson, E.J.; Mladenoff, D.J. Design, development and application of LANDIS-II, a spatial landscape simulation model with flexible temporal and spatial resolution. Ecol. Modell. 2007, 201, 409–419. [Google Scholar] [CrossRef]
  76. Lhermitte, S.; Verbesselt, J.; Verstraeten, W.W.; Veraverbeke, S.; Coppin, P. Assessing intra-annual vegetation regrowth after fire using the pixel based regeneration index. ISPRS J. Photogramm. Remote Sens. 2011, 66, 17–27. [Google Scholar] [CrossRef] [Green Version]
  77. Chambers, M.E.; Fornwalt, P.J.; Malone, S.L.; Battaglia, M. Patterns of conifer regeneration following high severity wildfire in ponderosa pine—Dominated forests of the Colorado Front Range. For. Ecol. Manag. 2016, 378, 57–67. [Google Scholar] [CrossRef]
Remotesensing 09 00743 g001 550
Figure 1. (A) The distribution of area classified as burned by the Burned Area Essential Climate Variable (BAECV) product between 1984 and 2015, and (B) the distribution of the high-resolution images used to validate the BAECV relative to a modified version of the U.S. Environmental Protection Agency Level I ecoregions. High-resolution images containing at least one burned area (n = 143) are shown in relation to images containing no burned areas (n = 143), burned area points are shown on top so some not burned image locations are not visible.
Figure 1. (A) The distribution of area classified as burned by the Burned Area Essential Climate Variable (BAECV) product between 1984 and 2015, and (B) the distribution of the high-resolution images used to validate the BAECV relative to a modified version of the U.S. Environmental Protection Agency Level I ecoregions. High-resolution images containing at least one burned area (n = 143) are shown in relation to images containing no burned areas (n = 143), burned area points are shown on top so some not burned image locations are not visible.
Remotesensing 09 00743 g001
Remotesensing 09 00743 g002 550
Figure 2. The correlation between the amount of burned area identified by the reference dataset and the amount of burned area identified by the Burned Area Essential Climate Variable (BAECV) across the conterminous U.S. (CONUS) where each point represents a high-resolution image. RMSE: root mean square error.
Figure 2. The correlation between the amount of burned area identified by the reference dataset and the amount of burned area identified by the Burned Area Essential Climate Variable (BAECV) across the conterminous U.S. (CONUS) where each point represents a high-resolution image. RMSE: root mean square error.
Remotesensing 09 00743 g002
Remotesensing 09 00743 g003 550
Figure 3. An example of differences in the spatial characteristics for a fire as shown by (A) a Worldview-2 image in natural color composite, and as mapped by (B) Worldview-2, (C) the Landsat Burned Area Essential Climate Variable (BAECV) product, and (D) the Moderate Resolution Imaging Spectroradiometer (MODIS) MCD45 burned area dataset.
Figure 3. An example of differences in the spatial characteristics for a fire as shown by (A) a Worldview-2 image in natural color composite, and as mapped by (B) Worldview-2, (C) the Landsat Burned Area Essential Climate Variable (BAECV) product, and (D) the Moderate Resolution Imaging Spectroradiometer (MODIS) MCD45 burned area dataset.
Remotesensing 09 00743 g003
Remotesensing 09 00743 g004 550
Figure 4. Comparisons between the reference dataset and the Landsat Burned Area Essential Climate Variable (BAECV) to demonstrate the visual effect of error rates. (A) The QuickBird-2 image was collected on 13 August 2003, the Landsat image was collected seven days prior. Error of omission and commission was 20% and 42%, respectively. (B) The four QuickBird-2 images were collected on 31 August 2005, the Landsat image was collected two days prior. Error of omission and commission was 19% and 65%, respectively. (C) The GeoEye-1 image was collected on 15 September 2009, and the Landsat image was collected six days later. Error of omission and commission was 28% and 39%, respectively. The high-resolution images are shown in false color where live vegetation is red.
Figure 4. Comparisons between the reference dataset and the Landsat Burned Area Essential Climate Variable (BAECV) to demonstrate the visual effect of error rates. (A) The QuickBird-2 image was collected on 13 August 2003, the Landsat image was collected seven days prior. Error of omission and commission was 20% and 42%, respectively. (B) The four QuickBird-2 images were collected on 31 August 2005, the Landsat image was collected two days prior. Error of omission and commission was 19% and 65%, respectively. (C) The GeoEye-1 image was collected on 15 September 2009, and the Landsat image was collected six days later. Error of omission and commission was 28% and 39%, respectively. The high-resolution images are shown in false color where live vegetation is red.
Remotesensing 09 00743 g004
Remotesensing 09 00743 g005 550
Figure 5. A comparison of: (A) the RapidEye-3 image (DOY 122, 2015); (B) the Landsat ETM+ image (DOY 120, 2015); (C) the Landsat ETM+ image using band combinations 7, 4 and 2 for red, green and blue, respectively; (D) burned area as classified by RapidEye-3; and (E) burned area as classified by the BAECV. The blue circles show a recent burned area agreed upon by the high-resolution imagery and Landsat BAECV. The yellow circles show an incorrect error of omission due to a new fire occurring in the two-day gap between the high-resolution and Landsat collection dates. The purple circles show the more common commission errors due to the BAECV classifying fields with minimal vegetation as burned. Moderate Resolution Imaging Spectroradiometer (MODIS) active fire points identified the fire in the blue circle (DOY 119) but no other fires in the image extent shown.
Figure 5. A comparison of: (A) the RapidEye-3 image (DOY 122, 2015); (B) the Landsat ETM+ image (DOY 120, 2015); (C) the Landsat ETM+ image using band combinations 7, 4 and 2 for red, green and blue, respectively; (D) burned area as classified by RapidEye-3; and (E) burned area as classified by the BAECV. The blue circles show a recent burned area agreed upon by the high-resolution imagery and Landsat BAECV. The yellow circles show an incorrect error of omission due to a new fire occurring in the two-day gap between the high-resolution and Landsat collection dates. The purple circles show the more common commission errors due to the BAECV classifying fields with minimal vegetation as burned. Moderate Resolution Imaging Spectroradiometer (MODIS) active fire points identified the fire in the blue circle (DOY 119) but no other fires in the image extent shown.
Remotesensing 09 00743 g005
Remotesensing 09 00743 g006 550
Figure 6. Examples of the persistence of burn evidence in agricultural areas in eastern Kansas from (A) DOY 65, (B) DOY 81, (C) DOY 94 and (D) DOY 104, 2016 (four images across 39 days) using imagery from RapidEye satellites. Although the burned area, marked by the red rectangle, is clearly visible at DOY 81, it rapidly becomes difficult to distinguish a fading burn from freshly tilled or bare soil using aerial imagery, alone.
Figure 6. Examples of the persistence of burn evidence in agricultural areas in eastern Kansas from (A) DOY 65, (B) DOY 81, (C) DOY 94 and (D) DOY 104, 2016 (four images across 39 days) using imagery from RapidEye satellites. Although the burned area, marked by the red rectangle, is clearly visible at DOY 81, it rapidly becomes difficult to distinguish a fading burn from freshly tilled or bare soil using aerial imagery, alone.
Remotesensing 09 00743 g006
Remotesensing 09 00743 g007 550
Figure 7. Two examples demonstrating the role of vegetation recovery in creating false errors due to a date gap between the high-resolution imagery and the Burned Area Essential Climate Variable (BAECV): (A) an example of mapping a burned area in New Mexico at a site dominated by shrub-scrub and grassland (p33r34); and (B) an example of mapping a burned area in California at a site dominated by mixed forest (p43r33). The high-resolution images are shown in false color where live vegetation is red.
Figure 7. Two examples demonstrating the role of vegetation recovery in creating false errors due to a date gap between the high-resolution imagery and the Burned Area Essential Climate Variable (BAECV): (A) an example of mapping a burned area in New Mexico at a site dominated by shrub-scrub and grassland (p33r34); and (B) an example of mapping a burned area in California at a site dominated by mixed forest (p43r33). The high-resolution images are shown in false color where live vegetation is red.
Remotesensing 09 00743 g007
Table
Table 1. Distribution of high-resolution images used to validate the U.S. Geological Survey’s Burned Area Essential Climate Variable (BAECV) by year.
Table 1. Distribution of high-resolution images used to validate the U.S. Geological Survey’s Burned Area Essential Climate Variable (BAECV) by year.
YearArid WestMountain WestGreat PlainsEastTotal Number of Images
20031686030
200402619
20051210619
2006795021
20070301013
20081200719
200911461435
20104152021
20119126532
20121071624
2013159015
201480151336
20150012012
Total90666862286
Table
Table 2. Accuracy statistics for the U.S. Geological Survey’s Burned Area Essential Climate Variable (BAECV) by ecoregion and for the conterminous United States (CONUS). Errors of omission and commission are presented for burned area only. Standard errors are shown in parentheses.
Table 2. Accuracy statistics for the U.S. Geological Survey’s Burned Area Essential Climate Variable (BAECV) by ecoregion and for the conterminous United States (CONUS). Errors of omission and commission are presented for burned area only. Standard errors are shown in parentheses.
Accuracy StatisticArid WestMountain WestGreat PlainsEastCONUS
Omission Error (%)14 (4)22 (3)13 (13)46 (3)22 (4)
Commission Error (%)36 (6)39 (5)70 (5)48 (5)48 (3)
Overall Accuracy (%)97 (1)98 (1)99 (2)97 (2)97 (1)
Dice Coefficient (%)73 (6)69 (3)44 (6)53 (4)61 (3)
Relative Bias (%)34 (19)27 (13)194 (27)3 (13)66 (10)
Landsat-Based Omission Error (%)31 (6)41 (7)62 (9)67 (8)42 (6)
Landsat-Based Commission Error (%)24 (3)32 (5)57 (9)47 (5)33 (3)
Number of High Res Images90666862286
Mean Image Date Gap (days)8.8 ± 0.914.9 ± 1.52.4 ± 0.97.1 ± 0.98.4 ± 0.6
Table
Table 3. Characteristics of satellites used to derive the reference dataset in relation to the characteristics of the Landsat sensors used to derive the Burned Area Essential Climate Variable (BAECV). Landsat TM: Landsat Thematic Mapper; Landsat ETM+: Landsat Enhanced Thematic Mapper Plus.
Table 3. Characteristics of satellites used to derive the reference dataset in relation to the characteristics of the Landsat sensors used to derive the Burned Area Essential Climate Variable (BAECV). Landsat TM: Landsat Thematic Mapper; Landsat ETM+: Landsat Enhanced Thematic Mapper Plus.
PlatformData Availability (years)Spatial Resolution (m)Data Collection TypeImage ExtentSpectral Range (μm)Spectral Resolution (# of Bands)Sponsor, Country
QuickBird-22001–20142On-Demand18 km0.43–0.924DigitalGlobe, U.S.
GeoEye-12008–Present2On-Demand15 km0.45–0.924DigitalGlobe, U.S.
Worldview-22009–Present2On-Demand16 km0.4–1.048DigitalGlobe, U.S.
RapidEye-1–52008–Present5On-Demand (2008–2013), Continuous 1 (2014–present) (1 to 24 day revisit)25 km0.44–0.855Planet, U.S.
Landsat TM1984–201130Continuous
(16 day revisit)		185 km0.45–2.35; 10.4–12.57NASA, U.S.
Landsat ETM+1999–2003, 2003–Present (scan-line corrector off)30Continuous
(16 day revisit)		185 km0.45–2.35; 10.4–12.58NASA, U.S.
1 Continuous during growing season.
Table
Table 4. The relative abundance of National Land Cover Database (NLCD) land cover types across the conterminous United States (CONUS) and within each of the four ecoregions modified from the U.S. Environmental Protection Agency Level I ecoregions.
Table 4. The relative abundance of National Land Cover Database (NLCD) land cover types across the conterminous United States (CONUS) and within each of the four ecoregions modified from the U.S. Environmental Protection Agency Level I ecoregions.
NLCD Land Cover Types (2006)CONUS (km2)CONUS (%)Arid West (%)Mountain West (%)Great Plains (%)East (%)
Deciduous Forest876,25712.10.43.93.026.7
Evergreen Forest934,12312.99.650.81.79.9
Mixed Forest161,8612.20.41.90.14.8
Shrub/Scrub1,746,33624.264.823.212.44.0
Grasslands/Herbaceous1,176,27616.39.310.736.92.7
Pasture/Hay537,5127.41.61.86.711.9
Cultivated Crops1,252,99817.35.41.330.416.0
Woody Wetlands312,4314.30.40.81.09.5
Emergent Herbaceous Wetlands105,0141.50.40.51.32.0
Other (developed, barren, open water)121,1051.77.75.16.512.5
Table
Table 5. The distribution of total burned area by ecoregion using three independent sources of data. Mtn: Mountain, MODIS: Moderate Resolution Imaging Spectroradiometer, MTBS: Monitoring Trends in Burn Severity, GeoMAC: Geospatial Multi-Agency Coordination.
Table 5. The distribution of total burned area by ecoregion using three independent sources of data. Mtn: Mountain, MODIS: Moderate Resolution Imaging Spectroradiometer, MTBS: Monitoring Trends in Burn Severity, GeoMAC: Geospatial Multi-Agency Coordination.
SourceArid West (%)Mtn West (%)Great Plains (%)East (%)
MODIS (MCD45) (2000–2015)3094417
MTBS (1984–2014)15222043
GeoMAC (2000–2015)5034.510.55
Average31.5222521.5
Table
Table 6. Landscape metrics calculated per burned area in FRAGSTATs for fires mapped by both the high-resolution and Landsat imagery. Agricultural field burns were excluded. Twenty-four of the 91 burn events were not identified by the Moderate Resolution Imaging Sepectroradiometer (MODIS) MCD45 burned area dataset. Significant differences (p < 0.01) between groups (high-resolution, BAECV, MCD45) are shown using different superscript numbers as determined from pair-wise t-tests. BAECV: Burned Area Essential Climate Variable.
Table 6. Landscape metrics calculated per burned area in FRAGSTATs for fires mapped by both the high-resolution and Landsat imagery. Agricultural field burns were excluded. Twenty-four of the 91 burn events were not identified by the Moderate Resolution Imaging Sepectroradiometer (MODIS) MCD45 burned area dataset. Significant differences (p < 0.01) between groups (high-resolution, BAECV, MCD45) are shown using different superscript numbers as determined from pair-wise t-tests. BAECV: Burned Area Essential Climate Variable.
Imagery SourcePatch DensityEdge DensityLandscape Shape IndexArea-Weighted Mean Patch SizeArea-Weighted Perimeter-Area Ratio
High-resolution30.1 ± 3.2 1204.0 ± 21.4 170.8 ± 7.4 11293.3 ± 136.5 1843.4 ± 88.5 1
BAECV (Landsat)0.25 ± 0.03 237.5 ± 3.9 211.1 ± 1.2 23463.1 ± 365.4 2134.1 ± 14.1 2
MCD45 (MODIS)0.02 ± 0.007 32.7 ± 0.3 31.9 ± 0.2 36913.4 ± 854.1 320.1 ± 2.5 3

Share and Cite

MDPI and ACS Style

Vanderhoof, M.K.; Brunner, N.; Beal, Y.-J.G.; Hawbaker, T.J. Evaluation of the U.S. Geological Survey Landsat Burned Area Essential Climate Variable across the Conterminous U.S. Using Commercial High-Resolution Imagery. Remote Sens. 2017, 9, 743. https://doi.org/10.3390/rs9070743

AMA Style

Vanderhoof MK, Brunner N, Beal Y-JG, Hawbaker TJ. Evaluation of the U.S. Geological Survey Landsat Burned Area Essential Climate Variable across the Conterminous U.S. Using Commercial High-Resolution Imagery. Remote Sensing. 2017; 9(7):743. https://doi.org/10.3390/rs9070743

Chicago/Turabian Style

Vanderhoof, Melanie K., Nicole Brunner, Yen-Ju G. Beal, and Todd J. Hawbaker. 2017. "Evaluation of the U.S. Geological Survey Landsat Burned Area Essential Climate Variable across the Conterminous U.S. Using Commercial High-Resolution Imagery" Remote Sensing 9, no. 7: 743. https://doi.org/10.3390/rs9070743

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop