Abstract
International policy makers and climate researchers use greenhouse gas emissions inventory estimates in a variety of ways. Because of the varied uses of the inventory data, as well as the high uncertainty surrounding some of the source category estimates, considerable effort has been devoted to understanding the causes and magnitude of uncertainty in national emissions inventories. In this paper, we focus on two aspects of the rationale for quantifying uncertainty: (1) the possible uses of the quantified uncertainty estimates for policy (e.g., as a means of adjusting inventories used to determine compliance with international commitments); and (2) the direct benefits of the process of investigating uncertainties in terms of improving inventory quality. We find that there are particular characteristics that an inventory uncertainty estimate should have if it is to be used for policy purposes: (1) it should be comparable across countries; (2) it should be relatively objective, or at least subject to review and verification; (3) it should not be subject to gaming by countries acting in their own self-interest; (4) it should be administratively feasible to estimate and use; (5) the quality of the uncertainty estimate should be high enough to warrant the additional compliance costs that its use in an adjustment factor may impose on countries; and (6) it should attempt to address all types of inventory uncertainty. Currently, inventory uncertainty estimates for national greenhouse gas inventories do not have these characteristics. For example, the information used to develop quantitative uncertainty estimates for national inventories is often based on expert judgments, which are, by definition, subjective rather than objective, and therefore difficult to review and compare. Further, the practical design of a potential factor to adjust inventory estimates using uncertainty estimates would require policy makers to (1) identify clear environmental goals; (2) define these goals precisely in terms of relationships among important variables (such as emissions estimate, commitment level, or statistical confidence); and (3) develop a quantifiable adjustment mechanism that reflects these environmental goals. We recommend that countries implement an investigation-focused (i.e., qualitative) uncertainty analysis that will (1) provide the type of information necessary to develop more substantive, and potentially useful, quantitative uncertainty estimates-regardless of whether those quantitative estimates are used for policy purposes; and (2) provide information needed to understand the likely causes of uncertainty in inventory data and thereby point to ways to improve inventory quality (i.e., accuracy, transparency, completeness, and consistency).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
A trading ratio specifies the relative value of emission allowances from two different sources. As described subsequently in Section 3.1, the trading ratio is the number of units of emissions from one source that is equivalent to (offset by) one unit of emission allowances purchased from another source. If the ratio is 1, one purchased allowance can be used to increase emissions by one unit. A lower ratio is more protective of the environment (i.e., increases the likelihood that the trade will result in environmental improvement). For example, a ratio of 1:2 requires that two units of allowances (representing emission reductions elsewhere) must be purchased for each additional unit of emissions offset. In this paper, trading ratios can be either greater or less than 1.
The Parties to the United Nations Framework Convention on Climate Change (UNFCCC) are in the process of adopting an inventory adjustment scheme that in some circumstances uses generic uncertainty estimates to adjust national inventory estimates that a UNFCCC expert review team finds to be deficient.
Throughout this paper, unless otherwise specified, the discussion focuses on a potential adjustment approach that relies on national estimates of uncertainty data. At the 2005 COP/MOP 1 in Montreal, the Parties to the Kyoto Protocol agreed on a simplified adjustment process that does not use uncertainty estimates from individual Parties. That process is not the focus of this paper.
The current adjustment approach under the Kyoto Protocol is based on the judgments of an expert review team and default uncertainty estimates (i.e., conservative factors). These default uncertainty estimates are based on expert judgment and are not specific to a Party’s inventory or related to the actual quality of a Party’s inventory. They are instead used as a justification for a conservative (i.e., punitive) adjustment to a Party’s inventory estimate. Expert review teams are also given flexibility to apply adjustments and conservative factors. (See FCCC/SBSTA/2003/L.6/Add.3 and FCC/SBSTA/2005/2.) This approach will be revisited later in the paper.
Additional discussion of potential adjustments, particularly under a trading regime, can be found in Cohen, Sussman, and Jayaraman (1998).
See, for example, submissions from Australia, Canada, China, New Zealand, Portugal, and the United States to the UNFCCC (2000).
For ease of exposition, in this paper, we refer sometimes to commitment years and sometimes to commitment periods. The analysis is appropriate for either, but is easier to conceptualize in terms of years. The Kyoto Protocol uses commitment periods, which are summed over 5 years.
Throughout this discussion, we assume that probability distributions for estimated emissions or emission reductions are normal and that the shape of the probability distribution of emissions for each country or source does not change significantly as emissions are reduced. This entire analysis also ignores the possibility that we might underestimate actual emission reductions (i.e., this analysis assumes that the purpose of investigating uncertainty is to ensure that we do not overestimate actual emission reductions).
Given the uncertainty (u%) range (assumed to be the end points of a 95% confidence interval) around estimated emissions (E), and assuming a normal distribution, the standard deviation of the distribution equals approximately u% E/ (1.96). If we are willing to accept that our emissions could be up to p% higher than the nominal emissions commitment, then the probability that the actual value lies below an upper bound of (100 + p)% E can be calculated from the table for a normal error integral found in standard statistics textbooks or using standard statistical software (including Excel). See, for example, Appendix A in Taylor (1997).
Uncertainty may also differ (and in fact may be lower) in the base year because of policy and political changes over time, including the effects of economic reforms. These changes can affect the definition of what types of sources and sinks are included in the emissions estimate.
A reviewer pointed out that removals are not normally accounted for in the base year under the Kyoto Protocol, except for some 3.4 activities.
It may not be immediately obvious how to calculate the uncertainty of emission reductions, as it will depend not only on uncertainty in the base and current year, but also on correlations between the two uncertainty estimates, since the factors that produce bias in one year may produce bias in another year. Winiwarter and Rypdal (2001) have looked at trend uncertainties for the Austrian inventory.
Constructing Table 2 requires two steps: (1) making necessary assumptions (e.g., about the uncertainty of emission reductions and the required level of confidence) and calculating the necessary adjustment in emission reductions to provide that level of confidence, and (2) translating the adjustment to emission reductions into an adjustment to the emissions estimate.
The UNFCCC approach uses adjustments to both the base and current year. Again, these adjustments are primarily designed to encourage the use of good practice inventory methods (while also providing some environmental benefit) and are not related to the uncertainty of the overall inventory or of a specific source category for a particular country.
In addition to general political considerations and the feasibility of negotiating an international system of adjustments that would require reductions beyond those already agreed to (as in the Kyoto Protocol), such a system could raise equity concerns if poorer nations were also those with greater uncertainty, especially if this were primarily due to the source composition of their inventory. In particular, nations with inventories that have a large component of non-energy sources will tend to have greater uncertainties that would be relatively expensive to reduce.
Statistical uncertainty results from natural variations (e.g., random human errors in the measurement process and fluctuations in measurement equipment). Statistical uncertainty can be detected through repeated experiments or sampling of data.
Systematic parameter uncertainty occurs if data are systematically biased. In other words, the average of the measured or estimated value is always less or greater than the true value. Biases arise, for example, because emission factors are constructed from non-representative samples, not all relevant source activities or categories have been identified, or incorrect or incomplete estimation methods or faulty measurement equipment have been used. Because the true value is unknown, such systematic biases cannot be detected through repeated experiments and, therefore, cannot be quantified through statistical analysis. However, it is possible to identify biases and, sometimes, quantify them through data quality investigations and expert judgments.
There are cases where cause and direction of a specific systematic bias may be known for a national statistical dataset, but for reasons of resource and time limitations or political constraints they cannot be quantified or corrected for in the official national statistics. Therefore, arguing that known systematic biases can be corrected for ignores the real complexities of collecting national statistical data.
The role of expert judgment can be twofold: First, expert judgment can be the source of the data that are necessary to estimate the parameter. Second, expert judgment can help (in combination with data quality investigations) to identify, explain, and quantify both statistical and systematic uncertainties. It is also important to recognize that it is difficult for experts to distinguish between statistical uncertainty and systematic biases. Therefore, elicited estimates of uncertainty tend to incorporate both.
For example, in the United States an early estimate of the uncertainty in methane emissions from manure management based on expert judgment was ±15%. The following year, improvements were made to the methodology to account for more regional differences and corrections were made to some activity data. The resulting change in the overall emissions estimate was 60%.
Economy in transition (EIT) is a term used under the UNFCCC to refer to the countries of the former Soviet Union and related East European satellite nations that are now undergoing a transition to a market-based economic system.
The impact of altered management practices at a farm, for example, will depend on the effectiveness of practices at the farm in reducing “edge of farm” nutrient discharges (which is highly site specific), on weather, on how spatially removed the farm is from an adjacent water body, and on conditions in adjacent receiving water (King & Kuch, 2003).
A “95% confidence interval” is an interval calculated from observational data such that the interval would be expected to include the unknown true value (e.g., total GHG emissions) for 95% of possible data sets, although we generally will not know whether or not this is true for a given data set. Since emissions inventory estimation often uses non-statistical methods (e.g., expert judgment) and methods not based on observational data, the term 95% confidence interval is here extended to mean any interval that in some sense is assumed to have a 0.95 probability of including the unknown true value. The upper bound is typically assumed to be the 97.5th percentile, and the lower bound the 2.5th percentile, so that the same 2.5% of the values lie above and below the confidence interval.
For simplicity, we assume that the uncertainty (expressed as a percentage) is unchanged for the sector or country by activities that increase or decrease emissions.
Strictly, this equation represents the standard deviation of the sum of emissions from A and B, multiplied by a scalar. The magnitude of the scalar (which may equal 1) depends on the width of the confidence interval for which the uncertainties are calculated and on the shape of the distribution of emissions. The scalar would equal 1.96 for a 95% confidence interval if emissions were normally distributed. It is assumed for this equation that the uncertainties represent the same level of confidence for both A and B.
The impact of the SD term depends on the ratio of the uncertainty products.
While this large uncertainty between countries is unlikely for developed countries, it is certainly possible for trades between source categories. Moreover, the large uncertainty serves to illustrate the workings of the trading ratio.
References
Bartoszczuk, P., & Horabik, J. (2007). Tradable permit system: Considering uncertainty in emission estimates. Water, Air, & Soil Pollution: Focus (in press) doi:10.1007/s11267-006-9110-x
Cohen, J., Sussman, F., & Jayaraman, K. (1998). Improving Greenhouse Gas Emission Verification, Final report prepared by ICF Consulting and submitted to Environment Canada, 2 January. Subsequently released as an Environment Canada Report, February.
ISO (1993). International vocabulary of basic and general terms in metrology, Second edition. Geneva, Switzerland: ISO (International Organization for Standardization).
Jonas, M., & Nilsson, S. (2007). Prior to economic treatment of emissions and their uncertainties under the Kyoto Protocol: Scientific uncertainties that must be kept in mind. Water, Air, & Soil Pollution: Focus (in press) doi:10.1007/s11267-006-9113-7
Kasa, K. (2000). Knightian uncertainty and home bias. Federal Reserve Bank of San Francisco Economic Letter. (October 6)
King, D. M., & Kuch, P. J. (2003). Will nutrient trading ever work? An assessment of supply and demand problems and institutional obstacles. 33 ELR 10352, Environmental Law Institute, Washington, DC, USA.
Knight, F. H. (1921). A treatise on probability. London, UK: Macmillan.
Monni, S., Syri, S., Pipatti, R., & Savolainen, I. (2007). Extension of EU emissions trading scheme to other sectors and gases: Consequences for uncertainty of total tradable amount. Water, Air, & Soil Pollution: Focus (in press) doi:10.1007/s11267-006-9111-9
Morgan, M. G., & Henrion, M. (1990). Uncertainty: A guide to dealing with uncertainty in quantitative risk and policy analysis. Cambridge, UK: Cambridge University Press.
Nahorski, Z., Horabik, J., & Jonas, M. (2007). Compliance and emissions trading under the Kyoto Protocol: Rules for uncertain inventories. Water, Air, & Soil Pollution: Focus (in press) doi:10.1007/s11267-006-9112-8
Nilsson, S., Shvidenko, A., Jonas, M., & McCallum, I. (2007). Uncertainties of a regional terrestrial biota full carbon account: A systems analysis. Water, Air, & Soil Pollution: Focus (in press) doi:10.1007/s11267-006-9119-1
Nishimura, K. G., & Ozaki, H. (2001). Search and Knightian uncertainty. Working paper, University of Tokyo, Japan.
Rousse, O., & Sévi, B. (2007). The impact of uncertainty on banking behavior: Evidence from the US sulfur dioxide emissions allowance trading program. Water, Air, & Soil Pollution: Focus (in press) doi:10.1007/s11267-006-9109-3
Rypdal, K., & Winiwarter, W. (2000). Uncertainties in greenhouse gas emission inventories – evaluation, comparability and implications. Environmental Science and Policy, 107–116.
Sussman, F. (1998). Compliance and uncertainty in emissions inventories. Presentation to UNCTAD meeting on verification and accountability, London, UK, 6 April.
Sussman, F., Cohen, J., & Jayaraman, K. R. (1998). Uncertain emissions inventories, compliance, and trading. Presentation at Global Climate Change: Science, Policy, and Mitigation/Adaptation Strategies, Annual meeting of the Air & Waste Management Association, Washington, D.C., USA 13–15 October.
Taylor, J. R. (1997). An introduction to error analysis: The study of uncertainties in physical measurement, Second edition. Sausalito, CA, USA: University Science Books.
UNFCCC (2000). Views from parties on national systems, adjustments and guidelines under Articles 5, 7 and 8 of the Kyoto Protocol. FCCC/SBSTA/2000/MISC.1, 24 February.
Webster, M., Forest, C., Reilly, J., Babiker, M., Kicklighter, D., Mayer, M., et al. (2003). Uncertainty analysis of climate change and policy response. Climate Change, 61, 295–320.
Winiwarter, W. (2007). National greenhouse gas inventories: Understanding uncertainties versus potential for improving reliability. Water, Air, & Soil Pollution: Focus (in press) doi:10.1007/s11267-006-9117-3
Winiwarter, W., & Rypdal, K. (2001). Assessing the uncertainty associated with national greenhouse gas emission inventories: A case study for Austria. Atmospheric Environment, 5425–5440.
Acknowledgement
Portions of this work were supported by Environment Canada and the US Environmental Protection Agency. The views expressed herein are entirely those of the authors.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Gillenwater, M., Sussman, F. & Cohen, J. Practical Policy Applications of Uncertainty Analysis for National Greenhouse Gas Inventories. Water Air Soil Pollut: Focus 7, 451–474 (2007). https://doi.org/10.1007/s11267-006-9118-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11267-006-9118-2