Environmental KPI Selection Using Criteria Value and Demonstration

Abstract

Determining key performance indicators (KPI) is a first step in achieving environmental sustainability of manufacturing operations. KPI selection is a multi-criteria decision making problem, because of various criteria that must be considered. Intuitively, one can rank candidate KPIs by specifying a numerical value indicating the effectiveness of a KPI in satisfying each criterion. However, linking selection criteria to KPI objectives, ranking how well a KPI satisfying a given criterion, and assigning a value to each rank lead to better KPI rankings. Values for each score are crucial. This paper shows steps to capture values to derive a criterion value function that is used to rank candidate KPIs. Selected KPIs can be used for assessing and monitoring sustainability performance, which must be considered together with including traditional (e.g., throughput) measures. A machine shop is used to show how an objective of reducing emissions from energy use in manufacturing can be pursued by monitoring the energy used to produce a unit product.

1 Introduction

Achieving sustainability of manufacturing processes requires efficient and effective methods for defining, selecting, deploying, and monitoring key performance indicators (KPIs). Selecting KPIs is a multi-criteria decision making problem for which several methods are available. One approach is to let stakeholders assign a score of a KPI in satisfying a selection criterion. This process is repeated for each candidate KPI and final KPI ranks are obtained from aggregation of these scores.

Ezell (2007) and Collins et al. (2016) showed enhancement to this approach by capturing stakeholder “value” for each score point on the Likert scale. Our previous research also used a multi-variate value model where stakeholders score (and provide value of) each candidate KPIs against each selection criterion (Kibira et al. 2017). The score represents the degree to which a KPI satisfies a criterion. Each selection criterion is linked to a KPI objective. This way, each KPI is evaluated for its contribution to the defined environmental objective. The developed procedure has been submitted to the American Society for Testing and Materials (ASTM) as a work item to become a standard guide for identification and selection of environmental KPIs for manufacturing processes (ASTM International). This procedure requires an in-depth understanding of developing and using value functions for selection criteria. This process and the linking of criteria to KPI objective are discussed in this paper within the context of environmental assessment of manufacturing processes. This paper also demonstrates KPI deployment and performance monitoring in a machine shop.

Typically, value functions capture experts’ assessment of the value of each score of a KPI against a criterion. To develop a value function, Duarte et al. (2006) first defined the minimum and maximum possible measures of the score but assumed a linear relationship between assigned score and value. Keeney and Lilien’s (1987) developed a method of assigning value at salient points on a common probability distribution. This paper derives criteria value functions using a combination of above-mentioned approaches.

The rest of the paper is organized as follows. Section 2 presents concepts of the value model and shows steps to develop a value function. Section 3 presents examples of developing and using value functions. Section 4 presents a demonstration of implementing KPIs for performance monitoring. Section 5 is a discussion and conclusion of the paper.

2 Criteria Value Based KPI Ranking and Deployment Process

Figure 1 shows our proposed approach for selecting and implementing KPIs for a manufacturing system based on value. This section overviews the steps and the process from identifying selection criteria to ranking the candidate KPIs.

Fig. 1.

Process and steps of KPI selection and demonstration

Identifying KPI Selection Criteria:

Selection criteria should be fundamental to the KPIs as opposed to being a means to another criterion (Keeney and Lilien 1987). For example, a criterion such as “quantifiable” can be a means to ensure that a KPI is “calculable.” Therefore, these two may not need to appear in the same criteria set. KPI objectives are used to identify criteria and are obtained from sustainability goal(s). For example, if reducing energy use by, say, 20% is the target, it implies that KPI should be measurable and/or computable. To obtain a complete representative list, criteria groups, e.g. financial-oriented or management-oriented criteria. To keep the analysis manageable, a decision can be made to select the best 5–10 criteria to make up the set.

Candidate KPIs:

Typically, candidate KPIs are proposed by top management and presented to organizational units responsible for KPI implementation so they can be evaluated. However, the candidate KPIs can also be identified if there is a gap between KPIs currently in use and those that are needed to achieve environmental objectives.

Value Model for KPI Ranking:

For a value model, (1) each criterion is weighted for its contribution, and (2) each candidate KPI is measured against each measure criterion. Most previous researchers used the additive model to compute total value of a candidate KPI (Keeney and Lilien 1987; Duarte et al. 2006; Ezell 2007; Collins et al. 2016). Thus:

$$ v(x_{1} ,x_{2} , \ldots ,x_{n} ) = \sum\limits_{i = 1}^{n} {w_{i} v_{i} (x_{i} )} $$

(1)

where v is the overall value function, and v _i (x _i ) are the individual criteria values at measurement level x _i , w _i are the scaling constants (weights), whose total should equal to 1.

$$ \sum\limits_{i = 1}^{n} {w_{i} = 1} ,\;w_{i} > 0 $$

(2)

Value Function Development

Horizontal Measurement: This measurement scale is used to indicate the degree to which a KPI satisfies a criterion. After identifying this scale, the minimum and maximum possible values are specified. For example, Table 1 shows measurement scales as well as minimum and maximum values for three of the criteria described in Kibira et al. (2017).

Table 1. Measurement scale of sample KPIs

Vertical measurement:

Values for each level on the horizontal scale are determined by analysts and subject matter experts (or stakeholders). KPI values increase with degree of satisfaction of each criterion by the KPI. Therefore, value functions for KPI selection would in general exhibit an increasing trend. Alarcon et al. (2011) proposed four relationships (i.e., linear, convex, concave, and S-shaped) that a value function can take, as seen in Fig. 2.

Fig. 2.

Common shapes of the value function

To determine the actual values, let the minimum measurement be designated x _min and maximum be x _max . On a 0–1 scale for value, v(x _min ) = 0 and v(x _max ) = 1. Keeney and Lilien (1987) preferred to start with the mid-value (designated x’). The subject matter expert or stakeholders determines this value, where v(x’) = 0.5. Other points between x _min and x’ and between x’ and x _max can be determined to yield additional points on the function v. If sufficient points can be garnered using experts, a sketch can complete the graph of the function.

KPI Ranking:

Next, for each KPI in the candidate set stakeholders independently assign a score showing agreement that the KPI satisfies the criterion. A value is obtained from the value function for each score. An average is calculated for the values obtained from all stakeholders for each criterion for each KPI.

3 Examples of Applying Steps in Criteria Value-Based KPI Selection

This section presents examples to illustrate how the steps may be applied. Some of the processes illustrated in Fig. 1 are demonstrated in Kibira et al. (2017). Discussion will be on KPI selection criteria, value function development and KPI demonstration.

Selection Criteria:

These are specified by production managers, supervisors, and shop floor workers. Lower-level KPI objectives for reducing material consumption include reduction in virgin material use and increase in use of recycled materials. The KPI objectives are used to identify criteria that would meet these objectives.

Value Function for “Cost Effectiveness” Criterion:

This criterion implies the degree of perceived cost benefit of implementing the KPI. Let the measure for this criterion be expressed as the “savings” measured on a scale from 0 to 10, which is the difference between the income (or saved costs) and expenditure of implementing a KPI. Let the minimum savings, S_min, be 0 and the maximum, S_max, be 10.

The next step is to determine the shape of the value function. If savings through monitoring KPIs is a new strategic approach, any efforts in that direction are greatly encouraged. Therefore, initial measures are highly valued. A concave shaped value function, where the increase in value is maximized at the point of minimum measure, is suitable. See Fig. 2b. As you progress towards the maximum, the curve is more horizontal as the decision maker would generally assign less value to additions to high-level savings.

Once the general shape is established, what follows is to determine salient points on the curve. The expert is asked to express “How much savings, say y, such that the value from the minimum to y is equal to the value from y to the maximum?” Let this savings be labelled S’ and x ₁ designate the cost effectiveness criterion. On a scale from 0–1, v₁(0) = 0, v₁(S’) = 0.5, and v₁(S_max) = 1. Proceeding from this point, mid-value assessments are made for additional pairs of cost effectiveness levels to generate other data points.

Ranking KPIs:

Stakeholders independently assigned a score on the measurement scale for each KPI against each criterion. The value corresponding to this score was obtained from the value function. The final value of a KPI was a sum of values obtained from all stakeholders for all the criteria. The values (obtained from the value function) are scaled to the 0–10 range. All three stakeholders perform the same process and their results averaged. The final ranking in an example used in (Kibira et al. 2017) is summarized in the chart in Fig. 3. This chart shows that the “energy per part” KPI has the highest rank. This is used for monitoring energy performance in the demonstrated machine shop.

Fig. 3.

Final assessment of individual KPIs (Kibira et al. 2017)

4 KPI Demonstration

This section shows the monitoring of performance using the highest ranking KPI, i.e., energy per part. Let us assume that the high-level goal was “to reduce global warming potential due to energy consumption in the manufacturing process without compromising throughput.” To evaluate achievement of the above goal, it is necessary to break down energy consumption into lower level objectives and to monitor the energy use.

We use a case study of a small machine shop that manufactures metal products. The shop comprises of a foundry, one milling machine, one lathe machine, a drilling machine, and an ultrasonic inspection center. There are two classes of products: A and B. Figure 4 shows the work flow. Production starts when the parts are loaded onto the shop. After casting, A requires turning operations while B requires milling. Final process is drilling although some of A do not use it. All parts pass through the inspection station.

Fig. 4.

Workflow through the shop

Energy Modeling and Simulation:

A discrete event simulation model of the shop was constructed using AnyLogic simulation software tool. To attribute energy to a unit part, we use a framework such as that developed by Seow et al. (2011). Two types of energy are distinguished: direct and indirect. Direct energy is the type used in the actual production process, e.g., heat to melt metal. Indirect energy is that used in the ambient working area such as heating, ventilation, and air-conditioning (HVAC), and lighting.

Direct Energy: The direct energy for the casting process is obtained by combining energy used to bring the metal from room temperature to melting temperature and the fusion energy required. Machining energy is related to machining time. Both these quantities are calculated using empirical expressions for machining of mild steel products (Sonmez et al. 1999; Sardinas et al. 2006). For product inspection, the energy consumed is equal to the energy rating of the ultrasonic tester multiplied by duration of the inspection.

Indirect Energy: Indirect energy consumed in each section of the shop depends on the type of manufacturing activity. Indirect energy is calculated by considering HVAC and lighting rating requirements for manufacturing activities carried out in these sections. Energy per part is obtained by dividing the result by the total of parts produced.

Simulation Output:

Simulation experiments are carried out to investigate the impact of batch size on both energy consumptions per part. Batch size is the variable used because it affects many factors including setup time and setup cost, inventory levels, lead times, safety stock, and order fulfilment. In general, small batch sizes are associated with higher overall set-up time while large batch size, without lot-splitting, can lead to increased idling and thus, reduced throughput. The effect of batch size is investigated for its effect on energy consumption and throughput in the multi-stage production environment. Note that set-up time for casting is not a constant for all batch sizes.

The energy consumption is shown in the graph in Fig. 5. For each experiment, Parts A and B are loaded alternately onto the shop in batches of equal size. Batches are varied from an initial size of 5 units. For a batch-size less than ten units, there is significant increase in energy use per part while any batch sizes exceeding 20 units, the decrease in energy use is not significant. On the other hand, the total number of units produced falls almost evenly with increase in batch size. The decision maker can use this graph to balance energy per part and throughput for each situation.

Fig. 5.

Variation of energy consumed per part with batch size

5 Summary and Discussion

This paper has discussed using value of a score for a KPI against a criterion to evaluate a candidate KPIs. This approach was adopted after realizing that the relationship between such scores and benefit from the score is not always linear. Thus, the concept of value of each score, as a basis for decision-making is relevant to the KPI selection process and largely hinges on constructing value functions for each criterion.

Expert knowledge and stakeholder contribution is crucial for deriving value functions used for ranking candidate KPIs. Selection for KPIs is based on the resulting ranks. Selected KPIs can be used for assessing and monitoring environmental measures such as energy consumption. Analysis and tradeoff can be made between different measures. Simulation modeling may be applied to further investigate performance due to decisions made at different control levels as well as possible interactions between different KPIs.