Techniques for impact evaluation of performance measurement systems

Organizations often introduce performance measurement systems (PMSs) in order to evaluate the level of their performance, make comparison with competitors, and plan their future activities. Since indicators may affect the behaviour of the monitored system, the design and implementation of a PMS should always include the analysis of the impact it may exert on the organization itself. The aim of this paper is to suggest a methodology to evaluate this impact.

The proposed approach is based on an impact reference model derived from the balanced scorecard (BSC) framework. The different perspectives of the BSC are interpreted as areas of impact within an organization. Structured steps for impact evaluation are described and specific techniques of analysis are introduced.

A series of case studies, together with an analysis of advantages and disadvantages of the proposed method, are presented. Results show that, although many sets of indicators are usually able to meet the role of a PMS, they may exert a different impact on the context they are applied. The proposed methodology results to be a useful instrument for choosing the right set of indicators from the impact point of view. Finally, possible research paths to be undertaken for further developments of the proposed methodology are traced.

The application of the method is based on the assumption that managers charged with the analysis have a profound understanding of the specific contextual factors which may determine a reaction of the organization to a performance indicator or a PMS. Furthermore, at the moment, the methodology does not consider the possibility of interaction among different indicators in producing the impact.

This paper may be used to guide the selection of the most appropriate PMS from the impact point of view. The proposed methodology can be very helpful instrument for an organization involved in the design of new PMSs. It guides the decision maker through the various phases: indicators definition, analysis of their properties, impact analysis, and choice of the set with the preferable impact profile.

The issue of impact has been long debated in literature. Many articles try to analyse the operative and strategic consequences of the introduction of a PMS in an organization. This paper proposes a methodology for a more structured and objective evaluation of the impact of new PMS before introducing it in a firm. This can result in a significant help for manager who have to find the best set of indicators for the performance evaluation of their organization or have to choose between two or more sets of indicator satisfying, in principle, the same representation objective.