Keywords

1 Introduction

Nowadays, as the world is getting further interlinked via technological platforms, business ecosystems represent a new era of business environments. The term Business Ecosystem was introduced by Moore [1, 2] as a metaphor inspired by natural ecosystems. Moore also stated that in a business ecosystem “companies co-evolve capabilities around a new innovation: They work cooperatively and competitively to support new products, satisfy customer needs, and eventually incorporate the next round of innovations” [1]. On the other hand, the research area of Collaborative Networks (CN) [3], which has a broader scope, identifies a business ecosystem as a form of CN, i.e. a case of Virtual organizations Breeding Environment (VBE), allowing a better understanding of its structure, lifecycles, endogenous characteristics, and exogenous interactions. As such, based on [4] and as defined in [5], a collaborative business ecosystem (CBE) can be modeled as an environment of agents, representing the organizations, which collaborate creating virtual organizations (VOs) to accomplish business opportunities.

Collaboration is expected to benefit both the participants of a CBE and the CBE as a whole. But the collaborative behavior of agents is likely to be influenced by the performance indicators adopted by the CBE. Thus, selecting a proper set of performance indicators for CBEs is of particular importance. In this work, a simulation model is proposed to study the evolution of a CBE when exposed to evaluation through a given set of indicators.

The remaining sections of this paper are organized as follows: section two presents the proposed performance indicators and metrics to be used for the CBE evaluation, acting as a mechanism of influence in the behavior of the organizations; section three presents the hypothesis proposing a performance assessment and adjustment model to address it; section four presents a simulation scenario using elements of agent-based modeling and system dynamics [6]. The last section discusses the contributions and future work.

2 Performance Indicators and Metrics for CBEs

Performance evaluation is an important issue being used in various fields, particularly in the business area, with the balanced score cards (BSC) [7] being the most well-known mechanism for individual organizations. However, for the present research we are interested on collaboration benefits and metrics, for which only limited contributions can be found in literature. As an example, in [8] an estimation model for business benefits in horizontal collaborative networks is proposed for product development. Another example suggests a set of collaboration benefits identifying cooperation variables and the corresponding target goals [9]. Another contribution for this research, is a conceptual model for value systems in collaborative networks [10], which suggests methods to assess the alignment of value systems of different members of a network [11]. For the case of supply chain collaboration there is a great multiplicity of metrics and methods, as found in [12,13,14] and [15], some of which can be adopted for collaborative networks in general and for CBEs in particular. Finally, the area of social network analysis provides a well-established set of metrics of structural network analysis, which can also contribute to CBEs assessment, namely as in [16, 17].

The performance indicators described in the next sub-sections, were chosen based on a previous literature analysis [18], and considering their relevance according to the dynamics and trends of current business environments [19]. Business ecosystems enable collaboration among multiple actors, the organizations, which diversity and collective ability leverage new ways to innovate and create value for society. The proposed performance indicators address these concerns by measuring the innovation due to collaboration, the new collaboration opportunities brought in or gained from the CBE, and the distribution of the collaboration among organizations in the CBE. These measures are expected to influence the organizations in self-adjusting their behavior, improving their performance and thus improving the CBE as a whole by promoting its sustainability.

Innovation Indicator.

The Innovation Indicator (IIi) of an individual organization member of a CBE, measures the potential of the organization to create new products, services or patents. The result is a ratio between the number of new products and the portfolio of the organization. Metrics and calculations for IIi are described in Table 1.

Table 1. Innovation indicator for an individual organization.
Full size table

The Innovation Indicator of the whole CBE (IICBE) sums the innovation potential of all organizations in the CBE. The resulting ratio is then weighted by the correlation (Spearman’s or Pearson’s) [20] between the number of collaborations (#VOs), and new products, services or patents (#NewPds) created. Metrics and calculations for IICBE are described in Table 2.

Table 2. Innovation Indicator of the whole CBE.
Full size table

Contribution Indicator.

The Contribution Indicator (CIi) of an individual organization member of a CBE, measures the capacity of the organization to create value, which is brought in or gained from the CBE’s new collaboration opportunities. Metrics and calculations for CIi are described in Table 3.

Table 3. Contribution indicator for an individual organization.
Full size table

The Contribution Indicator of the whole CBE (CICBE) is calculated by two indicators, the ratio of the total number of collaboration opportunities created in the CBE by the number of organizations in the CBE, and the degree to which the most active member exceeds the contribution of the others. Metrics and calculations for CICBE are described in Table 4.

Table 4. Contribution indicator of the whole CBE.
Full size table

Prestige Indicator.

The Prestige Indicator (PIi) of an individual organization member of a CBE, measures the influence/prominence of a member of the CBE, i.e., the likelihood of the organization be contacted towards potential collaboration opportunities. Metrics and calculations for PIi are described in Table 5.

Table 5. Prestige indicator for an individual organization.
Full size table

The Prestige Indicator of the whole CBE (PICBE) measures the average difference between the most influent member and that of all members of the CBE. Metrics and calculations for PICBE are described in Table 6.

Table 6. Prestige indicator of the whole CBE.
Full size table

3 Modeling a System for Performance Assessment of a CBE

The purpose of the indicators introduced in the previous section is anchored in the following hypothesis:

Performance indicators are a useful mechanism for assessing a CBE if they can contribute as a factor of influence for organizations to evolve, self-adjusting their behavior, and thereby improving the ecosystem performance and sustainability.

To verify the hypothesis, a CBE can be seen as a closed environment where organizations live, interact and collaborate with each other to realize business opportunities. The organizations are characterized by an endogenous behavior which can be influenced by the adopted performance indicators to evaluate the CBE (assessment engine). Then, according to the purpose of each indicator, organizations react differently depending on the characteristics of their profile which react to a set of factors of influence. As an example, organizations in a highly competitive and innovative CBE are likely to have a higher responsiveness to the innovation indicator. On the other hand, organizations in healthcare service delivery are more likely to respond to indicators of contribution and quality. In addition, organizations with a similar profile may react with different intensity to the same factors of influence, allowing the establishment of classes of responsiveness.

For the present study, considering the assumptions described above, we propose a Performance Assessment and Adjustment Model (PAAM) as depicted in Fig. 1. The purpose of this model is to explore the impact of indicators in the assessment of organizations in a CBE, namely how they can influence an improvement of behavior, thus contributing for a better performance of the CBE.

Fig. 1.
figure 1

PAAM (Performance Assessment and Adjustment Model) for a CBE.

Full size image

4 Modeling an Evaluation Scenario

In order to study the above ideas, the proposed PAAM was implemented as a system based on simulation, using agent-based modeling (ABM) and system dynamics (SD). ABM is used for simulating the actions and interactions of the autonomous organizations (the agents) in the CBE (the environment). SD is a mathematical modeling method to enhance learning in complex systems, supporting simulation models which help to understand the dynamics of complexity, allowing designing more effective policies and organizations [21]. Discrete elements (statecharts, events, timers, etc.) can be used to combine the different techniques and models [22], controlling state transitions, delays, or even capture exogenous values.

4.1 Setting-up the Simulation Model

Figure 2 illustrates the designed PAAM simulation system, where the organizations are autonomous agents, operating and collaborating in an environment which represents the business ecosystem. A system dynamics model simulates the evolution of the behavior of the organizations (and the CBE as a whole). This behavior is influenced by a mechanism of evaluation, i.e., when assessed according to the proposed performance indicators (II, CI, and PI).

Fig. 2.
figure 2

General schema of the simulation PAAM system.

Full size image

When a new business opportunity is acquired by an organization (Orgi), it is brought in the CBE as a new collaboration opportunity (CoOp), which triggers the creation of a virtual organization (VO) formed by a set of selected candidate organizations (the partners). The selection is based on the required competencies (matching skills) for that collaboration opportunity, and a ranking of the organizations according to a set of attributes which characterize their profile (attributes such as innovation index, accomplishment index, quality assurance index, and reputation index). These indexes (could be more) were selected for the simulation model, since they are directly related to the adopted performance indicators, i.e., when the CBE is evaluated through the calculation of the proposed performance indicators, this assessment is expected to induce some self-adjustment of the organizations’ profile, as there is a natural tendency of individuals and organizations to adapt to the way they are evaluated. Thus, the performance indicators act as factors of influence, causing different responses according to the profile and class of responsiveness of the organizations. For instance, as suggested in Fig. 2, the II influences de innovation index, the CI is more related to the accomplishment and quality assurance indexes, and the PI influences the reputation index.

4.2 Setting-up the Simulation Scenario

To set-up a complete simulation scenario to study de evolution of a CBE when assessed by the proposed performance indicators, we need to focus on the agent’s behavior. The agent’s behavior is modeled by system dynamics, and can represent a concern or a problem of the organization, as the generic example based on [21], depicted in Fig. 3.

Fig. 3.
figure 3

(based on [21]).

Behavior of a team trying to complete a project against a

Full size image

The example in Fig. 3, comprising a causal loop diagram, aims to explore the causes of late delivery for an organization’s design work [21]. The model shows the behavior of a team of engineers trying to conclude the tasks of a project against a deadline. The shorter the time remaining, the more schedule pressure they feel. When the pressure is high, the team has several choices. If they work overtime, the completion rate increases, thereby decreasing the number of remaining tasks and easing the schedule pressure (balancing loop B1). However, after a certain time, productivity drops due to fatigue, lowering the completion rate and increasing again the schedule pressure (reinforcing loop R1). Another way to meet the deadlines is to devote less time to each task, which increases the productivity as more tasks are done. Consequently, the schedule pressure is relieved (balancing loop B2), but the quality assurance is neglected, causing more faults, lowering productivity and forcing more scheduled pressure (reinforcing loop R2).

To complete the simulation scenario, the behavior of the agents, as the example illustrated in Fig. 3, should be expressed by a stock and flow map coupled to the PAAM system. Figure 4 depicts a zoom in one organization (Org1) showing the corresponding agent’s behavior. The factors of influence and class of responsiveness should be parametrized for each agent.

Fig. 4.
figure 4

(based on [21]).

Zoom in one organization (Org1), showing the corresponding agent’s behavior in a stock and flow map

Full size image

The ongoing work is focused on completing the development of the PAAM system using AnyLogic Multimethod Simulation Software [22], a tool that supports discrete event, agent based, and system dynamics simulation. The objective is setting-up a simulation scenario as described above, representing a CBE with organizations with different profiles (implying different factors of influence), and classes of responsiveness. By varying these parameters, it is expected that the model will allow understanding the evolution of the behavior of the organizations, namely their self-adjustment as a reaction to the assessment through the proposed performance indicators, and then verifying if that contributes for the improvement of both the organizations and the CBE as a whole.

5 Conclusions and Further Work

The performance indicators and metrics for CBEs presented in this paper, as well as the proposed PAAM system, contribute as a first approach to verify the hypothesis stated in section three, providing a simulation model to study the evolution of organizations in a CBE influenced by a given set of performance indicators. As such, the model is designed to allow identifying the indicators that lead to an improvement of the behavior of the organizations, thus improving the CBE and its sustainability.

The ongoing work, as mentioned above, comprises finishing the creation of the proposed PAAM system using AnyLogic Multimethod Simulation Software [22], to allow the configuration of multiple simulation scenarios, where the organizations in a CBE are represented by an ABM (agents in an environment), and the autonomous behavior of the organizations, by SD. “System dynamics seeks endogenous explanation for phenomena” [21]. The “interactions” between the agents and the variables, factors of influence, and responsiveness, are expected to provide a mechanism to study the effects of different sets of indicators.

The future work is aimed to continue the next steps of the modeling process [21]. i.e., after the problem articulation, dynamic hypothesis, and formulation, covered in the presented approach, comes next the testing phase, and the policy design and evaluation phase. The testing phase will consider concerns such as boundary adequacy, structure and parameter assessment, extreme conditions, behavior reproduction and anomaly, sensitivity analysis, among others. Finally, the policy design and evaluation phase allows designing and evaluating policies for improvement.

Thus, the proposed PAAM system, configured using several scenarios and parametrisation, is expected to reveal important insights, to make us understand the dynamics and evolution of a CBE when assessed by a given set of indicators.