1 Introduction

Process Mining (PM), a specialized form of data-driven process analytics, enables the extraction of detailed insights regarding process behavior, process performance, conformance of processes to existing process models, and process improvement opportunities from event logs (van der Aalst 2016). Due to their data-driven character, PM insights support evidence-based process improvement (Partington et al. 2015) and strategic decision-making (Mans et al. 2013), which are important enablers of digital transformation (Kerremans et al. 2020). The significance of PM is illustrated by its widespread adoption across a wide range of industries and the rise of PM vendors such as Apromore, Celonis and Fluxicon (Reinkemeyer 2020).

Over the last decade, substantial research in the area of PM has been conducted (Thiede et al. 2018). Significant efforts on the technical side have resulted in maturing PM algorithms. Recently, research attention has been devoted to topics such as predictive process monitoring (e.g., Kratsch et al. 2020; Teinemaa et al. 2019), data quality (e.g.,  Fischer et al. 2020), and algorithms for specific domains such as healthcare (e.g.,  Chiudinelli et al. 2020). While prior research has mainly focused on the development and improvement of PM algorithms, case studies reporting on the application of PM have also been published (e.g., Andrews et al. 2020). These case studies typically provide rich insights into the use of one or more PM techniques in specific organizational contexts and draw lessons regarding points of attention for the use of PM in the organization (e.g., Reinkemeyer 2020). Though being highly valuable, such insights are often limited to the boundaries of a single organization. A more generic and holistic understanding of the opportunities and challenges of using PM in organizational settings would complement existing insights based on case studies. Such an understanding is needed to guide research by revealing barriers for PM adoption and highlighting avenues that the PM research community should explore to contribute to successful PM initiatives. In industry, it could help to shape the roadmap for PM initiatives leveraging relevant opportunities, while being fully conscious about the prevailing challenges. Against this background, this paper investigates the following research question: What are the opportunities and challenges of using PM in organizations?

To approach the research question, we perform a Delphi study with PM experts from both academia and industry. Delphi studies are an explorative qualitative research method, consisting of multiple rounds in which experts provide and collaboratively reflect on insights into a given topic (Kerpedzhiev et al. 2021). Our Delphi study resulted in a list of 30 opportunities and 32 challenges relevant for the successful use of PM in organizations. The opportunities and challenges are structured along the BPM core elements (i.e., strategic alignment, governance, methods/information technology (IT), people, and culture; de Bruin and Rosemann 2007) and, for the opportunities regarding methods/IT, along the phases of the BPM lifecycle (i.e., process discovery, process analysis, process redesign and implementation, and process monitoring and controlling; Dumas et al. 2018). In addition to the list of opportunities and challenges, we also provide insights into their comparative relevance as judged by the expert panel, as well as the differences in the perceived relevance between academics and practitioners. Hence, this study imparts an understanding of the areas in which PM can contribute and indicates points of attention when adopting PM. Furthermore, it highlights important avenues for future research in the area of PM.

The rest of the paper is structured as follows: Sect. 2 provides background and related work. Section 3 outlines the study design and research method. Section 4 presents the main findings of our study. Section 5 presents a discussion, including relevant contributions and limitations of the study. The paper ends with a conclusion and outlook into future research in Sect. 6.

2 Background and Related Work

2.1 Business Process Management

BPM is “a body of methods, techniques and tools to identify, discover, analyze, redesign, execute and monitor business processes in order to optimize their performance” (Dumas et al. 2018, p. 6). A business process represents “a collection of interrelated events, activities and decision points that involve a number of actors and objects, which collectively lead to an outcome that brings value to at least one customer” (Dumas et al. 2018, p. 6). From a process lifecycle perspective, BPM includes the activities of process identification, discovery, analysis, redesign, implementation, and monitoring and controlling (Dumas et al. 2018; Rosemann and vom Brocke 2015c). Apart from the lifecycle perspective, BPM has been increasingly conceptualized as a corporate capability that can be decomposed into distinct capability areas, typically grouped into respective capability frameworks (Van Looy et al. 2014; Kerpedzhiev et al. 2021). Accordingly, many BPM capability frameworks and maturity models have been proposed over the years (Van Looy et al. 2017). One comprehensive and widely-adopted framework is that of de Bruin and Rosemann (2007), which consists of thirty capability areas grouped along the six core elements of BPM: strategic alignment, governance, methods, IT, people, and culture. Since we deemed the core elements accurately reflected experts’ input, we used them to structure the challenges and opportunities from the second round onwards. Section 3 provides details on this decision.

2.2 Process Mining

2.2.1 Development of the Field

PM, a field situated at the intersection between data science and process science, aims to extract knowledge from event logs to discover, monitor, and improve business processes (van der Aalst 2016). Hence, it supports various BPM lifecycle phases using data about the execution of a process (van der Aalst et al. 2012; van der Aalst 2016). Relevant research efforts tend to focus on the development and improvement of PM algorithms for various use cases. Initially, PM research centered on control-flow discovery, i.e. retrieving a process flow model from an event log. While control-flow discovery has remained an important use case (Augusto et al. 2018), PM research has broadened its scope over time to include techniques for checking conformance between a control-flow model and an event log (Carmona et al. 2018), gaining insights in the involvement of resources in a process (Song and van der Aalst 2008), or connecting PM to other techniques such as simulation and predictive process monitoring (Kratsch et al. 2020; Martin et al. 2016; Teinemaa et al. 2019). While many of the state-of-the-art PM algorithms have been integrated into the open-source platform ProM, the use of PM in organizations has been stimulated by the development of commercial tools such as Apromore, Celonis, and Disco (van der Aalst 2016).

2.2.2 Use of Process Mining in Organizations

In the related work regarding the use of PM in organizations, a distinction can be made between business reports and academic literature. Business reports typically list common use cases and vendors based on their assessment of the market. HSPI, for instance, compiled 551 case descriptions (HSPI 2020). In another business report, Koplowitz (2020) argues that PM enhances transparency in the organization and includes an overview of commercial vendors. An overview of vendors, is also part of Gartner’s annual market guide for PM, together with an outline of, e.g., the main drivers of PM adoption and common use cases (Kerremans et al. 2020). Recently, Deloitte surveyed early and mature PM adopters to gain insights in, amongst others, PM expectations, critical success factors, application areas, and the next steps in terms of PM deployment within the organizations (Galic and Wolf 2021).

In the academic literature, two main research streams regarding the use of PM in organizations can be distinguished: (1) finer granularity: targeted PM case studies and (2) coarser granularity: dedicated research on the general use of PM in organizations. Other works, focusing on PM tool comparison and selection (e.g., Agostinelli et al. 2019; Drakoulogkonas and Apostolou 2021; Turner et al. 2012), are relevant to support the uptake of PM, but are dedicated to the technical aspects of certain techniques or tools. As a consequence, they will not be elaborated upon.

Finer granularity: targeted PM case studies

Case studies focus on the application of existing PM techniques in a specific organizational context. Due to the profound analysis of the specific organizational setting, case studies can deliver rich insights regarding the opportunities and challenges of PM within this context. For instance, from an analysis of the patient transportation process after traffic accidents in Queensland, Andrews et al. (2020) conclude that the composition of the event log was very challenging and that PM algorithms could not always handle the process complexity. Moreover, domain expertise is deemed indispensable, both to obtain readable outcomes and to identify improvement options (Andrews et al. 2020). Similar to Andrews et al. (2020), Mahendrawathi et al. (2015) also observe that the composition of an event log was far from trivial. By studying the customer fulfillment process at a telecommunication company, they especially highlight the challenge of integrating data which is dispersed over different systems (Mahendrawathi et al. 2015). In his recent compendium, Reinkemeyer (2020) compiles twelve PM cases from various application areas such as manufacturing and finance. Besides the impact of PM, the cases also report on lessons learned. An example of such a lesson is that PM should be leveraged to improve processes for the future, not to blame staff members for suboptimal processes in the past (Reinkemeyer 2020).

While the aforementioned works report on original case studies, other contributions review published case studies. These reviews either cover several sectors (Emamjome et al. 2019; Thiede et al. 2018) or focus on a particular sector such as healthcare (Rojas et al. 2016) or education (Ghazal et al. 2017). Thiede et al. (2018) review 144 empirical studies and conclude that PM mainly focuses on a single system in a single organization, showing the need for efforts on cross-system and cross-organizational PM (Thiede et al. 2018). Following a similar research approach, Emamjome et al. (2019) survey 152 case studies and study their thoroughness over time. They observe that the thoroughness with which the problem context is considered during a PM project is not increasing. From this, Emamjome et al. (2019) conclude that PM is currently unable to deliver on its promises in a real-life context and they stress the need for more methodological guidance.

Case studies provide rich and detailed insights into the complexity of using PM in real-life contexts, but typically focus on a specific use case in a particular organization. In turn, conclusions of reviews of published case studies need to be reflected against their foundation on secondary data. Our study complements this line of research as it serves a different purpose, i.e., providing a generic and holistic understanding of the opportunities and challenges of using PM in organizations. To this end, primary data is collected from PM experts with extensive experience on the use of PM in organizations. Moreover, our study abstracts from particular PM use cases or organizational contexts, yielding a broad overview.

Coarser granularity: dedicated research on the general use of PM in organizations

Another stream of research, which is the closest to our work, studies the use of PM in organizations without starting from a particular use case. To this end, data is gathered to understand the organizational mechanisms behind the use of PM using methods such as questionnaires, interviews, or a focus group.

Early 2012, Claes and Poels (2013) surveyed 90 respondents (academics, practitioners and students) about the use of PM. At the time, the most frequently mentioned benefits of PM were its objective nature, as well as the speed to generate results once the data is ready for analysis. Commonly mentioned limitations of PM were limited data access, high costs of data preparation, issues with data quality, lack of intuitiveness and guidance, and difficulties to understand PM output (Claes and Poels 2013). In the same time period, Mans et al. (2013) identified six success factors for PM projects based on a literature review and eight interviews spread over four organizations. The resulting success factors were sufficient management support, adequate project management, presence of domain expertise, presence of technical PM skills, the level of structure in the PM approach, and the quality of raw data (Mans et al. 2013).

Recently, two targeted studies on the adoption and use of PM in organizations have been published. Syed et al. (2020) explore the adoption of PM in a Dutch pension fund which recently started to use a commercial PM tool. Based on nine stakeholder interviews, seven challenges and four enablers of PM adoption were identified. The challenges include the absence of governance mechanisms, a disconnect between the design team and end-users, and the presence of data quality issues. Conversely, PM adoption is enabled by the ability to generate actionable insights, the presence of confidence in the outcomes, the perceived benefits of PM, and the availability of adequate PM training (Syed et al. 2020).

While Syed et al. (2020) focus on a single organization in the early stage of PM adoption, Grisold et al. (2020) take a broader perspective by reporting on a focus group with 22 process managers. Their key findings include that (1) creating business value with PM is possible, but hard to quantify, (2) selecting the appropriate processes to use PM is challenging, (3) gathering the required data is far from trivial, (4) using PM requires coping with increased transparency, and (5) aligning PM initiatives with the strategy and operations is required. Despite the relevance of their work, the authors recognize the rather limited geographical spread of the experts (Switzerland, Germany and Liechtenstein) and the limited focus group duration as limitations of their design. Hence, they stress the need for additional studies which explore the organizational use of PM (Grisold et al. 2020).

The aforementioned works provide valuable pointers on the use of PM in organizations based on a single consultation of experts affiliated to one or several organizations. In our study, we aim to extend this stream of research by providing a structured and holistic overview of opportunities and challenges regarding the use of PM in organizations. To the best of our knowledge, such an overview is absent in literature. To achieve this goal, we conduct a Delphi study with an international panel of academics and practitioners, in which experts are involved in multiple study rounds, interspersed with feedback (Okoli and Pawlowski 2004).

3 Study Design and Research Method

3.1 Delphi Study as a Research Method

To approach the research question regarding opportunities and challenges for the use of PM in organizations, we decided to conduct a Delphi study. Delphi studies are a well-established method in IS and BPM research that strives for consensus on a specific topic with a panel of experts over multiple rounds utilizing questionnaires interspersed with feedback (Gupta and Clarke 1996; Skinner et al. 2015). Prominent examples of the use of the method in the IS and BPM literature include the identification of BPM capabilities in the digital age (Kerpedzhiev et al. 2021), the identification and rating of reasons for process deviance (König et al. 2019), the establishment of criteria for selecting cloud service providers (Lang et al. 2018), the identification of grand challenges for IS (Becker et al. 2015), the examination of the constituent values of a BPM-supportive cultural setting (Schmiedel et al. 2013), and the identification of perceived benefits of process modeling (Indulska et al. 2009).

The Delphi method poses that experts remain anonymous throughout the study to avoid any bias resulting from confrontation or defending preconceived notions (Okoli and Pawlowski 2004; Skinner et al. 2015). In each round, the experts’ opinions and feedback are anonymized, consolidated by the research team, and shared with the panel to be reviewed until predefined termination criteria are met (Paré et al. 2013). Different Delphi study setups such as a classical, policy, decision, and ranking-type Delphi have been proposed over the years. The corresponding rounds can focus on brainstorming, validation, narrowing-down, or ranking (Paré et al. 2013). To ensure the sound use of the Delphi method, multiple rigor criteria and good practices have been proposed, which we observed when conducting our Delphi study (Keeney et al. 2006; Okoli and Pawlowski 2004; Paré et al. 2013; Schmidt 1997). These criteria include, amongst others, reporting essential background information on the experts’ demographics and background, ensuring anonymity throughout the study, and providing clear instructions in all phases of the study.

3.2 Study Design Decisions

In line with our goal of identifying and ranking opportunities and challenges for the use of PM in organizations, this section outlines important design decisions in terms of setting up and conducting the Delphi study. We communicated these design decisions to the experts before and during the study and allowed the experts to comment on them.

Firstly, in accordance with the goal of our research—the holistic identification of opportunities and challenges for the use of PM in organizations, as well as getting insights into their comparative relevance—we followed the blueprint of ranking-type Delphi studies as proposed by Schmidt (1997). Ranking-type Delphi studies are the most commonly used Delphi blueprint in IS research (Paré et al. 2013) and include a brainstorming, narrowing-down, and ranking phase. In our study, we opted for rating (i.e., the assignment of opportunities and challenges to predefined ordinally-scaled relevance categories) instead of ranking (i.e., the assignment of opportunities and challenges to ordered ranks in line with their relevance) the opportunities and challenges for PM in organizations (König et al. 2019). Since ranking is only feasible for a small number of items, it would have required an artificial restriction of the number of opportunities and challenges to reduce the cognitive load on the experts. As this is not consistent with the holistic scope of our study, we opted for rating instead of ranking.

Secondly, we aimed for a comprehensive identification of opportunities and challenges for the use of PM in organizations, while ensuring unrestricted expert input. Therefore, we did not formally require that opportunities and challenges are exclusive to PM, provided that they have a PM-specific relevance or a specific interpretation in the context of PM. To ensure a common understanding of an opportunity and a challenge, we provided the experts with PM-centric definitions of both concepts.

Definition 1

(Opportunity) An opportunity is a favorable circumstance or an expected benefit for an individual, a team, or an organization enabled by the use of PM in organizations.

Definition 2

(Challenge) A challenge is a difficulty or an obstacle that arises when using (or intending to use) PM in organizations, and that requires a lot of energy and determination from an individual, a team, or an organization to overcome.

Further, from the onset of the study, we encouraged experts to consider a wide range of perspectives on PM by briefly referring to the well-established and comprehensive six core elements of BPM (i.e., strategic alignment, governance, methods, IT, people, and culture; de Bruin and Rosemann 2007). However, we purposefully refrained from using the six core elements as a structure for the expert input in the first round of the study and employed a greenfield approach to avoid constraining the experts into predefined categories. Based on the input we received in the first round, we reviewed several well-established frameworks from literature as a potential lens to structure opportunities and challenges. We chose the core elements as structuring elements as they fulfilled the goal of providing a holistic lens for the experts’ input. Furthermore, the core elements define the structure of a comprehensive and widely-adopted BPM capability framework—the one of de Bruin and Rosemann (2007). The core elements and the corresponding BPM capability framework are firmly anchored in the BPM literature: they have been used to structure the Handbook on BPM (Rosemann and vom Brocke 2015a, b), as well as real-world BPM success stories (vom Brocke and Mendling 2018). Further, Kerpedzhiev et al. (2021) have recently used the core elements to identify BPM capability areas in view of digitalization. In our study, we merged the core elements methods and IT because there are rarely methods without IT support in the PM field and vice versa (Kerpedzhiev et al. 2021).

Where appropriate, we further assigned specific opportunities within the core element methods/IT to the BPM lifecycle phases (process discovery, process analysis, process redesign, process implementation, and process monitoring and controlling; Dumas et al. 2018) to reflect the subtle nuances in the experts’ input. Thereby, we merged process redesign and process implementation as the expert input in the first round that related to these phases was closely intertwined. To ensure a common understanding of the relevant input categories, we provided the experts with PM-centric definitions of the used BPM core elements and lifecycle phases (Table 1).

Table 1 PM-centric definitions of the BPM core elements and lifecycle phases
Full size table

Thirdly, we invited experts from both academia and industry to participate in the study to account for the diversity of the PM field (Okoli and Pawlowski 2004; Paré et al. 2013). Thus, we aimed to recruit experts from different countries, backgrounds, and PM subcommunities (Schmiedel et al. 2013). Regarding the selection of academic and industry experts, we posed formal requirements that participants in the study had to fulfill (Okoli and Pawlowski 2004). Thus, we required academic experts to have a Ph.D. degree in a field related to PM as well as at least five years of relevant academic experience. Additionally, they had to have been involved in at least two real-life applications of PM over the five years prior to the study. Practitioners had to have at least three years of work experience and at least one year of experience regarding the use of PM. Inviting both academics and practitioners also allowed us to get valuable insights into differences between both groups regarding the comparative relevance of opportunities and challenges. Therefore, we intended to split the panel into an academic and industry subpanel in the rating phase of the study.

Finally, we judged the quality and convergence of the study’s results both quantitatively and qualitatively. From a quantitative perspective, we followed the well-established practice of measuring the experts’ satisfaction with the coding of opportunities and challenges (coding satisfaction) as well as their overall satisfaction with the study (Kerpedzhiev et al. 2021; Schmiedel et al. 2013). We employed a 7-point Likert scale: 1 (extremely dissatisfied), 2 (moderately dissatisfied), 3 (slightly dissatisfied), 4 (neither satisfied nor dissatisfied), 5 (slightly satisfied), 6 (moderately satisfied), and 7 (extremely satisfied). Using such measures ensured an objective indication of the convergence of the study in addition to the qualitative feedback of the experts in each round. It also allowed us to check for selection bias ensuring that satisfaction ratings had not risen because dissatisfied experts dropped out, but because the experts had become more satisfied with the results (Heckman 2010). Overall, we aimed for a positive trend in the quantitative measures, as well as a high level of satisfaction (mean above 6 and standard deviation below 1), accompanied with positive qualitative expert feedback (Paré et al. 2013).

3.3 Preparatory Activities

To ensure the most suitable expert panel composition, we invited experts in line with the selection criteria mentioned in Sect. 3.2 (Okoli and Pawlowski 2004). Given the required commitment and the level of experience necessary to successfully participate in the study, we primarily recruited experts from our academic and professional networks. We first identified a longlist of 87 candidates for the expert panel, 32 academics and 55 practitioners. Of the latter, 30 candidates had a PM function in their respective organizations, 14 were PM consultants, and 11 represented PM tool vendors. After reviewing the longlist of the 87 initially identified candidates, we determined that 57 fulfilled the established formal selection criteria (21 academic and 36 industry experts). We invited those 57 experts and allowed them to nominate further candidates they deemed suitable for the purposes of the study. 41 experts agreed to participate in the study, amounting to a high initial response rate of 72% (one expert did not participate in Round 1).

The panel was well-balanced in terms of technically- and business-oriented experts, as well as in terms of academics and practitioners. It also covered experts from 16 countries. The 17 academic experts held their Ph.D. for an average of 12.2 years and had 10.4 years of experience in PM research on average, while practitioners had an average of 20 years of work experience with an average of 6.4 years of experience with PM. Based on the open questions in the first round of the study, we determined that academic experts had experience in using PM in diverse fields such as healthcare, logistics, manufacturing, and financial services. More background information on the panel can be found in Appendix A (available online via http://link.springer.com).

To test the study setup specifically for the initial brainstorming in the first Delphi round, we conducted a pilot study in line with the established quality criteria for Delphi studies (Kerpedzhiev et al. 2021; Skinner et al. 2015). The aim of the pilot study was to ensure the understandability of the questionnaire for the first brainstorming round, identify confusing and/or ambiguous wording, and ensure an appropriate level of detail is achieved (Skinner et al. 2015). The questionnaire included a description of the goal of the research, a comprehensive definition of PM as well as the specific assignment along with the aforementioned definitions for an opportunity and a challenge. We distributed the questionnaire to eleven Ph.D. students in the field of BPM and asked them to fill it out. After the students had provided their input, we invited them to a joint live session to discuss any issues that compromised the understandability of the questionnaire. We received overwhelmingly positive feedback coupled with high-quality responses with direct relevance to the topic. Based on the feedback, we performed only slight changes to the wording of the questionnaire to further enhance its understandability.

Before starting the study, we also agreed on common guidelines when coding experts’ responses in the brainstorming and validation rounds. In each round, one designated co-author anonymized the experts’ responses so that the rest of the author team could code the input independently. We used axial and selective coding (Moghaddam 2006). Thus, each co-author involved in coding first aimed to summarize similar items in the experts’ input and merged the relevant explanations before proceeding to establish relationships among them by categorizing the data (Schmidt 1997). We iteratively consolidated the results of each involved co-author in joint workshops (Okoli and Pawlowski 2004). After each workshop, all coded responses were traced to the input of at least one expert to ensure that they accurately reflected the experts’ ideas and not ours. We also established and agreed on guidelines on the formulation of opportunities and challenges (Schmidt et al. 2001). In that respect, we strived for short denominations with positive polarity and single-sentence descriptions without domain-specific and/or technology-centered vocabulary.

3.4 Delphi Study Procedure and Key Figures

The Delphi study comprised six rounds and took two and a half months to complete. In each round, the experts had one week to fill in an online questionnaire.Footnote 1 They always received the option to provide open-ended comments on the ongoing round and the study in general. Additionally, as per relevant quality criteria for Delphi studies, the questionnaire for each round included detailed instructions, definitions, responses from the previous round, and an overview of the changes compared to the previous round (Keeney et al. 2006; Paré et al. 2013; Skinner et al. 2015). Table 2 provides an overview of the key figures regarding the Delphi study such as the number of participants and the satisfaction scores per round. In the remainder of this subsection, we provide a short overview of the Delphi procedure and elaborate on some key insights regarding the experts’ participation in and satisfaction with the study. To avoid introducing redundancy between Sects. 3 and 4, we already present key figures from the operationalization of the Delphi study procedure in the following. A detailed account of each round can be found in Appendix B (available online via http://link.springer.com). In Sect. 4, we will focus on the core results in terms of the final opportunities and challenges, including the respective ratings after the last round of the study.

Table 2 Overview of the Delphi study procedure and statistics
Full size table

In the brainstorming phase, we collected initial lists of opportunities and challenges for the use of PM (Round 1). We received 215 opportunities and 211 challenges, which we consolidated into 28 opportunities and 27 challenges as part of the coding procedure. In Round 2, we asked the experts to validate these and, based on the experts’ input, we decreased the level of aggregation. This resulted in an increase of the number of opportunities and challenges to 36 and 33, respectively. These served as the input in the narrowing down phase (Round 3), in which the panellists had to vote out the least relevant opportunities and challenges resulting from Round 2. We applied a simple majority rule, according to which we eliminated items which were voted out by more than 50% of the panel. Thus, six opportunities and one challenge were eliminated, resulting in a final list of 30 opportunities and 32 challenges. In the rating phase (Rounds 4-6), the experts rated the shortlisted items regarding their comparative relevance. In this phase, we split the panel into an academic and practitioner subpanel (as intended) to get insights into possible differences between the two groups regarding the rating of items. Note that experts were not requested to justify their ratings given the exploratory nature of the study and the efforts which were already asked from the experts throughout the study.

To investigate differences between the ratings of the subpanels, we firstly used the median of the rating distributions, secondly the mode, and lastly the Fisher’s exact test (Mehta and Patel 1983). The latter was included to enable us to also test quantitatively, for each opportunity or challenge, the null hypothesis that there is no association between the rating distributions and the subpanels. Hence, the two rating distributions can be seen as part of a p \(\times\) q contingency table where p represents the number of subpanels (p = 2) and q the number of rating categories (q = 4). Several statistical tests exist to test null hypotheses based on a contingency table, specifically, the chi-squared test or the Fisher’s exact test (Bland 2015). Since our sample size is relatively small (n = 32), the more complex but exact Fisher’s exact test is appropriate because it is also valid for small sample sizes (Bland 2015). In addition to the qualitative comparison of the rating distributions for each item, the statistical test allows us to quantitatively assess whether the rating distributions for respective opportunities or challenges are significantly different. Finally, both the qualitative and quantitative comparisons of the rating distributions help us discuss the study outcomes. We elaborate on the results of the tests and the study as a whole in Sect. 4.

Between 32 (Round 6) and 40 experts (Round 1) participated in each round, a figure complying with the recommendations in literature (Paré et al. 2013). This corresponds to an end-to-end dropout ratio of 20%, which is in normal bounds considering the prolonged commitment required when participating in a Delphi study comprising six rounds. The dropout ratio also reflects our strict policy of not inviting experts who had not participated in the previous round. The rationale behind this policy is that experts had to be aware of the communicated decisions, information, and outcomes regarding the previous round. The only exception to this rule was Round 3 (narrowing down), in which we invited all experts who had participated in Round 1, but not in Round 2. This exception is justifiable as Round 2 served as a validation round and experts who had not participated were, nevertheless, aware of the goals of the study and had contributed to the collection of opportunities and challenges in Round 1. Despite the expert dropout, the panel remained balanced throughout the study, i.e. there was no systematic drain of participants from a particular group based on country of activity, or their academic or practitioner background.

The overall satisfaction and coding satisfaction were very high from the beginning of the study and increased steadily, while the respective standard deviation decreased with two minor exceptions: the overall study satisfaction in Round 5 and the coding satisfaction in Round 4. We attribute the former to individual experts being dissatisfied with the general rating of challenges and opportunities in Round 4. The latter is most probably driven by the fact that some experts deemed certain challenges and opportunities that were eliminated in Round 3 very valuable. Nevertheless, the extremely high satisfaction values, as well as the positive trend of the overall and coding satisfaction, coupled with the positive qualitative feedback, lead us to believe that the results have converged after Round 6. After completing the study, we also checked for selection bias (i.e., experts leaving the study due to dissatisfaction causing satisfaction ratings to inflate artificially). A mean overall study satisfaction of 6.67 with a standard deviation of 0.52 in the round before dropout suggests that experts did not leave due to dissatisfaction (Appendix C, available online via http://link.springer.com).

4 Results

Table 3 Shortlisted opportunities for the use of PM in organizations
Full size table
Table 4 Shortlisted challenges for the use of PM in organizations
Full size table

Tables 3 and 4 summarize the main results of the Delphi study: the shortlisted opportunities and challenges, including the rating distributions regarding the comparative relevance of the items in the academic and industry subpanels. As previously stated, the opportunities and challenges are structured along the BPM core elements (de Bruin and Rosemann 2007), with the opportunities related to the core element methods/IT being further categorized along the BPM lifecycle phases (Dumas et al. 2018).

Each row in the tables includes the ID, the denomination, and a single-sentence description of the respective opportunity or challenge. Also, the final rating distributions of both subpanels, academics (blue vertical lining) and practitioners (red horizontal lining), are shown using bar charts, as well as percentage values in the third column. In Appendix D (available online via http://link.springer.com), Tables 9 and 10 also contain the median and mode rating values for both subpanels, as well as the p-value of the Fisher’s exact test statistics, indicating the significance of inhomogeneity among the subpanels for the rating of the respective opportunity or challenge. Asterisks are used in Tables 3 and 4 to label the denomination of an item in case of significant inhomogeneity of the subpanel rating distributions based on the Fisher’s exact test (\(0.01\le p<0.05\): significant*; \(0.001\le p<0.01\): very significant**; \(p<0.001\): highly significant***). When discussing commonalities and differences regarding the subpanel rating distributions, we use the median as the primary criterion since it is more robust for outliers than the mode (von der Gracht 2012). In case of different medians, we use the mode and Fisher’s exact test statistics to check for significant difference.

To illustrate the interpretation of the results tables, the first row of Table 3 contains the final results regarding the opportunity enabling inter-organizational value creation (O.1), which is described as “PM enables value creation by fostering inter-organizational interaction and collaboration”. 6.25% of the academics and 56.25% of the practitioners rated this opportunity as extremely relevant (62.50% of the academics and 12.50% of the practitioners as moderately relevant, 31.25% of the academics and 25.00% of the practitioners as slightly relevant, 0% of the academics and 6.25% of the practitioners as irrelevant). The median and mode of these distributions are moderately relevant for academics and extremely relevant for practitioners. The p-value resulting from the Fisher’s exact test statistics is 0.0026 indicating that the inhomogeneity of the subpanel rating distributions is very significant. Hence, the denomination is labeled with two asterisks (**).

As shown in Tables 3 and 4, the Delphi study resulted in 30 opportunities and 32 challenges considered relevant for the use of PM in organizations. The shortlists contain at least two items per category, a finding that emphasizes the diversity of our results. The fact that 13 of 30 (43%) opportunities and 23 of 32 (72%) challenges are not related to the core element methods/IT is congruent with our initial goal of also identifying opportunities and challenges beyond the traditional technical focus of PM.

Some of the shortlisted items are also applicable beyond the PM context, e.g., lack of management support as a challenge (C.2). Given the holistic nature of the study, we did not enforce that items should be exclusive for PM. Incorporating more generally applicable items also helps to identify which areas of the management literature can contribute to the successful use of PM. Even though exclusivity was not required, it was stressed that each item should have a specific relevance or interpretation in the PM context. Consequently, all items in Tables 3 and 4 are considered to be especially relevant for PM according to the panel. This is illustrated by the descriptions of items provided by experts in Round 1. For instance: lack of management support (C.2) has specific relevance for PM as it typically involves the commitment of a wide range of departments as processes cross organizational silos. Similarly, experts give a PM-specific interpretation for poor data quality (C.7) by indicating that events are not always labeled with a meaningful human-readable activity label, and that typically a single event is only recorded per activity, which limits the analysis potential.

In the overview of PM opportunities, a seemingly particular observation is that only one opportunity explicitly refers to the customer, i.e., supporting a culture of customer centricity (O.30). However, it should be noted that several items highlighted by experts in the first round of the study, which have been aggregated during subsequent coding, explicitly refer to customer-related aspects. For instance: analyzing business process variants and exceptions (O.11) covers expert input which is linked to understanding process variation to better serve different customer types, as well as to diagnose exceptional cases to identify customers requiring a specific treatment. Similarly, the ability of PM to enhance business process improvement and redesign (O.18) aggregates customer-related items regarding the improvement of customer experience and satisfaction.

When focusing on the comparative relevance of the opportunities and challenges, Table 5 summarizes the median rating distributions of the subpanels. When comparing the academics’ and the practitioners’ median distributions for all opportunities, the distributions indicate overall agreement between both subpanels regarding the comparative relevance of the opportunities. This claim is also supported by the fact that for 13 of 30 opportunities the median as well as mode values are equal and the opportunity-specific p-value is not significant. In terms of strong alignment between both subpanels, ten key opportunities are rated as extremely relevant by academics and practitioners according to the median and mode (Table 6). In contrast, when considering challenges, the subpanels exhibit significant differences reflected in their rating distributions. This claim is confirmed by the observation that for only 9 of 32 challenges the median and mode values are equal and the p-value does not point to a significant difference. This means that academics and practitioners assess the relevance of many challenges differently. Only three key challenges are rated as extremely relevant by both subpanels according to the median and mode (Table 6).

Table 5 Distribution of median values along opportunities and challenges
Full size table
Table 6 Opportunities and challenges rated as extremely relevant by both subpanels
Full size table

While this section provides a descriptive overview of Tables 3 and 4, we take a broader interpretative perspective in Sect. 5 to highlight some valuable insights and implications of the results for both academia and industry.

5 Discussion

This section discusses the obtained results. Besides the contributions and a general discussion (Sect. 5.1), key implications for academics and practitioners are presented (Sect. 5.2), as well as the limitations of the study (Sect. 5.3).

5.1 Contributions and General Discussion

As highlighted in the introduction and the outline of related work, PM has matured as a research field over the past decade primarily from a technical standpoint, providing various algorithms to understand and improve business processes using event logs. In contrast, there is limited systematic understanding of PM from an organizational perspective. In this respect, a holistic understanding of the opportunities and challenges regarding the use of PM in organizations is absent, but strongly needed. Against this background, we provide such an overview via a Delphi study with recognized academic and industry experts from various countries and backgrounds. Thereby, our study extends the state-of-the-art by means of its holistic nature, the extensive input collected by applying the Delphi method, and the presence of insights into the comparative relevance of challenges and opportunities. In this way, it surpasses the scope of existing related research efforts.

The primary contribution of the Delphi study is a list of 30 opportunities and 32 challenges considered relevant by academics and practitioners regarding the use of PM in organizations. Both opportunities and challenges are very diverse, covering each of the BPM core elements and addressing technical, managerial, and cultural aspects. For the opportunities, an additional distinction is made between the phases of the BPM lifecycle and items are present in each stage. Despite the historically strong focus of PM research on the technical side, this study shows that a significant number of opportunities and challenges have a non-technical character. Only 17 of 30 opportunities (57%) and 9 of 32 challenges (28%) are related to the methods/IT core element. The fact that relatively more opportunities than challenges are related to the core element methods/IT complies with the observation that PM is still mainly perceived as a technical practice, but also indicates that PM techniques are maturing. The fact that many challenges are situated in core elements other than methods/IT opens perspectives for diverse strands of non-technical PM research.

When connecting the listed opportunities and challenges to the related work (Sect. 2.2), our study confirms and significantly extends the state-of-the-art. While it is not trivial to map items from different studies due to their variety regarding the level of granularity, such a mapping is included in Appendix E (available online via http://link.springer.com). From this mapping, it can be concluded that all elements that come forward in the recent studies by Syed et al. (2020) and Grisold et al. (2020) can be related to opportunities/challenges in our study. For instance: Syed et al. (2020) mention unclarity regarding roles and responsibilities in PM initiatives as a challenge, which reflects an unclear organizational anchoring (C.10). Similarly, Grisold et al. (2020) highlight the difficulty to quantify the business value of PM, which relates to elusive business value (C.1). Conversely, a multitude of opportunities and challenges in our study were not explicitly mentioned in the state-of-the-art. Regarding opportunities, these include the items in the category governance (O.4–O.5), as well as specific items such as facilitating strategic decision making (O.2), and enabling business process comparison and benchmarking (O.14). Challenges that our study adds include unclear success factors (C.3), a difficult analysis of process exceptions (C.13), and difficult handling of unstructured data (C.14).

When focusing on challenges, as expected, several challenges mentioned in the recent studies by Grisold et al. (2020) and Syed et al. (2020) reoccur in our study. However, a noteworthy observation is that the most frequently raised limitations by Claes and Poels (2013), a study dating back to early 2012, are still relevant today. This relates to limited data access (cf. C.4), high costs of data preparation (cf. C.12), issues with data quality (cf. C.7), lack of intuitiveness and guidance (cf. C.6), and difficulties to understand PM output (cf. C.16). These challenges all relate to either the input side of PM, the understandability of PM outcomes, or the available guidance to implement PM in practice. As these challenges seem to be persistent for the use of PM in real-world settings, the PM community should investigate avenues to support end-users to alleviate them.

The secondary contribution of the study are insights into the comparative relevance of opportunities and challenges, whereby the views of academics and practitioners are purposefully treated separately. This perspective has not yet been considered in prior literature. As presented in Sect. 4, we observe stronger consensus between academics and practitioners regarding the comparative relevance of opportunities than regarding challenges. In terms of opportunities, the rating phase shows that some items are considered particularly relevant by both academics and practitioners. These tend to be located in methods/IT, suggesting that there seems to be agreement between both subpanels regarding the potential of PM from a technical perspective. However, practitioners attribute greater relevance to the opportunities related to strategic alignment than academics, reflecting practitioners’ experience in setting up and advancing PM initiatives.

In terms of challenges, academics and practitioners agree on the high relevance of data quality (C.7) and preparation (C.12) as well as management support (C.2). The rest of the challenges exhibit varying levels of difference between the two subpanels in terms of their relevance. Overall, there is a tendency for academics to rate challenges as more relevant than practitioners (cf. Table 5). While this could have been expected for methods/IT, given the profound knowledge of potential technical improvements among academics, it also holds for organizational challenges such as the ones related to people and culture. Differences are most obvious for insufficient technical skills (C.22), insufficient data orientation (C.24), and insufficient process orientation (C.25). This demonstrates the view of academics that techniques have come a long way, but people- and culture-related aspects present very prominent challenges. Academics feel more strongly that a PM mindset still needs to be instilled within organizations, accompanied by a broader dispersion of thorough technical training. One could argue that practitioners might have developed coping mechanisms to deal with such challenges. Academics might not be aware of some of these mechanisms when they are involved in specific projects and do not always see the full organizational picture. While all academics have active experience with the real-life use of PM (cf. Sects. 3.2, 3.3), we cannot guarantee that they were exposed to the full organizational complexity of their industry partners. This might have influenced their ratings. However, it is also likely that a PM mindset and expertise have already been, at least partially, established within the organizations represented by the practitioners in the panel as they are active PM users. Academics might be acquainted with a variety of organizations, some of which are less mature at this level.

The observed differences outlined above indicate a perceived misalignment between academics and practitioners. To push the boundaries of what PM is capable of in organizational contexts, a strong partnership between academia and industry is needed and a disconnect should be avoided. Such a disconnect would impede organizations to reap the full benefits of academic work due to prevailing challenges which academics are unaware of, or which are considered less relevant given the predominantly technical research interests of many PM researchers. Consequently, engaging in bidirectional knowledge transfer through roundtables or communities of practice, for instance, could ensure that academia solves scientifically challenging problems with real-world relevance, enabling the industry to benefit from the latest scientific insights.

5.2 Implications

5.2.1 Implications for Academics

The results of the study can serve as the basis for a PM research agenda. The comparative relevance of opportunities and challenges could further contribute to deriving research topics with high priority.

In terms of thematic focus, the strong presence of non-technical opportunities and challenges shows that there is ample potential for PM research from an organizational perspective. This includes specific research areas such as identifying the organizational characteristics supporting PM (i.e., its generic success factors—C.3), investigating how PM can be structurally embedded in organizations (C.1, C.10), or determining which team composition and skills are required to successfully leverage the potential benefits of PM (C.5, C.21, C.22, C.24, C.25). Within the PM field, related topics are largely uncharted research territory, providing a fertile base for future research. When tackling one of the non-technical challenges that are not exclusive to PM, for instance resistance to change (C.30), valuable inspiration can be drawn from the rich body of literature on IT adoption (Rad et al. 2018). In this respect, researchers need to carefully assess whether knowledge on IT adoption is directly applicable to PM, or whether the particularities of PM adoption and use require specific approaches.

Simultaneously, our results suggest that technical research should not be neglected as it is a key ingredient to permanently push the boundaries of PM. In particular, specific attention should be attributed to the persistent challenges which were also identified almost a decade ago. This involves research aiming to support data preparation (C.12), to handle data quality issues (C.7), and to improve the understandability of PM outcomes (C.16). This implies that important research challenges are still present on the input side of PM, as well as on the understandability of the outputs it generates. Besides these specific research directions, researchers can reflect upon the state-of-the-art in PM research against the opportunities/challenges to derive new algorithmic needs.

The diversity of the results and the perceived misalignment between academics and practitioners also indicates that PM research should be on the lookout for emerging topics or shifts in priority in existing topics. In this respect, we expect the identified opportunities and challenges to remain relatively stable in the coming years, primarily due to the stability of our Delphi results reflected, amongst others, in the high satisfaction ratings. However, the differences between the two subpanels regarding the comparative relevance of opportunities and challenges lead us to believe that priorities in PM may shift dynamically, requiring a suitable reaction in PM research. Moreover, the broad scope of topics covered in the results of our study could serve as an incentive to broaden the horizons of PM research to other fields, as well as to intensify efforts at the interface of BPM and IS research.

5.2.2 Implications for Practitioners

For practitioners, the list of opportunities and challenges can help shape the PM roadmap for their organization. Initially, it can support setting priorities to maximize the benefits, while being aware of challenges that can reduce the effectiveness or efficiency of using PM. Later, the lists can function as checklists during PM initiatives to determine whether all relevant opportunities are leveraged and challenges are tackled.

The results of our study also allow highlighting certain specific recommendations for practitioners. To be successful and draw support from management, practitioners need to enhance the PM business case by aligning PM projects with the organization’s priorities and by making returns explicit (C.1). Moreover, organizations should take actions to alleviate data access barriers (C.4) and enhance data quality (C.7), which appear to be persistent challenges for the use of PM in organizations. To this end, they can, e.g., give explicit consideration to PM in the organization’s data governance strategy and policies. Initiatives to enhance data registration can go hand in hand with increased PM training (C.22). When employees are aware of the benefits of PM and the impact of their working habits on the resulting event data, they are likely to be more receptive for policies aiming at improved data registration. Regarding training, our results suggest that organizations should not only invest in tool-based training, but also promote data and process awareness to set the stage for successful PM projects (C.24, C.25). This can, for instance, be operationalized by organizing inter-departmental workshops to unravel the dependencies between departments involved in the same process, as well as to demonstrate how departments can benefit from PM insights.

5.3 Limitations

The results of this study need to be reflected against some limitations, which are linked to the nature of Delphi studies and the design decisions that were made. Firstly, as with any Delphi study, the results are based on the perception of a limited number of experts recruited primarily from our networks. As the size of our panel fitted the exploratory nature of Delphi studies, but is rather small for statistical purposes, we cannot formally exclude the presence of biases in the panel and, thus, cannot make formal claims regarding the representativeness of the results. Therefore, the differences in comparative relevance for opportunities and challenges should be seen as trend statements whose underlying causality needs to be further substantiated (König et al. 2019). Nevertheless, the structured approach that was followed to compose the panel (with explicit selection criteria), together with the positive feedback and high satisfaction throughout the study, support our confidence in the validity of the results. A potential bias in the current panel’s view could be offset by replicating the study.

Secondly, to safeguard the holistic nature of the study, we did not formally require that opportunities and challenges are exclusive to PM. Nonetheless, they needed to have a specific relevance for PM or a specific interpretation in the context of PM. This was also communicated to the panel. However, we did not systematically require experts to elaborate on this matter. A more detailed investigation on this matter would be useful, especially for the challenges. Having a thorough insight into the specific relevance of a challenge for PM can unearth its root causes, which would pinpoint the aspects that future research should tackle. While this constitutes a valuable area for future research, it should be stressed that the presence of items in Tables 3 and 4 in our study clearly reflects that they are of particular interest for PM according to the panel.

Finally, while the rating results are valuable to gain insights into the comparative relevance of items, we have no insights into the reasons why a particular rating was given. This would have been especially interesting for items where substantial discrepancies between academics and practitioners were observed. Gathering an explanation for each rating would imply a substantial additional effort on the part of the experts, which we considered infeasible given the already significant time investment required to participate in the study. Hence, consistent with the exploratory nature of Delphi studies, further research will be required to gain profound insights into the exact motivations behind the provided ratings, considering the possibility that academics and practitioners in our study may have employed different benchmarks for their ratings.

6 Conclusion and Outlook

PM is an active research domain and the developed techniques have been successfully applied to understand and improve business processes. As a research field, the focus of PM is mainly situated on technical topics such as the development and evaluation of algorithms. However, to support the uptake of these algorithms, as well as of (non-)commercial tools, it is also important to gain insights into the use of PM in organizations. Until now, a holistic understanding of the opportunities and challenges of using PM in organizations is missing. Against this background, this study provides a structured overview of such opportunities and challenges based on a Delphi study with a panel of academic and industry experts. Moreover, it conveys insights regarding the comparative relevance of opportunities and challenges, distinguishing the views of academics and practitioners.

The implications for academics in Sect. 5.1 already highlighted several prominent challenges for future research, including conducting dedicated research on various topics regarding the organizational use of PM. Besides these recommendations, this study opens additional avenues for future research. Firstly, replications of the study with different experts would enable the assessment of the generalizability of the results, or might shed light on sectoral or international differences when experts from a specific sector or specific region are purposefully selected. After several iterations with different panels, a meta-analysis can be conducted to synthesize the findings. Secondly, future work could study the specific relevance or interpretation of various opportunities and challenges for PM in more detail, as well as the reasoning behind their associated ratings (e.g., by conducting in-depth interviews with experts or further case studies). These motivations could be used to gain more profound insights into each of the items, as well as to explain the observed differences in the views of academics and practitioners. Thirdly, more research is needed to understand the relationship between different opportunities and challenges. The current study does not consider such interconnections due to its exploratory nature. Studying the relationship between items can show that benefiting from an opportunity might make particular challenges more prominent, requiring integrated action. Fourthly, the results of our study could function as a stepping stone to develop PM maturity models and PM project portfolio management methods. The extent to which an organization can seize the opportunities and the challenges reflects its maturity. Once maturity levels have been established, guidelines can be developed for organizations to become more mature PM users. In the same vein, the results of this study could guide the development of PM project management and project portfolio management methods, insofar as the identified challenges are likely to recur across projects, while the opportunities can inform the selection, prioritization, and scoping of PM projects. Finally, a regular update of the overview of opportunities and challenges is valuable to investigate whether the included items or their comparative relevance will change as PM research and practice are still rapidly advancing.