FormalPara Key Points for Decision Makers

The introduction of new intravitreal treatments for retinal and vitreous diseases, including diabetic macular edema, has increased the complexity of clinical and economic management in the field of ophthalmology.

Multicriteria decision analysis is a set of techniques that can help to systematize decision-making in a structured way while increasing the consistency and transparency of decisions.

This work provides a useful framework for decision-making in diabetic macular edema management that considers criteria according to the explicit preferences of the multiple stakeholders involved in this process.

1 Introduction

Decision-making in healthcare is a highly complex process for stakeholders due to the presence of multiple factors and differing objectives that can influence the selection of a treatment or health intervention. To make optimal decisions, there is a need to integrate and prioritize all relevant criteria. In addition to considering clinical efficacy and safety variables, there are other clinical and nonclinical factors that can influence treatment selection, such as costs, efficiency, budgetary availability, patient-reported outcomes, and adherence to and persistence with treatment. Bypassing a formal decision-making process can result in variability, inconsistency, and a lack of transparency, which could lead to suboptimal decisions [1]. This process becomes even more complicated when the opinions of stakeholders from different fields are taken into account, such as the clinical, psychological, clinical management, health administration, and patient perspectives.

Multicriteria decision analysis (MCDA) offers the potential to overcome the challenges associated with traditional decision-making tools, especially for complex decisions that must consider multiple criteria, simultaneously evaluate quantitative and qualitative information, and involve multiple stakeholders [2]. MCDA comprises a broad set of techniques that have been applied in various areas within and outside of the health sector and facilitate a rigorous approach to decision-making in complex environments. MCDA helps to increase the consistency and transparency of these decisions. The techniques used in MCDA divide the decision-making process into different stages in a structured way and explicitly incorporate the preferences of the participants in the process (stakeholders and influencers), thus providing clarity on the relevant criteria and their relative importance [1, 2].

Diabetic macular edema (DME) is an ocular complication of diabetes mellitus. This disorder involves the accumulation of fluid in the central portion of the retina as a consequence of the failure of the blood–retinal barrier [3]. This affects the sensitivity of the macular photoreceptor cells, leading to changes in vision and vision loss [4]. DME is the leading cause of visual impairment and blindness in people with diabetes [5], and its incidence in Spain has increased in recent years [6], similar to the incidence of diabetes. Currently, pharmacological management of DME consists of administering corticosteroids or antiangiogenic drugs by intravitreal injection [3].

The arrival of new intravitreal treatments for retinal diseases has increased the complexity of clinical and economic management in ophthalmology. Sustainability challenges have appeared as a result of the much higher costs of the new treatments and the organizational impact of intravitreal administration. MCDA could aid decision-making in this environment, as it takes the perspectives of the different stakeholders into consideration.

The objective of this work was to determine the most relevant criteria for decision-making in the treatment of DME patients by considering the perspectives of all the stakeholders involved in this decision: physicians, hospital pharmacists, psychologists, experts in health management, local and national health authorities, and patient associations.

2 Methods

MCDA for the management of patients with DME was carried out according to the recommendations of The Professional Society for Health Economics and Outcomes Research (ISPOR), which defines the techniques that can be used and the phases to be followed throughout a MCDA [1, 7].

This MCDA, which was termed the MULTIDEX-EMD study, was performed in three phases: an initial phase (phase A), where the criteria most relevant to the decision-making process in the treatment of DME were defined and selected; phase B, where the criteria were screened, weighted, and prioritized; and phase C, where the results were discussed and the conclusions of the project were established in a deliberative process. This study was carried out from February to May 2018 (Fig. 1).

Fig. 1
figure 1

Phases of multicriteria decision analysis in diabetic macular edema management

Full size image

Twenty participants were selected to develop the whole project. They were chosen based on their influence on healthcare decision-making at different levels and/or specific involvement with the disease. Four groups of stakeholders who were from different fields and therefore had different perspectives on DME-related decisions were identified: clinical (7 physicians: 6 ophthalmologists specializing in retinal and vitreous disease and 1 endocrinologist), health management and health administration (3 representatives/former health authority representatives at national or regional levels, such as the Ministry of Health and Regional Health departments, and 3 health management experts representing medical management or hospital management positions and the academic field of health management and quality), hospital pharmacy (4 pharmacists), and the patient’s perspective (2 members of patient associations and 1 clinical psychologist specializing in the management of people with diabetes). Invitations were sent out to determine the availability and willingness to participate of the experts initially selected. Because some of those experts were unavailable, additional experts were invited to participate until the required number of participants was reached. A breakdown of the participants involved in each phase of the study is shown in Fig. 1.

2.1 Phase A: Defining the Criteria and Their Levels

This phase aimed to define the criteria of interest for decision-making in the management of a patient with DME and to establish the design of the subsequent analysis from these relevant criteria. A panel of experts was established; this panel then selected and defined the criteria.

During this phase, a questionnaire was developed that included a set of initially proposed criteria and their definitions. This questionnaire was sent to the participants for review, but it was also possible for participants to propose new criteria. Criteria were initially identified by means of a bibliographic review that included clinical trials of the options available, disease burden studies, and health technology assessment reports that were available for this pathology. Criteria that characterized a DME treatment were selected, such as efficacy, safety, costs, organizational impact, and economic impact. When identified criteria were similar, the more frequently used and comprehensive criterion was selected to avoid duplication. Clinical and safety criteria were generally obtained from clinical trials, and focused on the main endpoints assessed in these studies. Each of the criteria had two or more possible levels; these levels characterize the criterion. After completing the questionnaire, the panel of experts had a face-to-face meeting aimed at reaching a consensus regarding the final criteria and their levels. The selected criteria served as the basis for the next phase of the study.

2.2 Phase B: Screening and Weighting the Criteria

The main objective of this phase was to screen the criteria that were truly relevant to decision-making and to rank them by weighting their relative importance according to the preferences of the participating stakeholders. This prioritization process was performed in reference to the case of a 50- to 65-year-old diabetic patient with DME.

To obtain the preferences of the participants, the discrete choice experiment (DCE) method was used, which was implemented according to international good practice recommendations [8, 9]. An electronic questionnaire was administered to 19 stakeholders (Fig. 1). The questionnaire was designed based on the combination of the criteria and levels selected in phase A and consisted of several questions or items. Each item comprised a pair of hypothetical treatments characterized by a unique combination of criteria levels.

For each item, the participants were asked to choose which of the two hypothetical treatments was the best option based on their preferences. The minimum necessary number of items (i.e., pair of hypothetical treatments) to establish the relevance of the criteria with sufficient statistical significance was generated. For this purpose, an orthogonal design was chosen using the “Support.CEs” package [10]. This design was chosen due to a lack of prior information about whether there is a tendency of avoidance or preference towards the criteria. This design implies that each criterion level is compared with the rest of the criteria levels equally or proportionally.

To facilitate the completion of the questionnaire, the items were divided into several blocks, which generated four different questionnaires to be completed by the stakeholders.

After obtaining the answers to the questionnaires, two statistical analyses were carried out by fitting multinomial logit models—one to screen the criteria according to their relevance and the other to estimate the weights. The backward elimination algorithm was used to fit the general model used to screen the criteria. This algorithm was based on a complex general model (with a large number of criteria), and it sequentially eliminated the criteria with the largest p values until all of the criteria included in the model had statistically significant p values (p < 0.05). A second multinomial logit model was then fitted to estimate the weights of the screened criteria.

The relative importance index was used to weight the criteria:

$${\text{WD}}_{i} = \frac{{|\hbox{max} (\beta_{Di} ) - \hbox{min} \left( {\beta_{Di} } \right)|}}{{\mathop \sum \nolimits_{D1}^{Dn} \left| {\hbox{max} \left( {\beta_{Di} } \right) - \hbox{min} \left( {\beta_{Di} } \right)} \right|}} \times 100,$$

where \(\beta_{Di}\) is the estimator domain i and WDi is the percentage of weight domain i.

All statistical analyses were performed using R software (version 3.2.3, R Foundation for Statistical Computing, Vienna, Austria) [11].

After analyzing the answers to the questionnaires, the model that best predicted the decision-making process was established by selecting the most relevant criteria. In this model, each criterion was weighted based on the choice preferences of the participants.

2.3 Phase C: Deliberative Process

The final phase of the MCDA involved presenting the results and discussing and interpreting the screened criteria and their weights according to the preferences of the participants. The conclusions of the analysis were also established during this phase through a face-to-face meeting involving 12 of the stakeholders from the panel of experts (Fig. 1). The comments and conclusions that emerged from this meeting about the study and the respective results are included in Sect. 4 below.

3 Results

3.1 Defining the Criteria and Their Levels

In phase A, after the initial meeting, the panel of experts established and defined a total of 31 possible criteria that could be relevant in the management of patients with DME. These criteria and their levels are detailed in Table 1. Throughout the study, the criteria were divided into five categories to help the participants to manage them in a more structured way, considering the large number of criteria involved. However, the criteria were considered independently, and the importance of each criterion was weighted individually. The categories were efficacy/effectiveness, safety, organizational and economic impact, patient-reported outcomes (patient and caregiver), and other therapeutic features. By combining the levels of the 31 selected criteria, a set of 120 pairs of hypothetical treatments corresponding to the items of the DCE questionnaire were obtained using an orthogonal design.

Table 1 List of selected criteria for decision-making in diabetic macular edema management
Full size table

3.2 Criteria Screening

The results of the DCE indicated that 10 of the 31 criteria were considered relevant to decision-making in the management of a 50- to 65-year-old diabetic with macular edema (Table 2). In the fitted logit model, three of the seven efficacy/effectiveness criteria (those related to visual acuity and duration of effect) were statistically significant according to the analysis. Of the eight safety criteria selected, four were relevant (three ocular adverse events and one systemic adverse event). In relation to the patient-reported outcomes provided by the patient and/or caregiver, two of the five criteria were relevant (health-related quality of life and disability). Finally, only one (annual pharmaceutical cost per patient) of the six criteria relating to the organizational and economic impact established in phase A was statistically significant in the analysis.

Table 2 Results of the adjusted multinomial logit model
Full size table

3.3 Criteria Weighting

The screened criteria that had statistically significant coefficients were fitted using a logit model to estimate the weight of each criterion that was relevant to the decision-making process for the management of DME patients. Figure 2 shows the relative weight of each criterion. The considered criteria relating to safety constituted 47% of the overall weight in decision-making, and 35% when all the efficacy/effectiveness criteria were considered. The criteria concerning patient-reported outcomes accounted for 11% of the total weight, and the criterion associated with organizational and economic impacts comprised 7% of the overall weight.

Fig. 2
figure 2

Relevant criteria for decision-making in diabetic macular edema management (50- to 65-year-old patient). BCVA best corrected visual acuity

Full size image

The most relevant criterion for decision-making in the treatment of patients with DME was the mean change in best corrected visual acuity (BCVA) (5-letter improvement), pertaining to efficacy (weight: 17%). This was followed by adverse event criteria such as retinal detachment (16%) and acute myocardial infarction (13%). Another efficacy-related criterion with a high weight was a ≥ 15 letter improvement in BCVA (12%). The remaining criteria had weights of less than 10%.

4 Discussion

MCDA enables the establishment of a framework that stakeholders can use as a tool to probe the full range of DME treatment existing preferences and to systematize transparent decision-making. In this sense, the present study could serve as the basis for DME treatment selection, ensuring that there is a multistakeholder perspective that covers the criteria that are important to all decision-makers involved, rather than focusing only on the viewpoint of physicians or health authorities/payers. The results of this analysis highlight the overall importance of DME treatment safety to the stakeholders involved in this project; its relevance to decision-making accounted for almost half of the total weight across all criteria.

Although currently available treatments are considered very safe from the physician’s perspective, and there is no concern over their use or the management of possible events, the potential for the patient to suffer a serious adverse event could be a determining factor in the choice of treatment when a decision is made considering a broader perspective involving different stakeholders viewpoints. Another important aspect was found to be the efficacy of the therapeutic alternatives. Indeed, the ability to change the BCVA was the most important criterion, mainly because it is directly related to the visual capacity of the patient.

An important and positive aspect of MCDA is that it integrates existing methodologies that are widely used in other fields. Several recommended techniques with different complexities and theoretical foundations can be used to identify preferences, such as the analytic hierarchical process, DCE, and conjoint analysis [1]. DCE is an instrument with a strong theoretical foundation, so, unlike other techniques, it adds considerable robustness to the process of screening and prioritizing the criteria in a MCDA. In this study, criteria selection was based on the available evidence and the characteristics of the current DME treatments. Identifying specific criteria rather than using sets of generic criteria addresses and defines the health issue and the characteristics of the possible treatments more precisely without excluding any specific relevant criterion.

This analysis is not without limitations. In any analysis, the results are determined by the data available. In this sense, the weighting of the established criteria is subject to uncertainty, as it is influenced by the number of participating experts and, above all, by the proportion of those participants in each stakeholder group. The objective of the study was to provide a global perspective as an aid to decision-making in the treatment of DME, so the results from all stakeholder groups were included but they were aggregated to give the most representative multistakeholder perspective on DME treatment possible. However, it could be interesting to carry out a specific study to determine the criteria that were prioritized by each group of stakeholders. In addition, the prioritization of the criteria by each participant may have been influenced by their interpretation of the criteria included in the analysis and the levels included for each criterion. This interpretation would depend on the stakeholder group to which they belong and their familiarity with or knowledge of each criterion. To account for this, definitions of and information on the criteria were included in the questionnaires for each phase to aid participant understanding. MCDAs can be used to support health decision-making in multiple ways, for example in health planning and research, in the pricing and financing of health technologies, and in treatment selection [1, 2, 12]. According to statistics reported in [12], more than half of all healthcare-related MCDAs have been performed to support decisions regarding health intervention reimbursement and coverage. Another frequent reason for applying MCDA in healthcare is to inform decisions regarding the selection or prioritization of treatments. The main areas in which MCDAs are applied are in the diagnosis and treatment of pathologies, prioritization policies, and the evaluation of health technologies [2]. MCDAs have been used by various public and private health technology assessment organizations and agencies in different countries [13], such as Germany [14], Norway [15], Italy [16], Australia [17], and Thailand [18, 19].

In our country, Spain, only a few MCDAs have been published in the field of healthcare [20,21,22,23,24,25,26,27,28], although the number of healthcare-related publications that incorporate MCDA is increasing in Spain and globally.

To our knowledge, this is the first MCDA to be conducted on the management of DME as well as in the field of ophthalmology in Spain. The literature on the application of MCDA to DME or indeed ophthalmology in general is currently very limited. Recently in Italy, a MCDA was developed for the treatment of DME with the aim of complementing health technology assessment in several regions of the country [29,30,31,32]. This study included two phases: (a) the criteria were prioritized based on the general model from the European Network for Health Technology Assessment (EUnetHTA), and (b) each of the alternatives was scored according to its performance (which was assessed based on the previously prioritized criteria). One of the main differences of that study from the present work is that the former MCDA was developed according to the EVIDEM (Evidence and Value: Impact on Decision-Making) methodology, which involved comparative scoring of the alternatives and criteria prioritization using a direct classification method instead of a decompositional method such as a DCE. However, both studies found that safety criteria were assigned the highest priority, whereas efficacy was considered to be less of a priority than other criteria such as economic impact and general relevance. The results of that study concluded that, according to the perspectives of the stakeholders involved, dexamethasone could be the most advantageous treatment option, as it yielded the highest score based on the multiple criteria selected in the MCDA.

A MCDA that evaluated the most relevant health outcomes of opioid substitution treatment programs in patients with disorders resulting from opioid use was recently developed in Spain [21]. That study followed the same methodology as the present MCDA: the relevant criteria and their weights were established by identifying the preferences of experts through a DCE. The main differences of that study from the present analysis are that the aim of the former study was to evaluate exclusively health outcomes from a clinical perspective instead of including different stakeholders. One of the most interesting aspects of that study was that the analysis was conducted using different patient profiles, which led to different criteria weighting in each case.

Some MCDAs have been used in our country to compare alternative treatments for various pathologies, such as chronic dermatological inflammatory diseases, cancer, rare diseases, and cardiovascular diseases such as nonvalvular atrial fibrillation and pulmonary hypertension [24,25,26, 28]. Other applications of MCDA include the development of an assessment framework for rare diseases at a regional level in Catalonia [22] and the development of a decision-support tool for drug evaluation in a pharmacy and therapeutics committee setting at the hospital level [20]. MCDA was also used to establish an algorithm for predicting price and reimbursement decisions at national and regional levels [27].

Some studies have used the EVIDEM methodology to develop the analysis [20, 22,23,24,25, 28]. In those studies, the methods used to prioritize the criteria were fundamentally compositional, such as direct classification or the analytic hierarchical process [22,23,24,25,26,27,28]. Either a single-stakeholder perspective was used, such as that of patients [20, 21, 26] or evaluation agencies [22, 27], or a multidisciplinary perspective was employed, which included patients, physicians, and other stakeholders [23,24,25, 28].

5 Conclusions

According to the preferences of the multiple stakeholders involved, an appropriate DME treatment should guarantee the safety of the patient while maximizing improvements in their visual acuity for the longest effect duration. In addition, the DME treatment should have an affordable annual cost that contributes to the health system sustainability. Finally, it should have a positive impact on the health-related quality of life of the patient and prevent the onset or progression of disability.