Analysis of Patient Preferences in Lung Cancer – Estimating Acceptable Tradeoffs Between Treatment Benefit and Side Effects

Highlights

• Given the increased interest in the use of patient preference information in healthcare decision-making, evidence gathered from patient preference studies should be high quality and relevant to regulatory and policymakers.
• Tradeoffs that patients are willing to make between treatment benefits and harms should be carefully considered in treatment decision-making.
• Avoiding long-term side effects is important to patients with lung cancer and long-term side effects should be considered in healthcare decision-making processes.
• Heterogeneity in patients’ preferences indicates that differences in preferences for long-term and short-term side effects should be carefully considered in treatment decision-making.

Introduction

Lung cancer is one of the most commonly diagnosed cancers in the United States (US) representing 13% of all cancer diagnoses.¹ It is estimated that more than 430,000 people in the United States are patients with lung cancer or survivors (people previously diagnosed with lung cancer currently with no evidence of disease).¹ Lung cancer is the most common cancer-related cause of death in the US; despite improvements in lung cancer treatments, there is still only a 19% overall five-year survival rate.² Only 16% of lung cancer cases are diagnosed at an early stage, where five-year survival is much higher, at 57%.³

Recent developments in lung cancer treatment algorithms, including treatments targeting genetic alterations,^4–6 immunotherapy^7,8 and combination therapies,^9,10 are improving options for patients with lung cancer. In addition, early detection of lung cancer and the availability of more effective and targeted therapies may significantly improve the prognosis of future patients with lung cancer. With increased survival and length of cancer treatments, discussing long-term side effects and quality of life when making treatment decisions is of growing importance.

Interest is growing in quantifying patients’ preferences for treatment for use in benefit-risk assessments and other treatment decision-making.^11–14 Much of this literature has focused on quantifying tradeoffs patients are willing to accept between treatment characteristics such as benefits and risks (maximum acceptable risk) or cost (willingness to pay).¹¹ To fully explore preferences, the different tradeoffs that different patients are willing to accept also need to be explored. Traditionally, this was done by analyzing the preferences of different patient subgroups separately through stratification analysis. However, the limitations of stratification analyses are increasingly recognized¹⁵ and there has been an increased use of more complex analytical models including mixed (or random parameter) logit models or latent class models to explore preference heterogeneity.^11,16

Preferences surrounding long-term side effects vs survival are largely unexplored in lung cancer treatment decision-making. Prior preference studies have mostly focused on the preferences for survival benefits and short-term risks of treatment such as nausea, rash, or diarrhea.¹⁷ Most studies have not explicitly explored the tradeoffs patients are willing to make between benefits and risks or estimated existing preference heterogeneity.¹⁷ The studies that have explored these concepts have not combined them.¹⁸

This study sought to demonstrate a method to evaluate the importance of different treatment attributes to patients with lung cancer and caregivers. A discrete-choice experiment (DCE) was used to explore the willingness of patients with lung cancer and caregivers to trade progression-free survival (PFS) with lung cancer treatment side effects. We sought to demonstrate how to estimate the time equivalents for long-term physical, emotional, cognitive, and functional side effects, to present heterogeneity in the tradeoffs that participants would make between treatment health effects and length of disease control. The methods and findings from this study may help researchers to develop and conduct preference studies and may help clinicians consider the intricacies of tradeoffs between treatment benefits and harms that patients are willing to make. We hypothesized that patients assign importance to long-term side effects and are willing to consider tradeoffs between survival benefits and long-term side effects, but that preference heterogeneity between patients exists.

Patients and Methods

We conducted a DCE to estimate the treatment preferences of patients with lung cancer (any stage, type and treatment) and caregivers of patients with lung cancer (not grouped into dyads). DCEs are rooted in economic theory¹⁹ and are increasingly used to measure stakeholder preferences in health care.^11,20,21 In each choice task, participants are presented with two or more treatment profiles and asked to select the profile they prefer. DCEs are a systematic method to quantify the relative preferences of participants for various treatment characteristics, or attributes, and the tradeoffs they are willing to make.

DCEs are increasingly used to inform health decisions and the US Food and Drug Administration (FDA) has released guidance on conducting studies to measure patient preference information.¹⁴ We report the study according to FDA’s "study quality indicators" using italics to indicate specific indicators.¹⁴ We summarize these indicators and describe how we applied them, including our recommendations based on lessons learned in Table 1. We discuss our methods in four sections: development of the choice tasks, experimental design, participants, and analysis.

Table 1 Application of Quality Indicators from FDA Patient Preference Information Guidance

Development of the Choice Tasks

The study was conducted in accordance with the "good-research practices" and followed the ISPOR task force guidance documents.^22–24 The study incorporated a "patient-centered" approach (Table 1) by involving patients with lung cancer in the study conceptualization and survey development.²⁵ Survey development included a combination of qualitative methods.^26,27 Initial attributes and levels were identified from semi-structured interviews with members of a lung cancer patient action committee (PAC) as well as from previous patient preference studies in lung cancer.¹⁷ The initial selection of attributes and levels were incorporated into a pilot survey that was administered to 11 PAC members. Participants were given the chance to comment on the pilot survey. Pretesting interviews with four participants were conducted to explore "comprehension" of medical information provided, "communication" of attributes and levels, and whether the participants could adhere to "study conduct" by testing survey instructions.

To ensure "relevance," final attributes were adapted from and described using existing patient-reported outcomes. Attributes were adapted from domains of the FACT-L²⁸ and the quality of life model for cancer survivors.²⁹ Levels and descriptions were adapted from existing criteria.^30,31 Six attributes were selected: PFS, short-term side effect, functional long-term side effect (functioning at work and in social situations), physical long-term side effect (physical symptoms), cognitive long-term side effect (cognitive symptoms), and emotional long-term side effect (emotional symptoms). PFS could be presented at 6, 12, and 18 months. The short-term side effect attribute could be presented at severity levels of none, mild, and moderate. The long-term side effect attributes could be presented at severity levels of mild, moderate and severe. Levels of the short-term side effect and long-term side effect attributes were color coded to visually reflect their severity to increase "comprehension" of the risk levels (Figure 1).³² The descriptions of these severity levels were based, in part, on the Common Terminology Criteria for Adverse Events (CTCAE) to ensure "relevance" to clinical outcomes.³¹

A sample DCE choice task is presented in Figure 1. Participants were presented with treatment outcomes for two hypothetical patients who were receiving treatment for lung cancer. The treatment outcomes were the six listed above for both patients, but the levels of these outcomes differed between the two hypothetical patients. Participants were then asked to select the person they thought was better off. This third-person scenario was used to "minimize cognitive bias" from the participant’s own experience and to ensure that all participants were making choices considering the same disease stage.^33–35

Figure 1 Sample DCE choice task.

From the six attributes at three levels, we could generate 729 separate profiles or more than 500,000 profile pairs. To narrow down the number of choice tasks that participants had to complete, we developed a D-efficient, main-effects experimental design using Ngene software (ChoiceMetrics).²³ This design selected 13 paired comparison choice-tasks for participants to complete. The order of attributes or choice tasks was not randomized due to challenges related to the pen-and-paper nature of the survey so cognitive bias due to ordering side effects was not minimized. The preference section of the survey, including attribute definitions, can be found in the Appendix.

Participants and Survey Administration

The target population of this survey was patients with lung cancer and their caregivers that were constituents of a US-based lung cancer patient advocacy organization. To ensure "representativeness" (Table 2) of the study population, the survey was administered at a multi-day meeting organized by this lung cancer patient advocacy organization using a Research as an Event model.³⁶ Meeting attendees were eligible to complete the survey if they had been diagnosed with lung cancer and/or were the caregiver for someone with lung cancer (any stage, type, and treatment). Caregivers provided responses on disease history for the patient for whom they provided care. Meeting attendees had to be 18 years or older and able to read the survey in English. Healthcare providers were not eligible to participate, unless they were also the caregiver of a patient with lung cancer in their personal life or had been diagnosed with lung cancer. Meeting attendees were given a pen-and-paper survey in their registration packets. This survey also included a link to an online platform where participants could take the survey online. To ensure appropriate "study conduct," members of the study team were available at the meeting to answer any questions.

Table 2 Sample Characteristics

In addition to the DCE, the survey asked the participants to evaluate the acceptability of the DCE choice tasks using three questions on a 5-point Likert scale ranging from strongly disagree to strongly agree. These acceptability measures were used to evaluate the "logical soundness" of the results. In accordance with previous publications, 75% of participants needed to answer “agree” or “strongly agree” for each acceptability question to be met.³⁷ The survey also asked about demographic information and disease history. Participants were not compensated for completing the survey. The title page of the survey provided participants with study information. On this page, they were informed that continuing with the survey and submitting it to the study staff demonstrated implied consent. The Johns Hopkins School of Public Health institutional review board deemed the semi-structured interviews, pilot survey, and final survey to be exempt from human subjects review (IRB 6404; IRB 6948).

Analysis

Analytic Model

Stated-preference results are often based on a Random Utility model¹⁹ which has been extensively used to analyze individual choice data. In this model, a treatment is divided into different treatment characteristics. The utility ) a participant ( obtains from choosing a treatment ( is dependent on the combined utility the participant obtains from the different characteristics of this treatment and a random error (). In this study, we divide the characteristics of treatment into a vector of short-term side-effects and long-term side effects () and in a progression-free survival (PFS) time benefit . Then:

(1)

represents utility the participant gets from the short-term side effect and long-term side effect attributes, and represents the utility the participant gets from the progression-free survival attribute. is an individual-specific variance-scale parameter; a larger indicates that the participant is less consistent in how they answer the preference choice tasks. and are often referred to as preference estimates, although it can be difficult to separate the preference estimates from variance-scale^38.

The marginal rate of substitution is the rate at which someone can give up some amount of one good in exchange for another good and maintain the same level of utility. Time equivalents (TEs) represent the marginal rate of substitution between the time attribute and another treatment characteristic. They can be interpreted as the number of months that the participant would trade for the change in the other treatment characteristic. TEs are generally estimated by first estimating the utility model described above and then taking the ratio of the preference estimate of the non-time attribute to the preference estimate of the time attribute (this method is similar to how willingness-to-pay (WTP) is calculated).^39,40

(2)

Calculating estimates of measurement error, such as confidence intervals (CIs) for ratios of random variables, can be challenging. Issues with ratios have been mostly examined within a cost-effectiveness and WTP framework,^41,42 but the same principles apply when calculating TEs. Various methods, including the delta, Fieller, Krinsky-Robb, and bootstrap methods, to calculate CIs for ratios obtained from stated-preference methods have been examined.⁴³

The preference estimates needed to estimate TEs can be estimated through a variety of methods, but the mixed logit model, or random parameters logit model, provides a "robust analysis" and is becoming increasingly popular^11,20,21 The mixed logit model captures "preference heterogeneity" across participants by assuming that the independent variables vary across individuals according to a continuous distribution.⁴⁴ The model provides mean coefficients as well as a measure of the distribution around the mean coefficient in the form of a standard deviation (SD). A statistically significant SD might indicate that there is significant preference heterogeneity for that attribute.

However, the mixed logit model adds an additional level of complexity to estimating TEs as it estimates the distribution around the mean preference estimates. The TEs from a mixed logit model are therefore given by the ratio of two randomly distributed parameters; depending on the distributions of the numerator and denominator, the distribution of the TE estimates might be heavily skewed or might be undefined.⁴⁵ Different approaches, such as assuming that the denominator coefficient (time) is a fixed variable, have been used to circumvent this problem, but these approaches are often suboptimal.⁴⁵

An approach to circumvent these problems is to estimate the Time Equivalents directly. In the WTP literature, this approach is generally referred to as estimating in WTP space.^40,45,46 In this approach, the researcher can make an a-priori assumptions about the distributions of the TE estimates rather than about the preference estimates. Let us denote from equation 3 as and as . Then, equation 3 simplifies to:

(3)

Then, we can derive the TE through: and

(5)

, the time coefficient, incorporates any differences in variance-scale across respondents, while , the TE coefficients are variance-scale free. While the utility expressions in Equation (1) and (5) are equivalent, equation (5) allows us to specify distributions for the TEs () directly, circumventing some of the issues discussed above.^40,45,46

Empirical Estimation

We applied "robust analyses: according to "good research practices."²⁴ Data were analyzed in Stata 13 (College Station, TX). Time equivalents were directly estimated using the mixlogitwtp command as described above. This command uses maximum-simulated likelihood to account for "preference heterogeneity" by assuming that the TEs vary across individuals according to a continuous distribution.⁴⁴ In this study, the estimated TEs represent the number of additional months of PFS that participants deemed equivalent to a one-level change in the severity of a side effect (from none to mild, mild to moderate, or moderate to severe). The time equivalents can be interpreted as the additional number of months of PFS that a participant needs to be compensated with to be willing to accept a medication with a one-level higher severity of a side effect. It can also be interpreted as the number of months of PFS that participants would be willing to give up for a treatment with a one-level lower severity of a side effect.

The dependent variable in the model was a binary variable that indicated whether a treatment profile was chosen as preferred or not. Independent variables were the attributes presented in each profile. All independent variables were coded as continuous (for the side effect attributes, 0 = none, 1 = mild, 2 = moderate, 3 = severe). TEs were assumed to follow a log-normal distribution. The log-normal distributions around the mean TEs were used to calculate the probability that a given number of additional months of PFS would be enough to compensate for a one-level increase in the severity of a side effect.

These results were used to create acceptability curves that demonstrate unobserved preference heterogeneity between participants and to explore uncertainty. The acceptability curves demonstrate the probability that a participant would accept a treatment with a one-level increase (from none to mild, mild to moderate, or moderate to severe) in the severity of a side effect if it provided a certain TE of additional months of PFS. Alternatively, it can be interpreted as the proportion of participants that would accept a treatment with a one-level increase in severity for a certain additional TE of months of PFS.

Results

Participants

In total, 200 patients and 100 caregivers registered for the meeting of which 87 patients (response rate: 44%) and 24 caregivers (response rate: 24%) completed the survey, 3 participants did not answer whether they were patients or caregivers. All participants completed the survey with pen and paper, except for one participant who completed the survey online. Participant characteristics are summarized in Table 2.

On average patients (either patient respondents or those that the caregivers cared for) had been diagnosed 4.4 years ago and 72% had been diagnosed with adenocarcinoma. Most patients were in Stage IV (47.3%) or had no evidence of disease (43.8%). More than 60% of patients had received chemotherapy and more than half had received targeted therapy.

Most participants (77%) strongly agreed or agreed that the DCE choice tasks were easy to understand, 54% strongly agreed that they were easy to answer, and 89% of participants strongly agreed they answered the questions consistent with their preferences, meeting the threshold for two out of three acceptability questions.³⁷ Of 114 participants, 102 (89.4%) completed all choice tasks. No significant difference was identified between those who did and did not complete all choice tasks for all observed characteristics.

Time Equivalents

Table 3 presents the time equivalents (TE) estimated using the mixed logit model. On average, participants would trade the most PFS for a one-level reduction (going from severe to moderate, moderate to mild, or mild to none) in the severity of functional long-term effects; on average, they would accept a mean TE of 3.7 fewer month PFS (95% CI (CI): 3.3–4.1) if a treatment offered a one-level reduction in the severity of functional long-term effect (from severe to moderate, moderate to mild, or mild to none). Participants would trade the least PFS for a one-level change in the severity of emotional long-term effects (mean TE: 0.9 month PFS, CI: 0.4–1.4). Participants would trade 2.3 months (CI: 1.9–2.8) for a one-level change in the severity of physical long-term side effects, 2.3 months (CI: 1.8–2.8) for a one-level change in the severity of cognitive long-term side effects (mean TE: 2.3 months, CI: 1.8–2.8), and 1.8 months (CI: 1.4, 2.3) for a one-level change in the severity of short-term side effects.

Table 3 Time Equivalent Estimates

Figure 2 presents an acceptability curve for the time equivalents (TEs) of the various long-term side effects. Functional long-term side effects showed the most preference heterogeneity, or variability in preferences (SD of the TE: 8.8); 90% of participants would accept a treatment with a one-level increase in functional long-term treatments effects for the TE of 8.4 additional months of PFS. Emotional long-term side effects showed the least preference heterogeneity (SD of the TE: 0.8); 90% of participants would accept treatment with a one-level increase in emotional long-term side effects for 1.7 months of PFS. In addition, 90% of participants would accept treatment with a one-level increase in physical long-term side effects for an additional 4.6 month of PFS, and an increase in cognitive long-term side effects for 4.0 months of PFS.

Figure 2 Time equivalent acceptability curve.

Discussion

In this study, participants would accept treatments with fewer months of PFS for reductions in both short-term side effects and long-term side effects. Given the increased interest in the use of patient preference information in a healthcare decision-making setting,^47,48 it is important that evidence gathered from patient preference studies is high quality and relevant.¹⁴ Studies, therefore, need to follow good research practices and present patient preference data in policy-relevant ways. By designing and reporting this study using the FDA study quality indicators, we ensured that important components of the stated-preference study design were considered and transparently reported.

We categorized the 11 study quality indicators into four categories: patient-focused, choice task design, data quality, and analysis. In terms of conducting a patient-focused study, our survey was developed in collaboration with patients and a patient-advocacy organization. In addition, it was important to clearly state the study’s target population and to choose attributes that are both clinically relevant and important to patients. While considering choice task design, it was important to pre-test the survey to ensure comprehension of the attribute descriptions and descriptions of uncertainty, and how to complete the choice tasks. Because the survey was administered via pen-and-paper, the survey could not be customized based on different responses; it is, therefore, important to adapt the survey design to the mode of administration. To ensure data quality, we followed good research practices through the design, conduct, and analysis of the study. In addition, we included the DCE acceptability questions to be able to measure data quality but did not include additional choice tasks to limit the response burden. We also found that, because most preference surveys are self-administered, it is very important to ensure that all instructions are clearly written and easy to understand. Finally, we conducted all analyses according to good research practices which included accounting for uncertainty in the parameters and preference heterogeneity. While there are many different ways to conduct a patient preference study,⁴⁹ it is important that quality indicators and good research practices are carefully considered in the study design and planning phase before the study is actually conducted.

By including various domains of long-term side effects, the study showed that study participants were able to distinguish between and have different preferences for the different types of long-term side effects. The results show that a general short-term side effects attribute was valued less than three out of the four long-term side effects. This might suggest that treatments with significant short-term side effects are more acceptable to patients than treatments with significant long-term side effects and that long-term side effects need to be carefully considered in treatment decisions.

We describe preference heterogeneity in a novel way by presenting acceptability curves with the percentage of participants that are willing to accept a treatment associated with a one-level more severe long-term side effect if it provides additional months of PFS. Given that preference heterogeneity is of increased interest in healthcare decision-making,¹³ effective and concise ways to model and present heterogeneity of tradeoffs are needed. Further avenues for presenting trade-offs and preference heterogeneity for non-stated-preference experts should be explored, especially as these results might become more important in healthcare decision-making processes ranging from regulatory to clinical decisions. Due to the relatively low sample size of this study, we were not able to formally conduct further heterogeneity analyses such as stratifying the sample to explore differences between patients and caregivers or to identify characteristics of clusters of participants with the similar preferences using latent class analysis. Further studies with larger sample sizes should be done to explore heterogeneity for lung cancer treatment preferences through stratification analyses and/or scale adjusted latent class analyses.⁵⁰ 

Mühlbacher et al (2015)¹⁸ showed that participants would give up time without tumor progression (7.01 months) for an improvement in tumor-related symptoms. However, tumor-related symptoms were purposefully excluded from this study as Bridges et al (2012)³³ showed that the severity of tumor-related symptoms is related to preferences for PFS; in that, study participants actually preferred less PFS when tumor-related symptoms were severe. The participants in this study would give up 3.66 months of PFS for a decrease from severe to mild short-term side effects. Participants in Mühlbacher et al’s study would generally give up a similar number of months without tumor progression for a decrease from severe to mild for particular types of short-term side effects (between 2.72 months for a decrease in tiredness/fatigue and 3.90 months for a decrease in nausea and vomiting). However, since this study included a composite short-term side effect and long-term side effect attributes these results are difficult to compare.

Like previous preference studies in lung cancer,¹⁷ this study used the surrogate endpoint of PFS to represent treatment effectiveness. Using overall survival (OS) as an attribute in the DCE may lead to dominating preferences; participants will only focus on OS and will not pay attention to the other attributes included. Therefore, PFS, which is also often used as an endpoint in clinical trials, was chosen as a benefit attribute. How to best explain PFS to patients requires further exploration,⁵¹ but in our survey development process, the definition of PFS was tested with lung cancer patients and was found to be acceptable.

This study has several limitations. First, it made use of a relatively small, convenience sample of people with lung cancer and their caregivers that attended a national meeting for patients with lung cancer. Participants were actively engaged with a patient advocacy organization, and at least healthy enough to travel to the meeting. We did not sample sicker individuals or caregivers of people that had passed away. Furthermore, participants had been diagnosed a relatively long time ago (on average 4.4 years ago). Participants were therefore likely to be healthier and more engaged and have different treatment and disease experiences than the average patient. However, by administering the survey at this national meeting, we took advantage of a dense concentration of a hard-to-reach patient population. Second, the study sample was composed of a clinically heterogeneous group of patients with lung cancer with potentially very different preferences. Participant had been diagnosed with different types of lung cancer, were at different disease stages, and had undergone different treatments. Caregivers were also included in the sample contributing to further heterogeneity. Third, the study departed from the FACT-L²⁸ and the quality of life model for cancer survivors²⁹ by including cognitive side effects and the omitting social and spiritual domains. Cognitive side effects were added based on reports from patients with lung cancer during instrument development about “chemo brain” or clear thinking. The social and spiritual domains were omitted because it would have been difficult to attribute these to cancer treatment. Fourth, treating the short-term side effect and long-term side-effects attributes as continuous resulted in a parsimonious model. However, because of this specification, we did not explore whether patients assigned different time equivalence estimates to a change from a severe to a moderate side effect or a change from a moderate to a mild side effect. Rather, the model assumed that patients assigned the same importance to both changes. Fifth, this survey only met two out of three acceptability measures using the 75% threshold.³⁷ Less than 75% of participants agreed or strongly agreed that the choice tasks were easy to answer, which is also lower than other studies that have used this scale.^37,52

Conclusion

In this study of patients with lung cancer and caregivers’ preferences, we estimated disease progression-free time equivalents patients would trade for experiencing higher severity of long-term treatment side effects. Longest time equivalents were calculated for long-term functional side effects, followed by long-term physical side effects, and shortest for long-term emotional side effects. Calculating time equivalents for reductions in treatment harms might provide a valuable approach to evaluate current and future lung cancer treatments by determining the acceptability of the side effects with regards to survival gains for patients. The findings in this study can serve to inform further studies that examine patients’ preferences and willingness to trade between the survival benefits of cancer treatment and short-term side effects and different types of long-term side effects.