FormalPara Key Points for Decision Makers

There is uncertainty in the cost effectiveness of cetuximab or panitumumab therapy informed by KRAS testing for third-line metastatic colorectal cancer.

Using real-world data for a large cohort of patients with long follow-up, we found that a policy of KRAS testing and cetuximab or panitumumab are unlikely to be cost effective at a threshold of CA$50,000/LYG, unless considerable price reductions can be achieved.

1 Introduction

Colorectal cancer is the second-leading cause of cancer death in Canada, with projected deaths of 9400 in 2022 [1]. Survival among patients with de novo or relapsed metastatic disease has improved over time [2, 3], but remains relatively low [4]. Cetuximab and panitumumab, two anti-epidermal growth factor receptor (anti-EGFR) monoclonal antibodies, were introduced for third-line therapy of metastatic colorectal cancer in the mid-2000s. Initially, phase III clinical trials for cetuximab and panitumumab reported small improvements in progression-free survival (PFS) and had variable effects on overall survival (OS) [5,6,7]. In the absence of strong effectiveness data, early economic evaluation was not favourable [8]. Cetuximab and panitumumab were later found to only benefit patients without mutations in the KRAS gene [9,10,11]. Single-agent panitumumab and cetuximab in combination with irinotecan, informed by single-gene KRAS testing, were adopted in the province of British Columbia (BC), Canada in mid-2009 [12, 13].

Numerous cost-effectiveness analyses of cetuximab and panitumumab have been published [14,15,16,17,18,19,20,21,22,23,24,25] but the reported cost effectiveness varies widely. Analysis conducted alongside the Canadian CO.17 trial of cetuximab reported an incremental cost-effectiveness ratio (ICER) of CA$186,761/QALY [19]. A subsequent health technology assessment from the province of Ontario, Canada, resulted in considerably lower values of CA$42,710/QALY for cetuximab plus irinotecan and CA$47,795/QALY for panitumumab [20]. The evaluation by the National Institute for Health and Care Excellence (NICE) reported much higher values, of £88,000/QALY (approximately CA$140,000/QALY) for cetuximab plus irinotecan and £187,000/QALY (approximately CA$300,000/QALY) for panitumumab [22, 24]. Comparisons across studies are difficult due to differences in the scope of the analysis. Some evaluations, like the CO.17 analysis, only consider the cost effectiveness of the drugs in a KRAS wild-type population, while others include the cost of KRAS testing in the full, potentially eligible metastatic colorectal cancer population [24]. In practice, many patients who are tested and ultimately found to have KRAS mutations would accrue the cost of testing with no corresponding potential benefit. Furthermore, nearly all studies are model-based and rely on efficacy estimates from clinical trials.

KRAS testing and third-line therapy with cetuximab or panitumumab have been in use long enough to directly observe the long-term survival and cost implications of the policy change. Analysis of real-world data can capture the utilization of both KRAS testing and the two drugs in practice, to provide robust evidence of the long-term cost effectiveness of a policy of KRAS testing and third-line therapy with cetuximab or panitumumab. The strength of real-world data is its external validity: the data can capture diverse populations not typically included in clinical trials, receiving care as delivered in practice, not according strict trial protocols [26]. Real-world evidence can also resolve uncertainty in the health economic evidence from clinical trials or decision models by capturing potentially large, unselected patient populations, with longitudinal data collected over long periods of time [27].

The objective of this analysis was to conduct a real-world cost-effectiveness analysis of the policy change introducing KRAS testing followed by treatment with cetuximab plus irinotecan or single-agent panitumumab, for third-line therapy of metastatic colorectal cancer in BC. This study uses observational data to identify the value of cetuximab and panitumumab therapy achieved in practice, to better understand the implications of the policy change.

2 Methods

We used a pre-post study design to evaluate the cost effectiveness of the policy change at BC Cancer introducing KRAS testing and cetuximab or panitumumab treatment, in a historical cohort of colorectal cancer patients identified from administrative data. BC Cancer provides population-based, publicly funded systemic therapy for cancer patients in the province of BC.

Patients were included in the cohort if they had a confirmed diagnosis of colorectal cancer (ICD-O-3 site codes C18-C20) in the BC Cancer Registry, were referred to BC Cancer for treatment, and if they were potentially eligible for third-line therapy in 2006–2015, having previously received systemic therapy with both an irinotecan-based and oxaliplatin-based protocol [28, 29]. The last dispensing date of irinotecan or oxaliplatin, plus 14 days, was used as each patient’s index date and start of observation. Patients with an index date between July 1, 2009 and December 31, 2015 were included in the post-policy intervention group; patients with an index date between January 1, 2006 and June 30, 2009 were included in the pre-policy comparison group. Patients were excluded if they were not registered for public health insurance in BC at index date, if they were missing a personal health number or key demographic information, if they participated in a clinical trial, or if they were part of the pre-policy group and went on to receive cetuximab or panitumumab after the policy change.

Administrative data from BC Cancer and the BC Ministry of Health were linked using unique patient identifiers through Population Data BC. Data included patient demographics [30], cancer diagnosis and treatment [31], KRAS testing [32], hospitalization [33], physician services [34], outpatient prescription drugs [35], and death records [36].

To account for differences in baseline covariates, patients in the post-policy group were matched to patients in the pre-policy period using the supervised machine learning technique of genetic matching. The genetic matching algorithm uses an iterative process to assign weights to a propensity score and covariates used in matching, in order to minimize measures of total distance in the match [37]. Exploratory analysis was conducted to identify covariates that were associated with policy group, associated with outcome, or potential confounders associated with both policy group and outcome. The initial propensity score was constructed in a stepwise fashion by first adding age and sex, potential confounders, other associated variables, and higher-order terms to improve model fit [38]. Patients in the intervention group were matched 2:1 to controls, with replacement. Matching was performed with the MatchIt package in R [39], which calls functions from the Matching package [40]. Balance on baseline covariates was assessed after each match using absolute standardized difference and variance ratios, and covariates were added or removed until imbalance was minimal. The final propensity score included age, age-squared, sex, cancer site, health authority, log-transformed time since diagnosis, and indicators for surgery of the primary cancer, prior liver resection, prior rectal radiotherapy, and prior bevacizumab (including interaction between bevacizumab and age), and Charlson index (categorical). Other covariates that were explored included neighbourhood income quintile and rurality; they were not associated with group assignment or outcome and were not included in the match. Three alternative approaches to generating balanced groups were explored in sensitivity analysis: (1) genetic matching using a reduced number of variables in the propensity score, including only potential confounders associated with both outcome and policy group (age, health authority, prior liver resection, prior rectal radiotherapy, Charlson Index [categorical], and prior bevacizumab), (2) greedy matching, 2:1, on the original propensity score only, and (3) inverse probability of treatment weighting (IPTW) using the original propensity score.

Survival time was measured from index date to death or censoring. Patients were censored if they were no longer registered for public insurance, or still alive at the end of follow-up, December 31, 2019. Patient-level costs were calculated from the health system perspective from health services resource use identified in the administrative data (Table S1 in the electronic supplementary material [ESM]) [41]. Systemic therapy drug costs were available directly from the BC Cancer Pharmacy dispensing records; these costs reflect negotiated prices but do not account for rebates or additional discounts for branded drugs. Outpatient prescription drug costs, including both costs covered through the public drug insurance program, BC PharmaCare, and costs paid privately, were obtained from BC PharmaNet [35]. The unit cost of a KRAS test (CA$250 per test) was obtained from the BC Cancer Genetics and Genomics Laboratory. Costs were expressed in 2020 Canadian dollars using the Consumer Price Index for Health Care for BC, from Statistics Canada [42]. To account for censored cost and survival, both were weighted using inverse-probability weighting, using 7-day time intervals over an 8-year (416-week) time horizon. Future costs and survival time were discounted at a rate of 1.5% per year relative to index date [43].

The primary outcome of interest was the incremental cost-effectiveness ratio (ICER), expressed as dollars per life-year gained (LYG). To characterize uncertainty around the ICER, bootstrap resampling (1000 iterations) was used to generate 95% confidence intervals around the ICER, and a cost-effectiveness acceptability curve (CEAC) at a range of threshold values [44]. Additional scenarios were explored in sensitivity analysis, including discount rates of 0% and 3% [43], cetuximab and panitumumab cost reduction of 50%, KRAS test cost reduction of 50%, a limited cost scope of public costs only (excluding privately paid outpatient drug costs), and a shortened (4-year) time horizon.

To estimate the incremental quality-adjusted life-years (QALY) of the policy change, three health states were defined: (1) treatment with panitumumab; (2) treatment with cetuximab plus irinotecan, or other chemotherapy; and (3) post-treatment. Treatment health states were defined as the time between first and last prescription, plus 30 days; all remaining survival time was counted as post-treatment. The health states and mean quality-of-life weights were based on the NICE Assessment Group values, which had been calculated from the CO.17 trial data [19]. For each patient, in each bootstrap iteration, a utility weight for treatment with cetuximab or chemotherapy was sampled from a beta distribution with a mean of 0.75 (SD 0.075). Weights for treatment with panitumumab and post-treatment time were calculated as increases and decreases relative to this initial sampled value, to give means of 0.87 and 0.69, respectively. Time on treatment and time post-treatment were multiplied by the corresponding utility weights to calculate QALYs and an incremental cost-utility ratio.

3 Results

After applying exclusion criteria to the initial cohort of colorectal cancer patients previously treated with irinotecan and oxaliplatin, there were 1757 patients included the final cohort (Fig. 1), 453 patients in the pre-policy group and 1304 in the post-policy group (Table 1). In the post-policy group, 1113 patients (85.4%) received KRAS testing, 653 patients (57.1% of patients tested) were found to be KRAS wild-type, and of those, 420 patients (72.4%; or 32.2% of the post-policy cohort) ultimately received panitumumab or cetuximab therapy. In both periods, the average age was 63 years, and the majority of patients (59%–60%) were men. Significantly more patients in the post-policy period had been previously treated with bevacizumab, and significantly more patients in the pre-policy period had undergone resection of their primary cancer. After matching, the pre- and post-policy groups were well balanced, with only minor imbalance (standardized difference = 0.10) in neighbourhood income quintile.

Fig. 1
figure 1

Cohort selection and exclusion criteria. Initial cohort consisted of all patients with a confirmed diagnosis of colorectal cancer (ICD-O-3 site codes C18-C20) in the BC Cancer Registry, who were referred to BC Cancer for treatment, and had previously received systemic therapy with both an irinotecan-based and oxaliplatin-based protocol. PHN personal health number, MSP Medical Services Plan

Full size image
Table 1 Cohort characteristics by policy period, before and after matching
Full size table

The balance of baseline covariates achieved with alternative matching or weighting approaches in sensitivity analysis are presented in supplementary Table S2 (see ESM). The alternative models resulted in slightly more imbalance than the original match. In particular, genetic matching with the reduced model introduced more imbalance in colorectal cancer type (std diff. = 0.19), prior colorectal surgery (std diff. = 0.26), and prior radiotherapy (std diff. = 0.14). Greedy matching was unable to achieve balance in prior bevacizumab use (std diff. = 0.34). Of the alternative approaches, IPTW using the original propensity score performed best, with only mild imbalance in income quintile (std diff. = 0.13).

Survival was significantly longer in the post-policy period (p = 0.0029 from log-rank test), with a mean survival of 227 days (95% CI 200–254) in the pre-policy period, and 366 days (95% CI 301–430) in the post-policy period (Fig. 2) for the matched cohort. Results were similar for the unmatched, crude cohort (supplementary Table S3, Fig. S1, see ESM). Censoring was low in both periods, at 3.5% and 5.1% in the pre- and post-policy periods, respectively.

Fig. 2
figure 2

Overall survival by policy period, before and after introduction of KRAS testing and cetuximab or panitumumab treatment. Survival probability estimated using Kaplan-Meier method. Tick marks indicate censored cases. Shading indicates 95% confidence interval (CI). Difference is statistically significant using the log-rank test

Full size image

The average cost for patients in the pre- and post-policy periods were CA$24,414 (95% CI 21,321–27,696) and CA$43,209 (95% CI 39,759–46,944), respectively. The largest component of total cost was the cost of hospitalization (51% of pre-policy and 40% of post-policy costs), while the largest difference was seen in the cost of systemic therapy drugs (6% of pre-policy and 27% of post-policy costs; supplementary Fig. S2, see ESM).

The policy change introducing cetuximab and panitumumab informed by KRAS testing resulted in an incremental improvement in survival of 0.25 life-years (95% CI 0.09–0.39) at a cost of CA$73,759/LYG (95% CI 46,133–186,446; Table 2). At a threshold of CA$50,000/LYG, the probability that the policy change would be considered cost effective was 4.6%. The CEAC reached 95% probability at a threshold of CA$158,000/LYG (Fig. 3).

Table 2 Incremental cost effectiveness of policy change, before and after introduction of KRAS testing and cetuximab or panitumumab treatment
Full size table
Fig. 3
figure 3

Cost-effectiveness acceptability curve of policy change. Curve indicates the probability that the policy change was cost effective, by willingness-to-pay threshold. Dashed reference line indicates median. LYG life-years gained

Full size image

Utility weighting, to calculate QALYs, results in an incremental gain of 0.22 QALY (95% CI 0.10–0.31) and an incremental cost-utility ratio of CA$85,447/QALY (95% CI 58,336–168,862).

The results of the sensitivity analysis scenarios are summarized in Table 3. The ICER is not sensitive to assumptions regarding discounting, or scope of prescription drug cost data. Reducing the cost of cetuximab and panitumumab by 50% reduces incremental cost to CA$13,590 (95% CI 9275–18,770) and brings the ICER to CA$53,331/LYG (95% CI 31,873–130,611). Reducing the cost of KRAS testing has minimal impact. The ICER is sensitive to the choice of time horizon; shortening the time horizon to 4 years increases the ICER to CA$94,639 (95% CI 64,049–184,121) but reduces the uncertainty around the estimate. The results are somewhat sensitive to the matching method used; however, the results for genetic matching with the reduced model and greedy matching with propensity score in particular should be interpreted with caution due to the residual imbalance in baseline covariates and potential confounders described above. The ICER for the IPTW cohort is slightly higher than for the original matched cohort, at CA$81,530 (95% CI 57,587–326,760), due to a decrease in the estimated incremental survival. This result can be interpreted as the average incremental cost effectiveness of the policy change in the full cohort of metastatic colorectal cancer patients, rather than the incremental cost effectiveness of the policy change for the matched pre-policy group.

Table 3 Results of sensitivity analysis
Full size table

4 Discussion

Our real-world evidence found that a policy of KRAS testing with third-line cetuximab or panitumumab therapy was not cost effective at a threshold of CA$50,000/LYG. The change may be considered cost effective at higher threshold values, but there is considerable uncertainty around the cost-effectiveness estimate.

While many economic evaluations of cetuximab and panitumumab have been published, relatively few have evaluated the test–drug combination, and all rely on Markov models to estimate cost and survival outcomes. Previous cost-effectiveness analyses that included the cost of KRAS testing to identify cetuximab- or panitumumab-eligible patients report ICERs ranging from CA$43,000/QALY for cetuximab-irinotecan and CA$48,000/QALY for panitumumab [20] to approximately CA$180,000/QALY for single-agent cetuximab [23]. Our ICER results fall within this range, but the effectiveness estimated from the real-world data is less than half of the survival projections from decision models. In our analysis, we found incremental effectiveness to be 0.25 life-years or 0.22 QALYs. The incremental effectiveness values projected from decision models are 0.49–0.51 QALY [20, 21], using a lifetime time horizon. The real-world estimate may be lower for a number of reasons. First, our estimates use data for the full population of metastatic colorectal cancer cases, while the decision models rely on effectiveness estimates from clinical trial data. The real-world cohort includes patients who may not be considered eligible for clinical trials, including elderly patients, patients with more comorbidities or poor health status, and patients living in rural or remote areas [45]. Second, without a standard third-line therapy for patients ineligible for cetuximab or panitumumab, the study design relied on the end of second-line therapy to mark the start of observation. Many patients identified as potentially eligible for third-line systemic therapy using this definition would not realistically be candidates for therapy, due to deteriorating health status or death. A chart review from six Canadian cancer centres reported that only 43% of patients who received second-line therapy went on to receive third line [46]. The survival estimates presented here will likely be lower than for a study of prospectively identified candidates for third-line therapy, due to deaths shortly following the end of second-line therapy. Third, this study used an 8-year time horizon, rather than the lifetime projection in the decision models. The sensitivity analysis indicates that the ICER is sensitive to the time horizon, due to the differential impact on cost and effectiveness. Costs accrue early in the follow-up period, while survival benefits accrue later. Cost-effectiveness analyses with shorter time horizons have previously reported incremental 2.5-year survival of 0.18 LYG [23] and incremental 4-year survival of 0.29 LYG [47]. Based on the observed convergence of the survival curves, we expect that most benefits associated with the policy change have accrued by 8 years of follow-up, but the incremental survival may be underestimated.

We identified only one cost-effectiveness analysis of KRAS testing for third-line cetuximab or panitumumab that incorporated observational data. Uyl-de Groot et al. conducted a chart review of patients who received cetuximab or best supportive care (BSC) at eight hospitals in the Netherlands in 2009–2012, and used the data to build a Markov model to project long-term outcomes [47]. A challenge encountered by the authors, and a challenge common to the other model-based evaluations, was how to characterize the costs and survival of patients in the comparison arm, because there is no standard third-line therapy for patients who are ineligible for cetuximab and panitumumab. Even with detailed chart data, the authors were required to make assumptions about the progression-free survival in the BSC group to incorporate into their model. Furthermore, the costs of the BSC group were only one-tenth of costs for the cetuximab treatment group, suggesting that health services for the BSC group may not have been adequately captured by the hospital chart data. A strength of using administrative data to conduct this study is that it captures longitudinal health resource use across different services in the health care system. As a demonstration of the feasibility of combing real-world data with clinical trial data, the investigators of the CO.17 trial replicated their cost-effectiveness analysis using administrative data from Ontario, Canada [48]. They report ICERs very similar to the original trial results but conclude that the administrative data provide a far more complete assessment of benefits and cost, particularly for hospitalization and emergency department visits. The mean costs per patient in both treatment arms was roughly CA$12,000 to CA$15,000 higher using the administrative rather than the original trial data. In this analysis, we found that most of the incremental cost was made up of systemic therapy drug costs, but hospital costs, other outpatient prescription drugs, physician services, and other services contributed to the total. The use of real-world data provides a more comprehensive estimate of incremental cost and can directly capture cost and survival outcomes for patients receiving BSC, where there is no standard treatment protocol.

The results of sensitivity analysis scenarios show that the cost effectiveness of the policy of KRAS testing to inform cetuximab and panitumumab is not sensitive to the cost of the KRAS test. The ICER is most sensitive to changes in the costs of cetuximab or panitumumab, despite the fact that only one third of the post-policy cohort received either drug. There is the potential to considerably reduce the cost of anti-EGFR therapy with the introduction of biosimilars. The patents for both cetuximab and panitumumab have expired; there are currently no biosimilars on the market for either drug, but biosimilar cetuximab is reportedly in development [49]. A recent study re-analyzed data from the CO.17 trial to estimate the potential impact of biosimilar cetuximab on cost effectiveness [50]. The authors reported that at a price of CA$275.80 per 100 mg—a 15% reduction from the original study price of CA$324 per 100 mg—the ICER would be CA$261,126/QALY. In order to achieve a value of CA$100,000/QALY, the price would have to be lowered by over 80% [50]. In the current analysis, a 50% reduction in the cost of cetuximab and panitumumab resulted in an ICER near the threshold of CA$50,000/LYG. While a 50% reduction relative to the price of the branded biologic may not be attainable in practice, there is still significant opportunity to improve the value of therapy through the use of biosimilars [51]. In Canada, efforts are underway to coordinate the review and uptake of future biosimilars across provinces, through the Pan-Canadian Oncology Biosimilars Initiative.

4.1 Limitations

This study was subject to several limitations, largely arising from the nature of the real-world data. The first challenge was assigning the eligibility date with administrative data. Administrative data are well suited to identifying services or encounters with the health care system, but the data do not capture all relevant clinical endpoints. Information such as progression of disease must be approximated using service-related definitions, such as the end of a course of chemotherapy [52]. In this analysis, we defined potential eligibility for third-line therapy using the end of second-line therapy, because it was the last reference date available for all patients in the population of interest. This definition of eligibility date has likely introduced some error in the analysis but is unlikely to bias the incremental cost-effectiveness analysis. The definition of eligibility is the same in both time periods, pre- and post-policy, and any error would occur equally in both groups.

Similarly, the lack of standard third-line therapy for patients ineligible for anti-EGFR therapy meant it was not feasible to identify an appropriate comparator for the subset of cetuximab- or panitumumab-treated patients. By using the end of second-line therapy to mark the start of observation, we included patients who went on to receive third-line cetuximab or panitumumab, or other chemotherapy for symptom management, and patients who died before they could initiate a new line of therapy. With the current study design, we are not able to estimate the cost effectiveness of cetuximab or panitumumab therapy without introducing immortal time bias. Patients who received cetuximab or panitumumab had to survive long enough to initiate at least one cycle of therapy by definition, while patients in the pre-policy period, or patients in the post-policy period ineligible for anti-EGFR therapy did not have an equivalent treatment start date available in the data.

This study roughly estimates QALYs using simple assumptions. Economic evaluation guidelines recommend the use of QALY in the reference case of the analysis, but there is little guidance for how to incorporate quality weights into cost-effectiveness analysis using observational data. There are initiatives underway in Canada to routinely collect more real-world quality-of-life data and other patient-reported outcomes, but little data are currently available [53, 54].

Lastly, there is a risk of bias from using a historical comparison group. Other changes in practice may have influenced patients’ survival or treatment costs. Over the study period, the uptake and duration of bevacizumab use for first-line therapy increased, colorectal cancer screening became more widespread, generic irinotecan became available in Canada, and two new regional cancer centers opened in BC. The impact of most of these changes would be seen earlier in the disease trajectory, before patients progressed to third-line therapy, but there could potentially be residual effects on total cost and overall survival. Our genetic matching approach can help to reduce the risk of bias from measured confounders, but there may be unmeasured confounders, including historical changes, that are unaccounted for in the study design.

5 Conclusion

The introduction of third-line cetuximab and panitumumab therapy for metastatic colorectal cancer, informed by KRAS testing, led to an increase in mean overall survival, but this improvement came at considerable cost. Despite the large patient cohort and long-term follow-up in this real-world analysis, there is still considerable uncertainty around the cost-effectiveness estimates. Reducing the cost of cetuximab and panitumumab, through negotiated price reductions or access to biosimilars, could considerably improve the cost effectiveness of anti-EGFR therapy.