Introduction

There were an estimated 2.4 million patients evaluated with urothelial carcinoma of the bladder (UC) worldwide in 2013; approximately 79,000 new UC cases are diagnosed in the United States annually [1, 2]. Around 70% of patients with UC in industrialized countries present with non-muscle-invasive disease [3]. UC has a high level of recurrence: within 3 years, 25–75% of patients presenting with low-grade Ta/T1 primary tumor experience disease recurrence and 10–15% eventually progress to muscle-invasive disease [4,5,6]. Given that the majority of recurrences develop within 3 years of primary treatment [4,5,6], the American Urological Association (AUA), National Comprehensive Cancer Network and European Association of Urology guidelines recommend rigorous surveillance programs for disease recurrence involving a cystoscopy every 3 months for 2 years, every 6 months during years 3 and 4, and annually thereafter, depending on the patient risk profile [6,7,8,9].

As a consequence, UC is the most expensive cancer to diagnose, treat, and monitor per patient lifetime, with an estimated US$4.25 billion spent on managing patients with UC in the US in 2010. This figure is expected to rise to US$5.25 billion by 2020 [10]. Notably, surveillance and the treatment of recurrences account for approximately 60% of these costs [10].

Clinical utility of a molecular diagnostic test refers to a demonstration of its ability to provide actionable information for a clinician to use in patient management, including the ability to positively affect clinical decisions and improve outcomes for patients and the healthcare system. Clinically useful tests should improve relevant clinical outcomes such as facilitating targeted therapy and follow-up, as well as improving survival and quality of life [11,12,13].

An increasing body of evidence indicates that new non-invasive urine-based tests have the potential to improve the diagnosis, treatment and subsequent monitoring of patients with UC, thereby improving clinical outcomes and reducing management costs [10, 14]. For example, the US Food and Drug Administration has approved the use of the non-invasive urine tests for nuclear matrix protein 22 (NMP22) and UroVysion® fluorescence in situ hybridization (FISH) for monitoring recurrent UC. However, these biomarker tests have insufficient sensitivity and negative predictive value (NPV) to be useful for demonstrating clinical utility in the detection of recurrent UC [7, 15,16,17]. A urine-based test with higher sensitivity and NPV for detecting recurrent UC, especially early-stage and low-grade tumors, could have the potential to significantly improve patient outcomes [18, 19] by reducing the frequency of invasive testing, such as quarterly cystoscopies, in patients with a low-probability of recurrent UC. Avoidance of unnecessary testing could also lower the cost of care and avoid test or procedure-related complications such as infection or cystitis that have been well described in the literature [19].

Cxbladder Monitor is a new urine-based test for detecting recurrent UC [20]. The test measures the expression of five urine mRNA biomarkers (IGF, HOXA, MDK, CDC and IL8R gene expression) and two clinical variables (whether the previous tumor was primary or recurrent and the time since the previous tumor was resected). Cxbladder Monitor is a “rule out” test that segregates patients into a negative category with a low-probability of recurrent UC (hereafter referred to as “low-probability”) and a category described as “physician-directed protocol” (hereafter referred to as an “elevated-probability”) in the test report [20].

In clinical trials, Cxbladder Monitor has been shown to have a sensitivity of 0.95 with a negative predictive value (NPV) of 0.97 for detecting recurrent UC with a high risk of progression (all high grade and low grade disease, stage T1 or greater) and a sensitivity of 0.86 for low grade Ta. When Cxbladder Monitor was compared directly to UroVysion® FISH, using urine samples from the same patients collected at the same clinical visit, UroVysion® FISH had a sensitivity of 0.33 and an NPV of 0.92 and cytology and UroVysion® FISH performed sequentially yielded a sensitivity and NPV of 0.38 and 0.93, respectively, while in comparison, Cxbladder Monitor had a sensitivity of 0.91 and NPV of 0.96 [15].

The aim of this study was to investigate the clinical utility of Cxbladder Monitor by asking physicians to make decisions about the frequency and scheduling of procedures and tests ordered for patients undergoing surveillance for recurrent UC, before and after disclosure of Cxbladder Monitor results. Real-world clinical data from patients undergoing surveillance were selected for this study to assess changes in the pattern of investigative clinical procedures ordered for the monitoring of recurrent UC.

Methods

Case Selection

All patient case data sets were real-world patients who were undergoing routine guideline-advocated surveillance for recurrent non-muscle-invasive UC by US physicians. Cases were selected from a database of 1036 US patients enrolled in previous prospective clinical studies of Cxbladder Monitor who were undergoing surveillance for recurrent UC (Clinicaltrials.gov identifier: NCT02700659). The 30 cases selected were representative of the database in terms of disease grade, gender and with respect to the existence of recurrent UC as determined by cystoscopy or pyelography.

Each patient case was assigned a random case number, and all patient identifiable information was redacted as a condition of inclusion in this study.

All procedures performed in this study were in accordance with the ethical standards of the institutional and national research committees and with the 1964 Helsinki declaration and its later amendments. All cases in the present study were patients who had given informed consent to the anonymous use of their urine sample and clinical information for the evaluation of clinical utility.

Physicians

All 18 participating physicians (see Supplementary Material, Table S1) had experience in use of Cxbladder Monitor and included physicians from community practices, academic institutions and large urology practices. Selection of physicians with previous experience using Cxbladder Monitor was intended to minimize the potential for bias due to unfamiliarity with the test. Each physician was provided with a description of the study framework (Supplementary Material, Clinical Utility Assessment of Cxbladder Monitor) and a completed example question to illustrate the study process. Each physician individually evaluated real world patient data sets, with each patient dataset presented to each participating physician in the same sequence, under the supervision of a study coordinator. A standardised questionnaire format was used to collect all clinical data and assessments to enable each physician’s evaluation of each patients case and selected procedures and tests that they would use to evaluate the patient to be collected consistently.

Mode of Assessment

There were 828 diagnostic decisions (physician–patient decisions, hereafter referred to as “interactions”) made by 18 physicians on 46 real-world data sets from 30 patients undergoing surveillance for recurrence of UC. The 828 interactions comprised the clinical utility assessment dataset for this study. All 30 patients had one clinical visit and a corresponding Cxbladder Monitor test, giving rise to 540 interactions. A subset, 16 of the 30 patients, had a second clinical visit 3–6 months later with a corresponding Cxbladder Monitor test providing an additional 288 interactions, for a total of 828 physician–patient interactions. The additional 288 interactions provide a longitudinal perspective of the impact of Cxbladder Monitor results over two consecutive clinical visits.

Each physician–patient interaction data set included all clinical information and decisions made before and after disclosure of the Cxbladder Monitor data, using a previously validated experimental design [21, 22, and Supplementary Material, Clinical Utility Assessment of Cxbladder Monitor]. The clinical information contained data on the patient’s gender, age, ethnicity and risk factors including smoking history, any pre-existing conditions and a timeline summary of all available clinical history data (Supplementary Material, Timelines).

The evaluation of the clinical information comprised four steps. Firstly, following review of the clinical information from each patient, the participating physicians were asked to make an initial recommendation of whether the patient required an investigation (workup) for UC (hereafter referred to as “workup recommendation”). Secondly, following a workup recommendation to investigate for UC, the baseline data set was defined as the tests and procedures ordered for the workup, as well as the timing of those tests and procedures ordered by each physician before disclosure of the Cxbladder Monitor test results.

Thirdly, upon completion of the baseline data set for each patient, each participating physician was provided with the patient’s Cxbladder Monitor result (Supplementary Material, Example Report and Interpretation), an updated clinical history and timeline summary, and then asked to make a second decision on a workup recommendation.

Fourthly, if a workup recommendation was made to investigate the patient for UC, data on tests and procedures ordered was similarly collected on the standard assessment form. Cxbladder Monitor results for each patient were presented to each physician in the same test report format as the commercially available test [20] and Supplementary Material, Example Report and Interpretation.

This evaluation and data collection process was repeated for each of the 30 patients and again, separately and consecutively, for the 16 patient case subset where a second clinical visit had been scheduled and undertaken.

Test and Procedure Classification

Tests and procedures were selected by the participating physicians from a provided list of AUA guideline recommended procedures and tests for the surveillance of UC, with options to add alternative tests if required. For the purposes of this study, cystoscopy (flexible and rigid), computed tomography (CT) scans (contrast and non-contrast), retrograde pyelogram and preparation for biopsy were defined as invasive procedures. Cxbladder Monitor, urinalysis, urine cytology, ultrasound, UroVysion® FISH and cytology reflexive to FISH were defined as non-invasive tests.

Statistical Procedures

For each of the 828 physician–patient interactions, arising from this study, analysis included the average number of all procedures, invasive and non-invasive procedures, and each individual procedure ordered, as well as the average length in weeks for the future scheduling of flexible cystoscopy and CT scan (contrast). This data was used to determine the change between the baseline number of procedures ordered and the future schedule of procedures in weeks before and after disclosure of the Cxbladder Monitor results. All changes were analysed using a 95% t test confidence interval and change was considered statistically significant when the confidence interval did not include 0 (zero).

Heatmap Data Graphic

Graphical representation, to provide active visualisation of the results, at the level of the physician–patient interaction, have been presented as heatmaps. The heatmaps depict the total count of procedures ordered at each of the interactions at baseline and the change, relative to baseline, in the number of procedures ordered following the disclosure of Cxbladder Monitor results. The heatmaps have been drawn with columns representing individual physicians and rows represent individual patients. Each row and column intersection represents a physician–patient interaction and are colour-coded based on either an increase or decrease from baseline with green representing a decrease and red an increase. Patients (rows) are grouped by Cxbladder Monitor result, patient ID and test result. The number at each intersection on the baseline heatmap is the total number of tests and procedures ordered, while the number at each intersection of the “change” heatmap is the change in number of tests and procedures ordered (i.e. “0” means the same number of tests were ordered as at baseline and “− 3” means 3 fewer tests were ordered).

Study Endpoints

The co-primary endpoints were to determine: (1) the change in the number of total procedures ordered and (2) the change in the number of invasive procedures (including cystoscopy) ordered by the physician for each patient. Secondary endpoints included the number of tests or procedures added or avoided based on changes in the scheduling of future cystoscopies and CT scans as a result of the inclusion of the Cxbladder Monitor test results.

Primary and secondary endpoints were evaluated for the 540 interactions, to determine change in physician test ordering after the first Cxbladder Monitor test results were disclosed, compared to the pre-disclosure baseline. The same primary and secondary endpoints were evaluated for the additional 288 interactions, representing the subset where two successive Cxbladder Monitor tests were disclosed.

Results

Assessments of patient clinical data by participating physicians were completed within the framework of one or two consecutive office visits (study assessment structure, see Supplementary Material, Clinical Utility Assessment of Cxbladder Monitor). A total of 540 interactions were evaluated for patients who had one Cxbladder Monitor result; of these 73% had a Cxbladder Monitor negative test result and 27% had an elevated-probability test result. None of the patients with a low-probability test result had cancer, whereas all patients who had recurrent UC were in the elevated-probability group. Therefore, across the cohort there was a 27% probability of UC with a corresponding 3.75-fold increase in the probability of UC for those patients with a Cxbladder Monitor result of elevated-probability. An additional 288 interactions were generated from a subset of 16 patients with a second Cxbladder Monitor test, 3–6 months from their first test. The two monitoring visits, and their corresponding Cxbladder Monitor results totalling 828 physician–patient interactions, provided an opportunity to evaluate the change in physician test ordering, including the number of tests and procedures avoided, following disclosure of up to two successive Cxbladder Monitor results.

Changes in Test and Procedure Selection

The baseline data set included a total of 460 (85.2%) workup recommendations to investigate a patient for UC across the total cohort of 540 physician–patient interactions for the first visit (Table 1). Following the disclosure of the Cxbladder Monitor result to the physicians, this decreased to 364, a reduction of 20.9%. Within the group of patients who tested as Cxbladder Monitor low-probability (396 physician–patient interactions) there were 323 workup recommendations to investigate the patients for UC before disclosure of the Cxbladder Monitor results. Following the disclosure of the Cxbladder Monitor low-probability result, this decreased to 220, a reduction of 31.8%. Within the group of patients with Cxbladder Monitor test results of elevated-probability (144 physician–patient interactions), there were 137 workup recommendations to investigate for UC. Following disclosure of Cxbladder Monitor elevated-probability result, this increased to 144 or an increase of 5.1% (Table 1).

Table 1 Changes in initial decision to work up patients undergoing surveillance for recurrent urothelial carcinoma following disclosure of Cxbladder Monitor. Data are the number of physician workup recommendations to investigate patients for UC at visit 1 (540 physician-patient interactions) at baseline and again following disclosure of Cxbladder Monitor
Full size table

The heatmap data graphic (Figs. 1, 2, Supplementary Material Figures S1–S5) provides visual representation summaries for the total number of invasive procedures, total number of non-invasive procedures, total cystoscopies, total computed tomography (CT) scans and the scheduling of flexible cystoscopy, as ordered by participating physicians. Specifically, they provide a graphical representation of the baseline procedure(s) ordered by physicians and decision changes in the number of tests and procedures across each of the physician–patient interactions at the first visit. Figures 1 and 2 summarise the baseline and decision change for the ordering of total procedures and cystoscopy, (flexible and rigid) respectively. Physicians made consistent selections for baseline ordering of total procedures (Fig. 1a) and cystoscopy (flexible and rigid) (Fig. 2a) across all baseline interactions. Following the disclosure of the Cxbladder Monitor results, physicians also made consistent decisions and importantly, the changes in ordering were stratified such that a negative Cxbladder Monitor resulted in fewer tests being ordered and a positive Cxbladder Monitor led to more tests being ordered by participating physicians (Figs. 1b, 2b).

Fig. 1
figure 1

Heat maps representing the total number of diagnostic tests at baseline (a) and change (− 23.9%) relative to baseline after Cxbladder Monitor results (b). Green and red side-line bars represent patients with Cxbladder Monitor defined low-probability (− 38.7%) and elevated-probability (+ 11.5%) results, respectively. The horizontal black line emphasises this delineation across the heat map. Columns represent participant physicians and rows represent patients, for the 540 interactions following at the first clinical visit. Each cell represents a physician–patient interaction. In a, each cell includes the total count with darker shades consistent with higher count and in b, reds represent interactions with added procedures and greens represent interactions with removed procedures

Full size image
Fig. 2
figure 2

Heat maps representing the total number of cystoscopy (flexible and rigid) procedures at baseline (a) and change (− 24.6%) relative to baseline after Cxbladder Monitor results (b). Green and red side-line bars represent patients with Cxbladder Monitor defined low-probability (− 40.4%) and elevated-probability (+ 8.4%) results, respectively. The horizontal black line emphasises this delineation across the heat map. Columns represent participant physicians and rows represent patients, for the 540 interactions following at the first clinical visit. Each cell represents a physician–patient interaction. In a, each cell includes the total count with darker shades consistent with higher count and in b, reds represent interactions with added procedures and greens represent interactions with removed procedures

Full size image

Across all 540 physician–patient interactions at the first visit assessment, 23.9% fewer tests and procedures were ordered overall following disclosure of the Cxbladder Monitor results, without compromising the identification of patients with UC (Fig. 1, Table 2A), because no patients in the low-probability group had cancer. A statistically significant decrease of 301 total procedures were ordered from a baseline of 772 total procedures for patients who tested negative with Cxbladder Monitor at the first visit (Table 2A, Supplementary Material Table S2 for mean per physician–patient interaction).

Table 2 Changes in requested diagnostic tests and procedures in patients undergoing surveillance for recurrent urothelial carcinoma
Full size table

Consistent with the finding from visit 1, across the 288 interactions associated with visit 2, there were 567 total procedures ordered at baseline and 348 following disclosure of the Cxbladder Monitor result. This represents 219 fewer (38.7% reduction) total procedures ordered following disclosure of the Cxbladder Monitor result (Table 2B, Supplementary Material Table S2).

When the number of invasive procedures was analysed across all 540 physician–patient interactions (cystoscopy, computed tomography (CT) scans and retrograde pyelogram), the number of flexible cystoscopy procedures ordered was significantly reduced in all patients by 117 or 31.1% and by 111 or 43.0% in patients who had a Cxbladder Monitor result of low-probability (Fig. 2, Table 2A). Additionally, the total number of invasive procedures ordered on patients who had a Cxbladder Monitor result of low-probability was also significantly reduced by 158 or 37.2% (Table 2A, B, Supplementary Material Table S2). A total of 75% of requests for a cystoscopy, as well as approximately two thirds of CT scans (contrast), were scheduled for a later time after two consecutive Cxbladder Monitor low-probability test results (Table 2A, B; Supplementary Material Table S5).

For patients with a Cxbladder Monitor result of elevated-probability, the total number of procedures ordered increased by 11.5% from baseline (Table 2A). Specifically, orders for contrast CT scans and cystoscopy increased by 68.0% and 33.5%, respectively, in this group, which was offset by a 21.3% decrease in orders for non-invasive tests (Table 2A; Supplementary Material Table S2).

For those patients with Cxbladder Monitor test results of elevated-probability, cystoscopies were scheduled sooner in direct response to the higher risk of recurrence. Similarly for those patients with Cxbladder Monitor test results of low-probability, cystoscopies were avoided or scheduled for a later date in direct response to the change in the risk of recurrence (+ 18.6% and − 19.2%, respectively) compared to baseline (Supplementary Material Table S2).

Discussion

Cxbladder Monitor showed significant clinical utility in this study, where real-world clinical case notes data were evaluated, through the provision of actionable clinical information for physicians monitoring patients for recurrence of UC. Disclosure of Cxbladder Monitor results led to decreases in the number and type of tests and procedures ordered by physicians, without compromising the detection of UC. Evaluating clinical utility in traditional clinical trials often requires large numbers of patients to ensure that the variance in patient acuity, comorbidities, and complexity between physicians is taken into account. The design of this study eliminated the between-patient variance, as the cohort of patients presented to each participating physician was identical thereby facilitating consistent evaluation and decision making across all physicians.

The addition of Cxbladder Monitor to the physicians’ decision making resulted in a decrease in both the decision to investigate for UC and an overall decrease in procedures ordered. There was an overall 20.9% reduction in the physicians’ workup recommendations to investigate a patient for UC and this reduction was greater (31.8%) for those patients with a Cxbladder Monitor result of low-probability. Overall, there was a 23.9% reduction in the number of diagnostic tests and procedures ordered, including a 31.1% reduction in the number of flexible cystoscopies ordered. In particular, for patients with a low-probability Cxbladder Monitor result, physicians reduced the number of cystoscopies and contrast CT scans by 39.4% and 36.6%, respectively.

In this study, Cxbladder Monitor also demonstrated longitudinal clinical utility as physicians further reduced the number of diagnostic tests ordered after consecutive low-probability Cxbladder Monitor test results (67.7% reduction) including a 76.9% reduction in the number of flexible cystoscopies ordered, indicating increased confidence in incorporating Cxbladder Monitor into their decision making. In contrast, physicians ordered more tests and procedures in total, including more invasive tests and procedures, for patients with an elevated-probability Cxbladder Monitor result.

Given the high rate of recurrence observed amongst patients with a history of UC, and the risk of progression to muscle-invasive disease, it is essential that adequate investigation is performed in patients with an elevated-probability of recurrence [4, 6]. In this study, physicians prioritised the ordering of invasive diagnostic tests and procedures for those patients identified as elevated-probability for recurrence by Cxbladder Monitor, reflecting the increased probability of recurrent UC (e.g. through expedited ordering of a cystoscopy). The study cohort included 3.75-fold greater prevalence of recurrent UC for patients with elevated-probability relative to the total cohort of patients justifying the increased use of invasive procedures in this group and ensuring an appropriate and timely diagnosis.

The present study has some limitations, namely that participant physicians made decisions based only on the information that was available in the case notes for each patient undergoing surveillance. While this may not reflect real-world practice, it was the most efficient and pragmatic method of assessing changes in physician decision behavior whilst providing full flexibility for the physician to order any new test or procedure. Also, it is possible that physicians who are less experienced with Cxbladder Monitor may not make use of the test results in the same way as the physicians in this study. However, in this study, while the 18 physicians ordered a range of additional procedures before review of the Cxbladder Monitor results, there was remarkable uniformity in the changes in the tests and procedures ordered to fewer or more invasive procedures based on the results of Cxbladder Monitor. Physicians were asked to ignore patient cost considerations for the purpose of this study. Real-world decisions may be different where patients have co-payment or co-insurance amounts. Lastly, physicians were asked to consider the situation where Cxbladder Monitor could be used according to AUA guidelines and in so doing, reduce any possible bias that could be attributed to Cxbladder Monitor not being included in the guidelines at the time of the study.

Conclusions

In this study, Cxbladder Monitor has shown significant potential clinical utility from the addition of clinically actionable information to physicians monitoring patients for recurrence of UC. Our primary endpoint is confirmed as the addition of Cxbladder Monitor results reduced the total number of workup recommendations and the total number of diagnostic tests and procedures, including flexible cystoscopy, in patients with low-probability for recurrence without compromising the detection of UC. For patients with an elevated-probability for recurrence, physicians ordered a more intensive workup. The addition of Cxbladder Monitor enabled physicians to avoid a number of cystoscopies. The avoidance of cystoscopies therefore reduced the total number of cystoscopies that the patient is likely to be exposed to across the guideline defined 5 year period of evaluation. To the extent that use of Cxbladder Monitor safely reduces the number of procedures, the total cost of monitoring and the number of avoided complications from those unnecessary tests, may also be reduced.