Main

Delivering high quality care is key to modern health-care system. External quality assessment programmes are more and more considered as a cornerstone in the assessment and improvement of quality. Internationally, accreditation is the most commonly used method. In the Netherlands, external peer review (visitatie in Dutch, meaning ‘to visit’) is the dominant external quality assessment method and this approach is slowly gaining popularity in Europe (Heaton, 2000; van Weert, 2000). Organisational external peer review for cancer care was introduced in the Netherlands in 1994. In England, National Cancer Peer Review (NCPR) was introduced as a part of the National Cancer Programme in 2004, after a first round of peer review was conducted at a regional level in 2001 (National Cancer Peer Review Programme, 2012). The English programme focusses on performance for specific tumour groups, whereas the Dutch programme targets the multidisciplinary cancer care organisation in hospitals as a whole.

When the external peer review programme was introduced in the Netherlands in 1994, treatment of cancer patients was predominantly monodisciplinary. Since then, multidisciplinary cancer care has become the standard. The programme was introduced by the Comprehensive Cancer Centre North Netherlands. The nine Comprehensive Cancer Centres, covering the whole Netherlands, are regional network organisations of health-care professionals and institutes for cancer and palliative care aiming at improving cancer care through research, guideline development, knowledge exchange and organisational improvement without having a treatment function themselves. The peer review programme was first introduced in the Northern Netherlands and gradually spread over the country. Surveyors are all specially trained medical and nursing specialists, which results in a natural understanding of the daily challenges faced in the treatment of cancer patients; in this way the system is supposed to generate recognition and involvement of professionals. Participation is voluntary and hospitals are advised to participate every 4–5 years. A majority of Dutch hospitals has gone through the procedure at least once and in some regions already thrice. Using self-assessment, on-site observation and interviews, the state of cancer care in a hospital is evaluated and recommendations for improvement are given. Major topics of recommendations were the organisation of weekly multidisciplinary patient care meetings, shared decision making between specialists, oncological specialisation of medical specialists, dedication of oncology committees to policy making, introduction of integrated care pathways, referral policies for low volume tumours and highly complicated interventions and working according to evidence-based guidelines.

Few studies have been published on the clinical impact of external peer review. Roberts et al (2010, 2012) report on the 1- and 3-year evaluation of peer review for chronic obstructive pulmonary disease in the United Kingdom. Findings after 3 years indicated an association with improved quality of care, service delivery and changes that promote quality improvement (Roberts et al, 2012). The 1-year evaluation revealed no differences showing that changes need a longer period to occur (Roberts et al, 2010). More studies have been done on the effects of accreditation but the evidence remains uncertain (Greenfield and Braithwaite, 2008; Hinchcliff et al, 2012). Due to high financial and labor investments, calls have been made for more research concerning the clinical impact of these programmes (Shaw, 2001; Greenfield and Braithwaite, 2009).

The purpose of our study is to investigate whether (1) the participation in the external peer review programme focussing on multidisciplinary cancer care and (2) the extent of the implementation of the peer review recommendations impacted multidisciplinary treatment patterns (such as combined treatment modalities) and survival of colorectal cancer patients. Colorectal cancer was among the first type of cancer requiring multidisciplinary treatment. Due to new treatments, the quality of care has improved significantly in the last 20–30 years (Elferink et al, 2010a; van Steenbergen et al, 2010a). Studies have proven that regional and inter-hospital treatment variation exists that cannot be explained by medical factors only. It is suggested that hospital characteristics have a role in this variation (Elferink et al, 2010b, 2010c). Three major therapy changes requiring multidisciplinary cooperation have been introduced in the period under study: (1) the introduction of adjuvant chemotherapy in stage III colon cancer, (2) the introduction of preoperative radiotherapy in T2–T3 rectal cancer and (3) the introduction of preoperative chemoradiation in T4/M0 rectal cancer and tumours with an expected positive circumferential margin (CRM) (National Working Group on Gastrointestinal Cancer, 2012). We hypothesised that the willingness of a hospital to have external peer review and to follow the recommendations from it is correlated with the hospital giving higher quality of colorectal cancer treatment measured by the introduction of new multidisciplinary therapies and better survival of the colorectal cancer patients.

Material and methods

Design and patients

We selected all patients diagnosed with primary invasive epithelial colorectal cancer (ICD-O3 codes: colon C18.0–18.9, rectum C20.9) between 1 January 1990 and 31 December 2010 from the population-based Netherlands Cancer Registry. Patients diagnosed at autopsy, during an emergency operation or with previous malignancies were excluded. Patients from hospitals from the two regions where the programme was introduced first (Northern Netherlands and Rotterdam/South-west regions) form the intervention group. All hospitals in these regions voluntarily participated in the programme. The control group consists of patients from seven hospitals with otherwise comparable characteristics that did not participate before 2009, because the programme as such was not yet available in all regions.

Within the Netherlands Cancer Registry clinical administrative data of every newly diagnosed cancer patient in the Netherlands are collected. Topography and morphology are coded according to the International Classification of Diseases for Oncology and staging according to the TNM classification. Follow-up of vital status is achieved by linking the registry to municipal records. Quality of the data is high (Schouten et al, 1993a) and data completeness is estimated to be at least 95% (Schouten et al, 1993b).

The treatment and survival analyses are based on the hospital where the patient was diagnosed. Some patients may have been referred for treatment but this is considered as a good standard of care of the referring hospital. Furthermore, referral policies for low volume tumours were an important topic of the external peer review programme. We used the implementation proportion (IP) of all the recommendations given in the final reports of each peer review as a measure for the willingness of a hospital to implement the recommendations and the quality of colorectal cancer care. Data on the IP were obtained by studying the peer review reports, follow-up correspondence, hospital documents and interviews with stakeholders. Implementation was ranked per recommendation on a scale from 0 to 4 in which 4 represents total implementation and 0 a not implemented recommendation (Appendix 1). Scores per hospital were expressed as a percentage of the total score that could be achieved. When implementation of a recommendation could not be assessed (no data), the recommendation was subtracted from the total possible score. We used the average IP of all peer reviews per hospital because the time period in which changes can occur is unknown and quality improvement is a continuous process. Data from three cycles of peer review (1994–2009) were used from the Northern Netherlands and data from two cycles (1996–2006) from the Rotterdam region. A third cycle was completed in the Rotterdam region between 2009 and 2011 but the follow-up time was too short to monitor the IP. We did not make assumptions on what a high or a low IP is and therefore dichotomised the hospitals in the intervention group into two categories: (1) hospitals with the highest IP and (2) with the lowest IP.

Hospitals were asked for permission to use their data from the Netherlands Cancer Registry and programme reports. We excluded university hospitals and hospitals that merged during our study period since it was impossible to determine the IP.

Multidisciplinary treatment patterns

We studied the impact of the programme on the introduction of three major changes in multidisciplinary treatment: (1) adjuvant chemotherapy in stage III colon cancer, (2) preoperative radiotherapy in T2-T3/M0 rectal cancer and (3) preoperative chemoradiation in T4/M0 rectal cancer. Preoperative chemoradiation is also recommended in rectal cancer patients with tumours with an expected positive CRM (other than T4 tumours) but the Netherlands Cancer Registry does not provide data on the expected margin. We therefore focussed on the T4/M0 patients for the implementation of the preoperative chemoradiation. All patients with T2–3/M0 tumours, irrespective of their CRM, are therefore included in the analyses for preoperative radiotherapy as this is the minimal treatment they should have received.

Survival

The association between the programme and the survival was evaluated for the complete cohort subdivided into colon and rectal cancer patients. To examine the impact of the different programme phases on survival, we compared the 5-year survival of patients with the phase in which the hospital was at the time of diagnosis.

Statistical analyses

Clinical stage was used in our analyses for preoperative radiotherapy for T2–3/M0 rectal cancer patients and preoperative chemoradiation for T4/M0 rectal cancer patients. In case clinical stage was unknown it was substituted by pathological stage. Pathological stage was used for analysing the use of adjuvant chemotherapy in stage III colon cancer and survival, clinical stage was used when pathological stage was unknown. We excluded patients aged ⩾75 years for the analyses of the introduction of chemotherapy in stage III colon cancer and chemoradiation in T4/M0 rectal cancer to prevent a bias as elderly patients are known to receive systemic therapy less frequently (Elferink et al, 2010b, 2010c).

Multivariate logistic analysis was used to analyse the variation in treatment and the influence of participating in the programme and the IP, corrected for gender, age at diagnosis, year of diagnosis, average hospital volume of diagnoses and the presence of an in-hospital radiotherapy department. Because all analyses concern adjuvant therapy, only operated patients are included in the treatment analyses.

Using Cox’s proportional hazards model, we examined differences in hazard of dying adjusted for gender, age at diagnosis, year of diagnosis and average annual hospital volume of diagnoses. Survival time was defined as the period from incidence to date of death (all causes) or censuring (31 December 2011 or emigration date). For all analyses, STATA version 12.0 (StataCorp, College Station, TX, USA) was used.

Results

Population

We requested permission of 26 hospitals from the Northern Netherlands and Rotterdam region to use the data from their peer reviews and the Netherlands Cancer Registry, 23 hospitals gave their permission. Seven out of twelve hospitals without experience with the programme agreed to be included in the control group. In total, 45 705 patients were diagnosed with colorectal cancer in these 30 hospitals (about 1 out of 3 of all hospitals in the Netherlands) between 1990 and 2010, 31 890 patients with colon cancer and 13 815 patients with rectal cancer. A schematic overview of the study population at each phase of our study is presented in Appendix 2.

Implementation of programme recommendations

In the three cycles of peer review in the Northern Netherlands and two cycles in the Rotterdam region 727 recommendations were given to the hospitals. This is an average of 12 recommendations per peer review per hospital. The intervention group was dichotomised in 12 hospitals with a high IP (average 62.6%) and 11 hospitals with a low IP (average 44.8%). Table 1 shows the patient characteristics of the population of colon and rectal cancer patients for the intervention and control groups.

Table 1 Characteristics of the cohorts of colon (N=31 890) and rectal cancer patients (N=13 815) per hospital category, 1990–2010, data are no (%)
Full size table

Multidisciplinary treatment patterns

Out of the colon cancer patients, 4969 surgically treated patients under 75 years of age patients had stage III. Patients with stage III colon cancer who were diagnosed in hospitals in the intervention group received adjuvant chemotherapy more frequently compared with the control group (Table 2). This was seen in both the high IP and the low IP intervention hospitals (OR 1.48, 95% CI 1.25–1.75 and OR 1.19, 95% CI 1.00–1.41, respectively). An early adopter effect is seen in Figure 1 between 1990 and 2000, but the effect is also seen in the later years. Concerning the rectal cancer patients, 7804 patients were included in our treatment analyses with stage T2–T3/M0 cancer and 689 with T4/M0 rectal cancer. The analyses of the use of preoperative radiotherapy for T2–T3/M0 rectal cancer initially showed no difference between the control group and the intervention group (Table 2). Here the IP mattered as can be seen in Figure 2; patients who were diagnosed in the intervention hospitals with a high IP received preoperative radiotherapy more often (OR 1.31, 95% CI 1.11–1.55) while patients of intervention hospitals with a low IP had a lower chance (OR 0.75, 95% CI 0.63–0.88) compared with the control group. No differences were seen in the use of preoperative chemoradiation for T4/M0 rectal cancer patients (Table 2). Figure 3 shows that after the year 2000 the proportion of patients receiving preoperative chemoradiation rose but no statistically significant differences were seen between the intervention and control groups (Table 2).

Table 2 Odd’s ratio’s (OR) for receiving new multidisciplinary treatment per hospital category, adjusted for age, gender, year of diagnosis, average annual hospital volume of diagnoses
Full size table
Figure 1
figure 1

Introduction of adjuvant chemotherapy for stage III colon cancer (standard since 1990) per hospital category based on the implementation proportion (IP) of recommendations given in the programme. *represents statistical significance per year (P<0.05).

Full size image
Figure 2
figure 2

Introduction of preoperative radiotherapy in T2–T3/M0 rectal cancer per hospital category based on the implementation proportion (IP) of recommendations given in the programme. Official guideline introduction was in 2003. *represents statistical significance per year (P<0.05).

Full size image
Figure 3
figure 3

Introduction of preoperative chemoradiation in T4/M0 rectal cancer (recommend since 2005) per hospital category based on the implementation proportion (IP) of recommendations given in the programme. *represents statistical significance per 5-year period.

Full size image

Survival

The hazard of dying for patients with colon cancer was slightly lower in the intervention group compared with the control group (HR 0.97, 95% CI 0.94–0.99; Table 3). No statistical significant differences were found in the hazard of dying of rectal cancer between the intervention and the control groups (Table 3). Furthermore, there was no correlation seen between the average 5-year survival and the programme phases for both colon and rectal cancer (Figures 4 and 5).

Table 3 Hazard ratio’s (HR) for colon and rectal cancer patients per hospital category, adjusted for age, gender, year of diagnosis and average annual hospital volume of diagnoses
Full size table
Figure 4
figure 4

Average 5-year survival of colon cancer patients per hospital category and phase of the programme at the time of diagnosis. IP=implementation proportion.

Full size image
Figure 5
figure 5

Average 5-year survival of rectal cancer patients per hospital category and phase of the programme at the time of diagnosis. IP=implementation proportion.

Full size image

Discussion

To our knowledge, we present a unique long-term evaluation of the relationship between external peer review and treatment and survival of cancer patients based on the population-based data. There are some indications that the peer review increased process-related quality of care. Participation in the peer-review programme and the proportion of implementation of recommendations were associated with a higher proportion of stage III colon cancer patients who received adjuvant chemotherapy, whereas for rectal cancer the implementation of recommendations seems more relevant as patients diagnosed in hospitals with a high IP received preoperative radiotherapy more often. On the other hand, we did not find a difference in the percentage of patients receiving preoperative chemoradiation for T4/M0 rectal cancer related to participation or IP. Furthermore, a survival difference could only be shown for colon cancer, but not for rectal cancer and this did not seem to correlate with the phase of the programme.

A complicated association exists between external peer review, multidisciplinary care patterns and survival outcomes as many internal and external factors may be influential. Some differences in stage III colon cancer treatment for instance were already apparent before the introduction of the programme in 1994 (Figure 1). Does the programme as such have added value or do quality oriented hospitals also act as early adopters and simply implement recommendations better? The absence of a baseline measurement of organisational quality and innovative behaviour makes it impossible to answer this question. Based on our findings either relation is possible, but we tend to conclude that quality focussed hospitals are more likely to work on continuous quality improvement and to behave as early adopters.

Inter-hospital and regional variation in the treatment of colorectal cancer patients was shown in previous national studies (Elferink et al, 2010b, 2010c). Before 2000 there was both regional variation in guidelines and in their implementation in the Netherlands. This might partly explain differences in the results for adjuvant chemotherapy for stage III colon cancer as these are most prominent before the year 2000. The results in our study may also have been influenced by clinical trials conducted before official guideline recommendation. However, despite possible guideline and trial influences, recent studies still stress the patient- and hospital-dependent variation in adjuvant therapy, even after official guideline introduction of new therapies in the Netherlands and it is unlikely that this is different in other countries (Lemmens et al, 2005; Berrino et al, 2007; Elferink et al, 2010b, 2010c; van Steenbergen et al, 2010b). Although it is unlikely that guideline and trial influences alone can explain the treatment variation, it is very difficult to correct for these factors in studying the impact of peer review.

The average 5-year survival of colon and rectal cancer patients did not appear to be related to the phase of the peer review programme in which the hospital was at the time of diagnosis. Hazard ratio comparisons showed a significant difference in the risk of dying from colon cancer, favouring patients diagnosed in the intervention group. In rectal cancer, hazard ratios were comparable but not statistically different between the intervention and control groups. These differences are small and promising but need to be considered with caution. We analysed the risk of dying in the complete cohorts of colon or rectal cancer patients instead of the subgroups in which we studied the introduction of new multidisciplinary therapies. Reason for this is that especially in rectal cancer these new therapies predominantly have an effect on local control. Adjuvant chemotherapy is associated with improved survival in stage III colon cancer (Moertel et al, 1990). For rectal cancer, the benefit of preoperative radiotherapy and chemoradiation is mainly local control, the impact on survival is smaller and the evidence is more ambiguous (Colorectal Cancer Collaborative Group, 2001; Kapiteijn et al, 2001; Folkesson et al, 2005; Bosset et al, 2006).

The main weakness of our study (and most studies in this field) is that the impact of other, and possibly many, confounding factors could not be assessed. Survival and mortality are generally used as the ultimate indicators of quality of care in cancer studies, but are influenced by a complex set of internal and external factors and it is difficult to single out the programme impact. Hospitals in the control group are likely to have introduced changes in their organisation as well, but we are not aware of similar programmes that have been executed. Five hospitals from the approached control group and three from the approached intervention group did not give permission to use data from the NCR which might influenced our results, though the participating set seemed sufficiently representative for the Dutch situation.

Our research had several characteristics adding to a better understanding of the added value of the programme. It was possible to include patients from hospitals without experience in participating in the programme, creating a quasi-experimental situation. Because all hospitals in the intervention region participated in the programme (even though three did not give permission to use their data in this study), there was no programme participation bias. The Netherlands Cancer Registry provided us with reliable data over a long time period making it possible to analyse our results on a ‘patient-level’. We did not single out recommendations and neither did we address a ‘rank’ of importance to them to assess the programme impact instead of the impact of single recommendations.

Conclusion

Organisation focussed external quality assessment programmes have difficulties in demonstrating their added value on clinical care. All hospitals are willing to participate in external peer review but the proportion of implementation of recommendations of the programme differs. Our data show that some positive effects on cancer care can be expected but the results need to be interpreted cautiously. Future research on different types of cancer should assess whether our results can be generalised. A qualitative study can examine the perceived impact and influence on the sense of ownership among cancer specialists.

Improved organisation may be a value per se, especially in complex multidisciplinary treatment. However, if external quality assessment should provide measurable benefits for individual cancer patients, programmes probably need to focus more on specific aspects of the delivery of care and clinical outcomes. This will increase the possibilities to quantitatively evaluate the impact on the quality of care. Links with clinical audit systems and national cancer registries may lead to the reduction in administrative workload of these programmes and improved acceptance for continued external peer review for cancer care in the future.