Abstract
This editorial discusses 1) why data from real life should be included in clinical practice guidelines, and 2) how to integrate this real-life evidence in the guideline development process https://bit.ly/3wdLBDH
Introduction
Throughout the past decades, European Respiratory Society (ERS) task forces have produced and published clinical practice guidelines (CPGs), statements, technical standards and other documents to synthesise and summarise bodies of evidence for caregivers, and thereby improve healthcare quality in respiratory medicine. Among these various types of documents, only CPGs can propose recommendations for clinical practice. As such, they need to rely on a very strong methodology to limit the risk of recommending suboptimal care.
Traditionally, the highest levels of evidence come from randomised controlled trials (RCTs) [1, 2]. However, this does not mean that other types of research should be excluded from evidence synthesis as part of guidelines development processes. Indeed, they could usefully complement RCTs, provided that their methods are rigorous, and the results are properly analysed and transparently interpreted [3, 4]. In this editorial, we summarise how real-life evidence could and should be integrated in ERS CPGs.
The development of ERS clinical practice guidelines
The first crucial task of an ERS task force developing a CPG is to carefully consider the research questions and outcomes of interest. Only thereafter, a systematic review of the literature, as well as an assessment of the quality of evidence, can be performed (figure 1). For the latter, the ERS uses Grading of Recommendations, Assessment, Development and Evaluation (GRADE), an approach adopted and recommended by many organisations including the National Institute for Health and Care Excellence, the American Thoracic Society and the World Health Organization [5]. This rigorous method considers a number of factors, in addition to risk of bias, for assessing the quality of evidence. Moreover, it ensures a transparent link between evidence and recommendations when applying the Evidence to Decision (EtD) framework for grading of the strength of a recommendation [6].
The GRADE approach can be used for data originating from RCTs as well as observational studies [5]. When following the GRADE approach, the developers of guidelines must evaluate the risk of bias, inconsistency, indirectness and imprecision, as well as publication bias, to assess the certainty of evidence (table 1) [5]. This applies to data from both RCTs and observational studies. For the latter, large effect sizes, dose responses and opposing biases may lead to an upgrading of the quality of evidence. This systematic approach that takes every aspect of a published study into account makes it possible to ultimately adjust and grade the quality of evidence as “very low”, “low”, “moderate” or “high”. In a next step, the GRADE EtD framework allows for additional considerations, such as balance of benefits and harms, values and preferences, feasibility, equity, acceptability and resource use [6].
This systematic approach makes it possible for the expert panel to transparently draw their final conclusions while taking various aspects and perspectives into consideration, and make recommendations that are supported by the evidence [7].
Real-life evidence
The historical understanding that RCTs produce the evidence with the highest quality has often been challenged because patient populations in these studies often are selected, not reflecting the patients seen in everyday clinical practice [8]. Hence, clinically important data from real life may be missed and not sufficiently emphasised by healthcare professionals and policymakers. A brilliant example of the disparity between patients included in RCTs and real-life cohorts has recently been published by Brown et al. [9]. When comparing data from 342 patients against trial eligibility criteria from 37 RCTs evaluating biological therapies for severe asthma, less than 10% of patients in their real-life cohort were found to be eligible [9]. Similar concerns have been reported regarding other major lung diseases, for example COPD, lung cancer and bronchiectasis [10–16]. In such cases, the recommendations derived from RCTs might not be applicable to most patients.
Although the ERS applies a very strict methodological approach and considers both randomised and observational studies, the generalisability of our CPGs might be improved further by the inclusion of real-life evidence from other sources, such as administrative databases or healthcare registers [3]. These sources can complement RCTs by: 1) confirming or challenging their generalisability for different populations or settings, 2) exploring clinically relevant outcomes not available in RCTs, 3) providing safety data, and 4) allowing to explore possible determinants of treatment effects to be further confirmed in prospective RCTs [3]. In some cases, when it is not possible to perform an RCT due to, for example, ethical or feasibility reasons, real-life evidence might even be the only way to generate new data.
The ERS promotes an integrative approach in science and guideline development
Several ERS initiatives were implemented recently to close the gap that often exists between evidence generated from RCTs and real life. With the ERS Clinical Research Collaborations (CRCs), the respiratory community has the possibility to build international research networks to conduct pan-European pragmatic trials that are generalisable and sufficiently large to impact clinical practice (https://www.ersnet.org/science-and-research/clinical-research-collaboration-application-programme/). Furthermore, the ERS offers investigators the opportunity to promote their research by endorsing pragmatic trials (https://www.ersnet.org/science-and-research/pragmatic-trials-endorsement/). Pragmatic trials can be endorsed when they investigate respiratory diseases, meet stringent criteria of quality and are not dependent on a single sponsor from the pharmaceutical industry or another for-profit entity. In addition, CRCs provide an excellent platform to collaboratively develop and use data from healthcare registers [17]. Hence, the ERS promotes every type of evidence that can lead to a better understanding of a certain respiratory condition, as long as the highest quality standards are satisfied [3].
Conclusion: how to integrate real-life evidence in ERS CPGs
The crucial mission for ERS task forces developing CPGs is to appreciate not only data originating from RCTs, but also other sources, to get the best picture of the current evidence and draw solid conclusions [3, 4]. For CPGs, the ERS Guidelines Working Group recommends a thorough process with the selection of a limited number of clinical questions in a PICO (Population, Intervention, Comparison, Outcome) format that can include data both from RCTs and observational studies, and that are assessed via a systematic review and the application of the GRADE process [18]. These questions can be complemented with additional non-comparative questions that are addressed via a narrative review of the literature. The guideline panel then chooses outcomes that are critical or important for clinical decision making and that are relevant for the patients. For the final recommendations, the EtD framework should be systematically applied both for PICO and narrative questions. Real-life evidence that has not been considered in the systematic or narrative reviews should be considered and taken into account in the EtD framework [4].
With this approach, we ensure that ERS CPGs give recommendations that are transparent, trustworthy and clinically relevant both for clinicians and patients.
Shareable PDF
Supplementary Material
This one-page PDF can be shared freely online.
Shareable PDF ERJ-01718-2021.Shareable
Footnotes
Conflict of interest: M. Fally has nothing to disclose.
Conflict of interest: B. Nagavci has nothing to disclose.
Conflict of interest: T. Tonia acts as a methodologist for ERS.
Conflict of interest: M. van den Berge has nothing to disclose.
Conflict of interest: A. Bush has nothing to disclose.
Conflict of interest: C. Brightling has nothing to disclose.
Conflict of interest: N. Roche reports grants and personal fees from Boehringer Ingelheim, Novartis, GSK and Pfizer, personal fees from AstraZeneca, Chiesi, Sanofi and Zambon, outside the submitted work.
Support statement: M. Fally and B. Nagavci were recipients of a European Respiratory Society (ERS) Fellowship in Guidelines Methodology (MTF 2018-01 and MTF 2018-02).
- Received June 17, 2021.
- Accepted June 22, 2021.
- Copyright ©The authors 2021. For reproduction rights and permissions contact permissions{at}ersnet.org