Quality appraisal of clinical practice guidelines for motor neuron diseases or related disorders using the AGREE II instrument

Objectives: This study aimed to systematically assess the quality of CPGs for motor neuron diseases (MNDs) or related disorders and identify the gaps that limit evidence-based practice.

Methods: Four scientific databases and six guideline repositories were searched for eligible CPGs. Three researchers assessed the eligible CPGs using the Appraisal of Guidelines Research and Evaluation II instrument. The distribution of the level of evidence and strength of recommendation of these CPGs were determined. The univariate regression analysis was used to explore the characteristic factors affecting the quality of CPGs.

Results: Fifteen CPGs met the eligibility criteria: 10 were for MND and 5 were for spinal muscular atrophy. The mean overall rating score was 44.5%, and only 3 of 15 CPGs were of high quality. The domains that achieved low mean scores were applicability (24.4%), rigor of development (39.9%), and stakeholder involvement (40.3%). Most recommendations were based on low-quality evidence and had a weak strength. The CPGs that were updated, meant for adults, and evidence based, and used a CPG quality tool and a grading system were associated with higher scores in certain specific domains and overall rating.

Conclusion: The overall quality of CPGs for MNDs or related disorders was poor and recommendations were largely based on low-quality evidence. Many areas still need improvement to develop high-quality CPGs, and the use of CPG quality tools should be emphasized. A great deal of research on MNDs or related disorders is still needed to fill the large evidence gap.

1. Introduction

Motor neuron diseases (MNDs) or related disorders, such as MND, spinal muscular atrophy (SMA), and post-polio progressive muscular atrophy (PPMA), are a group of disorders characterized by progressive weakness secondary to degeneration of the motor neurons (1). MND is caused by the loss of motor neurons leading to muscle atrophy, paralysis, and ultimately death within 3–5 years after the onset of the disease (2). Although the treatment options for MND are limited and patient care primarily focuses on controlling symptoms and optimizing functioning and quality of life (3), better multidisciplinary care and a better understanding of interventions may allow patients with MND to live longer life (4). The data from the Global Burden of Disease study show that the burden of MND is increasing (5).

SMA is an autosomal recessive disease caused by biallelic mutations in the survival motor neuron 1 gene, causing progressive muscle weakness and atrophy (6). It imposes a significant burden on patients, caregivers, and the healthcare system (7), but the financial burden could be significantly reduced by expanding newborn screening for SMA in combination with early treatment interventions for newborns with SMA (8). For example, newborn screening for patients with SMA is associated with earlier diagnosis and intervention, and motor milestones are often achieved at an earlier age, compared with clinically diagnosed patients (8). PPMA, also known as post-polio syndrome (PPS), is a chronic progressive disorder that may appear decades after the initial acute polio infection (9), affecting 20–40% of polio survivors and manifesting as neuromuscular complications (10). Although the incidence of paralytic poliomyelitis has declined significantly (11), PPS will remain a major health problem for many years (12). At present, the diagnosis and management of MNDs or related disorders remain a challenge for clinicians and vary widely in practice.

One of the foundations of improving healthcare work are clinical practice guidelines (CPGs) (13). CPGs are defined as “statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of benefits and harms of alternative care options” (14). CPGs help improve the quality of health care, such as informing clinicians about treatment decisions for patients and determining the appropriate standard of care, thus identifying gaps between evidence and practice (15).

The usefulness of CPGs depends primarily on quality, rigorous methodology, and transparent development processes (16). Better CPG quality appears to be associated with more positive treatment outcomes (17). Some common problems with CPGs include the sheer volume available, large amount of documentation that is difficult to assimilate or use, lack of clear supporting evidence, exclusion of key stakeholders, insufficient editorial independence, and poor applicability (18–20). Besides, determining the level of evidence on which the recommendations are based is important (21, 22). However, the systemic appraisal of CPGs for MNDs or related disorders remians unreported, and the distribution of the level of evidence on which these recommendations are based has not been described.

Various tools have been developed to assess the quality of CPGs (23). The Appraisal of Guidelines Research and Evaluation (AGREE) Instrument was first published in 2003 and updated in 2009 (AGREE II). It is designed to assess the methodological quality of CPGs and also provide methodological strategies for the development of new CPGs (24). The AGREE II instrument is currently the preferred tool for the quality assessment of CPGs in the world and can be used to assess CPGs of multiple diseases (15, 24–27).

Therefore, the present study aimed to assess the quality of CPGs for MNDs or related disorders using the AGREE II instrument, identify the distribution of the level of evidence and strength of recommendations of these CPGs, and also identify the potential influencing factors for the quality of CPG. The findings of this study would help identify the gaps that hinder evidence-based practice and highlight potential opportunities for improvement.

2. Materials and methods

2.1. Eligibility criteria

CPGs were included if they: (1) focused on the diagnosis and management of MNDs or related disorders, including MND, SMA, and PPMA (1); (2) were published between January 1, 2006, and September 4, 2022; and (3) were written in English. Consistent with previous studies (25, 26), both evidence-based and consensus-based CPGs were included. If the CPG was updated, the latest version was included.

CPGs were excluded if (1) full texts were unavailable; (2) they were editorials, comments, reviews, letters, and correspondence studies; (3) they were interpretation, translation, and adaptation of CPGs; and (4) they were duplicate publications.

2.2. Literature search

Scientific databases, including PubMed, Embase, Cumulative Index to Nursing and Allied Health Literature, and Physiotherapy Evidence Database PEDro, were systematically searched under the direction of a reference librarian. Besides, the following six online guideline repositories were also searched: National Institute for Health and Clinical Excellence (NICE), Scottish Intercollegiate Guidelines Network, Guidelines International Networks, Agency for Healthcare Research and Quality, National Health and Medical Research Council, and World Health Organization. Keywords and Medical Subject Headings related to CPGs and MNDs or related disorders were combined in the database search. The specific search strategy is displayed in Supplementary File 1. The search range was from January 1, 2006, to September 4, 2022.

2.3. Study selection and data extraction

Two researchers (Jia-yin Ou and Jun-Jun Liu) independently performed the study selection and data extraction. Any disagreements between the two were resolved through discussion or consultation with a third researcher (Jing Xu). For study selection, the search results were first exported into the EndNote X7 literature management software (Thomson Reuters Corporation, CA, United States), excluding duplicates. Two researchers reviewed the titles and abstracts of the studies for screening to exclude the explicitly irrelevant studies, and then read the full texts of the remaining studies to determine whether they were finally included. For each CPG included in the end, the accompanying technical and supporting documents were thoroughly searched to better inform our evaluation.

The data were extracted into a specially designed spreadsheet. The extracted variables included the year of publication, disease, development organization, first author (if applicable), country/region of origin, version, age range of target population (adult/children/all ages), development method, search dates covered, CPG quality tool used, CPG methodologist included, title, funding sources, and accompanying documents. The country/region of origin was described as “Europe” if the CPG was jointly developed by multiple European countries, and “international” if it was jointly developed by multiple countries from different continents. The development method was described as “evidence based” if the CPG performed a systematic search and evaluation of evidence and made recommendations based on the evaluation results during the CPG development process; otherwise, it was described as “consensus based.”

2.4. Quality assessment

Three researchers (Jia-Yin Ou, Jun-Jun Liu, and Jing Xu) assessed the quality of each CPG independently using the AGREE II tool under the guidance of a methodological expert (Liming Lu). The AGREE II tool included 23 key items organized in 6 domains followed by 2 global rating items (“Overall Assessment”). The six domains included scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, and editorial independence. Each item was rated on a 7-point scale, with 1 indicating strongly disagree and 7 indicating strongly agree. Strongly disagree meant no information was relevant to an item, and strongly agree meant the quality of reporting was exceptional and the full criteria and considerations were met for an item. A score between 2 and 6 was assigned when the reporting did not meet the full criteria or considerations for an item. According to the calculation formula, the scores in each domain were calculated and the calculation formula was as follows: each domain score = (obtained score – minimum possible score)/ (maximum possible score – minimum possible score) × 100%. Consistent with a previous study (28), the score of a domain or overall rating ≤ 40% was considered as a low rating, >40 and ≤ 70% as a moderate rating, and > 70% as a high rating.

In the overall assessment, the first global rating item (“overall rating”) was scored on a 7-point scale and then calculated as a percentage, which was the same method used to calculate domain scores, as described in previous studies (15, 29). For the second global rating item, a CPG was rated as high quality when the score of three domains considered as the most important was ≥50% of the maximum possible score, consistent with previous studies (15, 30, 31). The three domains were as follows: stakeholder involvement (domain 2), rigor of development (domain 3), and editorial independence (domain 6).

Before the assessment, researchers received online training using the AGREE II Online Training Tool. Then, a meeting was held to discuss the specific assessment criteria of AGREE II, and researchers assessed some CPGs of different levels and discussed the results with each other. The formal assessment was performed only when the intraclass correlation coefficient (ICC) was >0.8. During the assessment process, researchers carefully read the document of each CPG and its accompanying documents or information on the Internet to make an accurate judgment. If the researchers’ score on an item differed significantly (more than 2 points), a consensus was reached through discussion.

2.5. Statistical analysis

The researchers’ AGREE score was entered into Microsoft Excel (Microsoft, WA, United States), and the standardized score of each domain and the overall score of each CPG were calculated. Continuous variables were expressed as mean ± standard deviation (SD) (normal distribution) or median (Q1–Q3) (skewed distribution), and categorical variables were expressed as frequencies and percentages. The independent-sample t test/nonparametric tests (Kruskal-Wallis test)/chi-square tests/Fisher exact tests were used to compare the differences between the two groups. As the overall scores of AGREE II domain, overall rating, and item of included CPGs had both normal and nonnormal distributions, consistent with the previous studies (27), the mean (SD) and median (Q1–Q3) of overall scores were both presented for the convenience of observation and comparison. The number of each level of evidence and the strength of recommendation of each CPG were evaluated. The univariate linear regression model and the logistic regression model were used to assess the associations between the characteristics and each AGREE II domain score and overall assessment of included CPGs. An ICC with a 95% confidence interval (CI) with a two-way random-effects model was used to detect the inter-rater agreement to ensure that researchers’ understandings of each item were basically the same, and ICCs for each domain and overall rating scores were calculated. Consistent with previous studies (25, 26) and according to Landis and Koch (32), the degree of agreement between 0.01 and 0.20 was considered minor, between 0.21 and 0.40 as fair, between 0.41 and 0.60 as moderate, between 0.61 and 0.80 as substantial, and between 0.81 and 1.00 as very good.

In the sensitivity analysis, a series of analyses were performed to test the robustness of our findings. First, other criteria were used for the overall assessment to identify whether the overall rating score and the number of high-quality CPGs differed from the initial results. For the first global rating item (“overall rating”), the score of each CPG was based on the average score of the six domains, consistent with previous studies (27, 33). A stricter standard was used for the second global rating item, consistent with previous studies (27, 34). A CPG was classified as of high quality if the score of domain 3 (rigor of development) was >70% and the scores of all other domains and overall rating were > 50%. Second, the univariate regression analysis, restricted to CPGs published after 2015 or evidence based or MND as the target disease, was performed separately. This helped assess the association between characteristics and each AGREE II domain score and overall assessment of included CPGs, determine whether these associations were consistent across different types of CPGs, and reduce confounders.

All analyses were performed using the following software packages: R (http://www.R-project.org, The R Foundation), EmpowerStats (http://www.empowerstats.com, X&Y Solutions, Inc., MA, United States), and SPSS 23.0 (IBM, IL, United States). A p value less than 0.05 (two-sided) indicated a statistically significant difference.

3. Results

3.1. Study selection

A total of 5,899 records were yielded initially, and 15 CPGs were finally included for assessment after screening by title, abstract, and full text (Figure 1). Two CPGs were both published in two parts but considered as one (6, 35–37).

FIGURE 1

Figure 1. PRISMA flow diagram. CPG, Clinical practice guideline; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

3.2. Characteristics of CPGs

The characteristics of included CPGs are displayed in Table 1 and Supplementary Table S1. Supplementary Table S2 shows the results of descriptive statistics of these characteristics. Nine CPGs were published after 2015. Eight CPGs were developed by a medical society and five by an expert panel. Ten CPGs were developed for MND and five for SMA; however, no CPG existed for PPMA. Five CPGs were originally from individual European countries, five were from individual North American countries, and the remaining were international, Europe, and Brazil. Six CPGs were updated. Three CPGs were for adults, one for children, and five for all ages; the rest did not specify the age group. Eight CPGs were considered evidence based and used a grading system. Seven CPGs reported the search dates. Only three CPGs used the CPG quality tool, such as the AGREE II tool. One CPG reported the inclusion of methodologists in the development team. Five CPGs were funded, four were not, and the remainder did not report the funding status. Compared with SMA CPGs, MND CPGs were significantly more likely to state the search dates (70.0% vs. 0.0%, p = 0.026) (Supplementary Table S3).

TABLE 1

Table 1. Characteristics of included CPGs.

3.3. Quality assessment of CPGs

Table 2 displays the overall mean (SD) and median (Q1–Q3) scores for each AGREE II domain, item, and overall rating of included CPGs. The mean overall rating score for all CPGs was 44.5% [SD = 24.7%, median = 39.0% (Q1–Q3, 25.0–58.5%)]. Overall, the highest domain score was for clarity of presentation (domain 4) [mean = 80.3% (SD, 14.8%), median = 85.0% (Q1–Q3, 67.0–92.0%)]. The next highest score was for scope and purpose (domain 1) [mean = 68.1% (SD, 23.8%), median = 67.0% (Q1–Q3, 52.0–88.5%)]. The lowest domain score was for applicability (domain 5) [mean = 24.4% (SD, 24.2%), median = 15.0% (Q1–Q3, 10.5–27.0%)], rigor of development (domain 3) [mean = 39.9% (SD, 29.5%), median = 28.0% (Q1–Q3, 14.0–57.5%)], and stakeholder involvement (domain 2) [mean = 40.3% (SD, 22.2%), median = 37.0% (Q1–Q3, 26.0–51.0%)]. The moderate domain score was for editorial independence (domain 6) [mean = 55.9% (SD, 34.4%), median = 47.0% (Q1–Q3, 32.0–94.0%)]. Compared with SMA CPGs, MND CPGs had a significantly lower domain 6 score [median = 80.5% (Q1–Q3, 47.8–96.2%) vs. median = 42.0% (Q1–Q3, 17.0–47.0%), p = 0.036] (Supplementary Table S4).

TABLE 2

Table 2. Overall mean (SD) and median (Q1–Q3) scores for each AGREE II domain, item, and overall rating of included CPGs.

Item 1 [mean = 6.1 (SD, 1.6), median = 7.0 (Q1–Q3, 6.0–7.0)] and item 15 [mean = 6.1 (SD, 0.9), median = 6.0 (Q1–Q3, 6.0–7.0)] had the highest scores. Item 21 [mean = 2.1 (SD, 1.8), median = 1.0 (Q1–Q3, 1.0–2.0)], item 19 [mean = 2.1 (SD, 2.0), median = 1.0 (Q1–Q3, 1.0–2.0)], and item 13 [mean = 2.1 (SD, 2.1), median = 1.0 (Q1–Q3, 1.0–1.0)] had the lowest scores.

Table 3 displays the AGREE II domain scores and overall assessment of each CPG. Figure 2 displays the distribution of the degree of score across each domain and the overall rating. In the domain scope and purpose, seven CPGs received high ratings, which were scored >70%. Six CPGs received moderate ratings, which were scored >40% and ≤ 70%. The National Institute for Health and Clinical Excellence (NICE) CPG received the highest scores (100%), and the Canadian Thoracic Society (CTS), Brazilian Medical Association, and the 2020 Canada CPG also got high scores, which were all >90%. The distribution of the rating degree was similar in the stakeholder involvement and rigor of development domains, with eight CPGs receiving low ratings, which were ≤ 40%. The 2020 Canada CPG received the highest scores in these two domains (domain 2: 89%, domain 3: 91%). In addition, the NICE CPG also received high ratings in both domains (domain 2: 70%, domain 3: 76%). The CTS received the second highest score in domain 3 (90%). Regarding the domain clarity of the presentation, the results were satisfactory. More than half of the CPGs received high ratings, and none received low ratings. In the domain applicability, thirteen CPGs received low ratings and only two CPGs received high ratings, and these two CPGs were the CTS (85%) and NICE CPG (74%). In the editorial independence domain, five CPGs received high ratings (33.3%) and six received moderate ratings (40.0%). Many CPGs lacked funding information and statements of competing interests. In the overall rating, eight CPGs received low ratings, and only the CTS (89%), NICE (78%), and 2020 Canada CPG (83%) received high ratings.

TABLE 3

Table 3. AGREE II domain scores and overall assessment of included CPGs.

FIGURE 2

Figure 2. Distribution of the degree of score across each domain and overall rating. CPG, Clinical practice guideline.

Among the 15 CPGs, 3 were of high quality, which was consistent with the result of the overall rating. These three CPGs were all for MND, while the NICE and CTS CPGs were for adults, and the 2020 Canada CPG was for all ages. The CTS and 2020 Canada CPGs were from Canada, and the NICE CPG was from the UK. All three CPGs were evidence based and used the CPG quality tool.

The inter-rater reliability was very good for all domains and the overall rating (Table 4). Supplementary File 2 displays the individual scoring of the AGREE II tool for each CPG.

TABLE 4

Table 4. Inter-rater reliability for AGREE II domain and overall rating.

3.4. Level of evidence and strength of recommendation of CPGs

Among the 15 CPGs, 8 were evidence based. Table 5 displays the grading systems used and the distribution of the level of evidence and strength of recommendation among these CPGs. Two CPGs used the adapted Grading of Recommendations Assessment, Development and Evaluation (GRADE) system, one used the combined Oxford Centre for Evidence-based Medicine and GRADE system, one used the American Academy of Pediatrics criteria, two used the adapted American Academy of Neurology (AAN) criteria, one used the AAN criteria, and one did not report the name of the grading system. Four CPGs graded evidence by a body of evidence, whereas four CPGs graded only individual studies. One CPG did not grade the strength of recommendations, and two CPG did not report the strength of specific recommendations.

TABLE 5

Table 5. Grading systems used and the distribution of the level of evidence and strength of recommendation among included evidence-based CPGs.

Although the CPGs used different grading systems, their grading criteria for the level of evidence and strength of recommendations were similar. Most of the evidence for which CPGs graded specific evidence (85.7%) were C, low, D, very low, EC, III, and IV level, indicating low-quality evidence. In the same way, most of the recommendations for which CPGs graded specific recommendations (88.2%) were graded as C, GCPP, U, 2, and weak strength, indicating weak recommendations.

3.5. Relationship between characteristics and AGREE II domain score and overall assessment of CPGs

Table 6 displays the results of the relationship between characteristics and AGREE II domain score and overall assessment of included CPGs. Compared with the MND CPGs, SMA CPGs had a lower domain 6 score (β = -37.9, 95% CI: −70.3 to −5.5). Compared with the CPGs of the first version, the updated CPGs had a higher domain 5 score (β = 24.9, 95% CI: 2.7–47.1), domain 6 score (β = 37.2, 95% CI: 6.3–68.1), and overall rating score (β = 26.4, 95% CI: 4.3–48.6). Compared with the CPGs for adults, CPGs for all ages had a lower domain 1 score (β = −36.8, 95% CI: −64.9 to −8.7), domain 3 score (β = −39.8, 95% CI: −76.5 to −3.1), and domain 5 score (β = −34.1, 95% CI: −64.7 to −3.6). CPGs that did not state the age range of the target population had a lower domain 1 score (β = −32.0, 95% CI: −59.3 to −4.7), domain 2 score (β = −30.7, 95% CI: −58.8 to −2.5), domain 3 score (β = −43.8, 95% CI: −79.3 to −8.3), domain 5 score (β = −36.7, 95% CI: −66.2 to −7.1), and overall rating score (β = -37.0, 95% CI: −68.1 to −5.9).

TABLE 6

Table 6. Relationship between characteristics and AGREE II domain score and overall assessment of included CPGs.

Compared with the consensus-based CPGs, evidence-based CPGs had a higher domain 1 score (β = 33.7, 95% CI: 16.7–50.7), domain 2 score (β = 26.2, 95% CI: 7.7–44.7), domain 3 score (β = 46.4, 95% CI: 28.3–64.4), and overall rating score (β = 38.6, 95% CI: 23.4–53.9). Compared with CPGs that stated the search dates, those that did not state the search dates had a lower domain 6 score (β = −42.3, 95% CI: −70.3 to −14.3). Compared with the CPGs using the CPG quality tool, those not using the CPG quality tool had a lower domain 1 score (β = −35.7, 95% CI: 60.1 to −11.2), domain 2 score (β = −42.2, 95% CI: −60.1 to −24.2), domain 3 score (β = −57.2, 95% CI: −80.2 to −34.3), domain 5 score (β = −50.3, 95%CI: −66.6 to −34.1), and overall rating score (β = −48.6, 95% CI: −67.3 to −29.8). Compared with the CPGs including a CPG methodologist, those not including a CPG methodologist had a lower domain 5 score (β = −43.0, 95% CI: −81.9 to −4.1), and those not stating had a lower domain 5 score (β = −63.3, 95% CI: −102.2 to −24.3). A comparison between CPGs using and not using the grading system showed the same results as that between consensus-based and evidence-based CPGs.

We also found that the year of publication, type of development organization, country/region of origin, and funding sources were not associated with the AGREE II domain score and overall assessment of included CPGs.

3.6. Sensitivity analysis

Supplementary Table S5 displays the overall assessment of included CPGs using other criteria. The mean overall rating score for all CPGs was 51.5% [SD = 19.9%, median = 51.0% (Q1–Q3, 36.5–61.0%)]. For the second global rating item, only the CTS CPG and the NICE CPG were of high quality under the stricter evaluation criteria. The 2020 Canada CPG was rated as low quality because its domain 5 score was less than 50%. Supplementary Tables S6–S8, respectively, display the relationship between characteristics and AGREE II domain score and overall assessment of included CPGs published after 2015, evidence-based CPGs, and CPGs for MND. The relationship between characteristics and each AGREE II domain score and overall assessment of different types of CPGs were generally consistent with the main analyses that the results in all included CPGs.

4. Discussion

This study was novel in assessing the quality of CPG for MNDs or related disorders using the AGREE II instrument and identifying the distribution of the level of evidence among these CPGs. The quality of CPGs for MNDs or related disorders varied widely. The overall quality of these CPGs was generally poor, and only three CPGs were rated as high quality. However, in a more stringent assessment standard, only two CPGs were of high quality. In comparison, the highest domain score was for clarity of presentation (domain 4) and the lowest domain score was for applicability (domain 5). Eight CPGs were considered evidence based, and most of the evidence (85.7%) on which these CPGs’ recommendations were based was low-quality evidence. Despite the improvement in the quality of CPGs in recent years, contemporary CPGs for MNDs or related disorders still lacked a consolidated evidence basis to provide recommendations for clinical practice. We also identified some factors that could affect the quality of CPGs, which could also serve as aspects for improvement.

Consistent with other studies (15, 30, 31), we used a less-stringent criterion in the overall assessment to judge whether a CPG was of high or low quality, but most CPGs were of poor quality. The development of CPGs requires significant resource consumption. Spending resources on low-quality CPGs with ineffective care recommendations is a waste and confusing for users (15). Coexisting problems are the uneven geographical distribution of CPGs and the duplication of CPGs. Most CPGs were from Europe and North America, only two CPGs were international, and one was from Brazil. Only Canada and the United Kingdom had high-quality CPGs, while other countries lacked them. All CPGs were for MND and SMA, of which SMA CPGs were all of low quality. No CPGs existed for PPS, although the prospect for the future is a continuous and ever-increasing demand for rehabilitation programs and management of patients with PPS (48, 49); currently around 18 million people are still affected by paralytic poliomyelitis (11). Dedicating resources to develop fewer, higher quality, and less-“redundant” CPGs can help reduce inefficient resource usage and user confusion (15, 51). Cooperation between countries and associations should be strengthened to reduce overlapping efforts and focus efforts and resources on developing high-quality CPGs and areas that need to be addressed.

Another problem is the inconsistent terminology used by CPG developers to describe the MND condition for which the CPGs are used. Also CPGs for MND include some CPGs for “amyotrophic lateral sclerosis (ALS)” and some for “MND.” Terminology can be confusing. MND is mainly divided into four categories: ALS, progressive bulbar palsy, progressive amyotrophy, and primary lateral sclerosis. When patients present with both upper and lower motor neuron signs, the disease is referred to as ALS, which is the most common form of MND (52). However, in the United Kingdom and Australia, MND is used as a general term for these disorders and also refers to the ALS subtype; however, in the United States, ALS is more commonly used as a general term and also denotes the ALS subtype (52). Consistent terminology is needed to define MND conditions, regardless of developer/professional group, to reduce CPG duplication and user confusion.

Many aspects of CPG development need to be improved. The domains of AGREE II that these aspects relate to are stakeholder involvement (domain 2), rigor of development (domain 3), applicability (domain 5), and editorial independence (domain 6). CPGs having problems in these domains were consistent with other studies (15, 19, 51). Among these, the problems of poor applicability and editorial independence of CPGs always existed, although the overall quality of CPGs in different health fields improved (51).

Stakeholder involvement reflects how well the CPG represents the views of its intended users, including patients. During the CPG development process, patients’ views, preferences, and expectations regarding care have become increasingly important (53). However, the overall score was low, with 53.3% of CPG scores ≤40% and only one CPG scoring greater than 70%. Most CPGs do not provide details about the involvement of patients or their representatives. Patients are important stakeholders and should be involved in the development process of CPGs, although this may introduce patient biases about costs, cultural background, and expectations (54).

Rigor of development is the most critical domain, as it significantly affects confidence in implementing CPGs (55), while nonsystematic development tends to lead to poor-quality CPGs (56). The strong heterogeneity found among the scores of included CPGs (Table 2) attested to the existence of significant gaps in the methodological development of the CPGs. Seven CPGs were not considered evidence based and did not use the grading system, or describe literature search and selection methods. The low score might be due to insufficient methodological consultation (55) or unfamiliarity with CPG development standards, poor reporting (19), or poor performance of the external peer review and update process (53).

The domain applicability considers factors that may affect guideline implementation, including identifying facilitators and barriers, providing tools for applying recommendations, identifying potentially relevant resources, and auditing standards (24). Thirteen CPGs received low ratings scoring ≤40%. The low applicability significantly hindered the implementation of CPG recommendations. CPG developers should conduct pilot tests to ensure feasibility before publication (57). In addition, all CPGS were just published as articles in journals. New ways to increase user adoption and usability should be considered by CPG developers (15), such as the use of smartphone applications (58, 59) or digital CPG platforms (60).

Editorial independence is also an important domain of CPG quality (61), and with only two statements required, it should be easy to score high (15). However, the overall score in this domain was not high [mean = 55.9% (SD, 34.4%)], with only five CPGs receiving high ratings, scoring >70%. Six CPGs did not state funding sources. Conflicts of interest are a common source of bias (62) and are often underestimated (63). CPG developers should clearly report conflicts of interest, including rigorous review processes and transparent review rules (64).

Overall, the NICE, CTS, and 2020 Canada CPGs (3, 41, 65) had the highest overall rating scores and were also rated as high quality. Based on this, these three CPGs should be favored by clinicians and policy makers, and are worthy of application in clinical practice.

Besides focusing on improving the transparency and methodological rigor of the CPG development process, CPGs should rely more on the growing body of evidence (26). However, nearly half of the CPGs for MNDs or related disorders were not considered evidence based, and most of the recommendations (85.7%) in the evidence-based CPGs were based on low-quality evidence, which were largely from observational studies or expert consensus. This finding constituted an obstacle to establishing CPGs for MND or related disorders, as recommendations were based on low-quality evidence, and it also showed a gap between clinical practice evidence and current medical research. Further research is needed on managing MND or related disorders, and more evidence-based recommendations would be extremely important to improve the standard of care for patients with MND or related disorders. Moreover, given that certain clinical questions may not addressed by high-quality research, it is expert consensus that can fill these knowledge gaps. Despite expert consensus may lack support from high-quality evidence, they can still provide valuable information. Consequently, CPGs developers should pay more attention to the rigor and standardization of consensus method and explore how to guide users to adopt expert consensus accurately.

Some characteristics and factors were found to be associated with the quality of CPGs. Specifically, among the CPGs for MND or related disorders, CPGs that were for MND, stated search dates, and included CPG methodologist were associated with higher scores in some specific domains, whereas CPGs that were updated, for adults, and evidence based, and used CPG quality tool and a grading system were associated with higher scores in both some specific domains and overall rating. In addition, the three CPGs that used the CPG quality tools were all rated as high quality, two of which used the AGREE II instrument. Therefore, the use of CPG quality tools, especially the AGREE II instrument, needs to be emphasized and improved during the development of CPGs. In addition, attention should also be paid to the use of the grading system, which has the greatest impact on the domain 3 score. The GRADE system is used to assess the level of evidence, while the AGREE II instrument is used to guide the CPG development process and set reporting standards (27), and they complement each other. Moreover, methodologists should be brought into the development team and should pay attention to CPG development details, such as providing the search date range of literature evidence.

This study had several limitations. Firstly, the search might have missed some CPGs, although the authors systematically searched major scientific databases and online guideline repositories. Secondly, only CPGs published in English were included, which might have excluded high-quality non-English CPGs, resulting in a lack of representation of CPGs from less-developed countries. Thirdly, the assessment of CPGs might reflect the researcher’s perspective, although our research team was multidisciplinary (including neurologists and other specialists, methodologists and other researchers), these limitations were unavoidable. Fourth, the AGREE II scoring system relied on the understandability and comprehensiveness of CPGs’ reporting and did not reflect the quality and strength of evidence (29). However, this study additionally analyzed the distribution of the level of evidence among included CPGs. Fifth, the AGREE II instrument did not provide a clear cut-off point to distinguish between high-quality and low-quality CPGs. To this end, based on some previous studies, we used a more widely used and less-stringent method to distinguish between high-quality and low-quality CPGs as the main analysis. Also, we used a more sensitive and stringent method to distinguish between high-quality and low-quality CPGs as a sensitivity analysis. Sixth, due to the small sample size, only univariate analysis was used to explore the relationship between CPG characteristics and AGREE II scores. However, we tried to reduce confounders by limiting the analysis to CPGs with certain characteristics, as a sensitivity analysis.

5. Conclusion

The quality of CPGs for MNDs or related disorders varied widely, and the overall quality was poor. No CPG existed for PPMA. Most recommendations were based on low-quality evidence. Many areas still need improvement, especially in the domains of stakeholder involvement, rigor of development, and applicability. CPGs for MNDs or related disorders should formulate recommendations with high-quality evidence and should be developed through rigorous methodology and transparency to minimize bias from external sources, and CPG quality tools should be used. In addition, a significant amount of studies on MNDs are still needed to fill the large evidence gap in the CPGs for these diseases.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding authors.

Author contributions

J-YO, LW, L-ML, and H-FP: study conception or design. J-YO, J-JL, JX, J-YL, and YL: data acquisition. J-YO, J-JL, JX, and J-YL: data analysis. J-YO, J-JL, and JX: manuscript writing. LW: funding acquisition. J-YO, J-JL, JX, J-YL, YL, Y-ZL, L-ML, H-FP, and LW: revision of manuscript. All authors contributed to the article and approved the submitted version.

Funding

This study was supported by the Youth Program of the National Natural Science Foundation of China (NO: 81904297), the Innovation Team and Talents Cultivation Program of National (No: ZYYCXTD-C-202004), the Special Project of “Lingnan Modernization of Traditional Chinese Medicine” in 2019 Guangdong Provincial R & D Program (NO: 2020B1111100008), the Key Laboratory Program of Universities in Guangdong Province (NO:2018KSYS006), and the Discipline Collaborative Innovation Team Program of Double First-class and High-level Universities for Guangzhou University of Chinese Medicine (NO: 2021XK01).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fneur.2023.1180218/full#supplementary-material

References

1. Lau, D. Neuromuscular disease affecting the larynx. Adv Otorhinolaryngol. (2020) 85:144–57. doi: 10.1159/000456694

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Liu, B, Li, M, Zhang, L, Chen, Z, and Lu, P. Motor neuron replacement therapy for amyotrophic lateral sclerosis. Neural Regen Res. (2022) 17:1633–9. doi: 10.4103/1673-5374.332123

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Shoesmith, C, Abrahao, A, Benstead, T, Chum, M, Dupre, N, Izenberg, A, et al. Canadian best practice recommendations for the management of amyotrophic lateral sclerosis. Can Med Assoc J. (2020) 192:E1453–68. doi: 10.1503/cmaj.191721

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Hardiman, O, al-Chalabi, A, Chio, A, Corr, EM, Logroscino, G, Robberecht, W, et al. Amyotrophic lateral sclerosis. Nat Rev Dis Primers. (2017) 3:17071. doi: 10.1038/nrdp.2017.71

CrossRef Full Text | Google Scholar

5. Logroscino, G, Urso, D, and Tortelli, R. The challenge of amyotrophic lateral sclerosis descriptive epidemiology: to estimate low incidence rates across complex phenotypes in different geographic areas. Curr Opin Neurol. (2022) 35:678–85. doi: 10.1097/WCO.0000000000001097

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Mercuri, E, Finkel, RS, Muntoni, F, Wirth, B, Montes, J, Main, M, et al. Diagnosis and management of spinal muscular atrophy: part 1: recommendations for diagnosis, rehabilitation, orthopedic and nutritional care. Neuromuscul Disord. (2018) 28:103–15. doi: 10.1016/j.nmd.2017.11.005

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Paracha, N, Hudson, P, Mitchell, S, and Sutherland, CS. Systematic literature review to assess the cost and resource use associated with spinal muscular atrophy management. PharmacoEconomics. (2022) 40:11–38. doi: 10.1007/s40273-021-01105-7

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Servais, L, de Vivo, D, Kirschner, J, Mercuri, E, Muntoni, F, Tizzano, E, et al. SMA - treatment. Neuromuscul Disord. (2021) 31:S138. doi: 10.1016/j.nmd.2021.07.315

CrossRef Full Text | Google Scholar

9. Boyer, FC, Tiffreau, V, Rapin, A, Laffont, I, Percebois-Macadré, L, Supper, C, et al. Post-polio syndrome: pathophysiological hypotheses, diagnosis criteria, drug therapy. Ann Phys Rehabil Med. (2010) 53:34–41. doi: 10.1016/j.rehab.2009.12.003

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Saraswathy, R. The genetics of post-polio syndrome - much to be unravelled! Indian J Med Res. (2016) 144:9–10. doi: 10.4103/0971-5916.193277

PubMed Abstract | CrossRef Full Text | Google Scholar

11. World Health Organization. International classification of diseases 11th revision. (2022). Available at: https://icd.who.int/en

Google Scholar

12. de Lira, C, Santos, D, Viana, RB, Guimarães, JM, Oliveira, J, Sousa, BS, et al. Knowledge of healthcare professionals about poliomyelitis and postpoliomyelitis: a cross-sectional study. São Paulo Med J. (2021) 139:464–75. doi: 10.1590/1516-3180.2020.0617.16032021

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Eccles, MP, Grimshaw, JM, Shekelle, P, Schünemann, HJ, and Woolf, S. Developing clinical practice guidelines: target audiences, identifying topics for guidelines, guideline group composition and functioning and conflicts of interest. Implement Sci. (2012) 7:60. doi: 10.1186/1748-5908-7-60

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Trustworthy, IOMU, and Guidelines, CP. Clinical practice Guidelines we can trust. Washington (DC): National Academies Press (US) (2011).

Google Scholar

15. Lin, I, Wiles, LK, Waller, R, Goucke, R, Nagree, Y, Gibberd, M, et al. Poor overall quality of clinical practice guidelines for musculoskeletal pain: a systematic review. Br J Sport Med. (2018) 52:337–43. doi: 10.1136/bjsports-2017-098375

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Birring, SS, Kavanagh, J, Lai, K, and Chang, AB. Adult and paediatric cough guidelines: ready for an overhaul? Pulm Pharmacol Ther. (2015) 35:137–44. doi: 10.1016/j.pupt.2015.01.007

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Brouwers, MC, Kho, ME, Browman, GP, Burgers, JS, Cluzeau, F, Feder, G, et al. Development of the AGREE II, part 1: performance, usefulness and areas for improvement. Can Med Assoc J. (2010) 182:1045–52. doi: 10.1503/cmaj.091714

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Baiardini, I, Braido, F, Bonini, M, Compalati, E, and Canonica, GW. Why do doctors and patients not follow guidelines? Curr Opin Allergy Clin Immunol. (2009) 9:228–33. doi: 10.1097/ACI.0b013e32832b4651

CrossRef Full Text | Google Scholar

19. Alonso-Coello, P, Irfan, A, Sola, I, Gich, I, Delgado-Noguera, M, Rigau, D, et al. The quality of clinical practice guidelines over the last two decades: a systematic review of guideline appraisal studies. Qual Saf Health Care. (2010) 19:e58. doi: 10.1136/qshc.2010.042077

PubMed Abstract | CrossRef Full Text | Google Scholar

20. van Dijk, MR, Steyerberg, EW, and Habbema, JD. A decision-analytic approach to define poor prognosis patients: a case study for non-seminomatous germ cell cancer patients. Bmc Med Inform Decis Mak. (2008) 8:1. doi: 10.1186/1472-6947-8-1

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Isaac, A, Saginur, M, Hartling, L, and Robinson, JL. Quality of reporting and evidence in American Academy of Pediatrics guidelines. Pediatrics. (2013) 131:732–8. doi: 10.1542/peds.2012-2027

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Khan, AR, Khan, S, Zimmerman, V, Baddour, LM, and Tleyjeh, IM. Quality and strength of evidence of the Infectious Diseases Society of America clinical practice guidelines. Clin Infect Dis. (2010) 51:1147–56. doi: 10.1086/656735

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Siering, U, Eikermann, M, Hausner, E, Hoffmann-Eßer, W, and Neugebauer, EA. Appraisal tools for clinical practice guidelines: a systematic review. PLoS One. (2013) 8:e82915. doi: 10.1371/journal.pone.0082915

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Brouwers, MC, Kho, ME, Browman, GP, Burgers, JS, Cluzeau, F, Feder, G, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. J Clin Epidemiol. (2010) 63:1308–11. doi: 10.1016/j.jclinepi.2010.07.001

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Shen, WQ, Yao, L, Wang, XQ, Hu, Y, and Bian, ZX. Quality assessment of cancer cachexia clinical practice guidelines. Cancer Treat Rev. (2018) 70:9–15. doi: 10.1016/j.ctrv.2018.07.008

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Jiang, M, Guan, WJ, Fang, ZF, Xie, YQ, Xie, JX, Chen, H, et al. A critical review of the quality of cough clinical practice Guidelines. Chest. (2016) 150:777–88. doi: 10.1016/j.chest.2016.04.028

PubMed Abstract | CrossRef Full Text | Google Scholar

27. O'Shaughnessy, SM, Lee, JY, Rong, LQ, Rahouma, M, Wright, DN, Demetres, M, et al. Quality of recent clinical practice guidelines in anaesthesia publications using the appraisal of Guidelines for research and evaluation II instrument. Br J Anaesth. (2022) 128:655–63. doi: 10.1016/j.bja.2021.11.037

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Bakaloudi, DR, Chrysoula, L, Poulia, KA, Dounousi, E, Liakopoulos, V, and Chourdakis, M. AGREEing on nutritional Management of Patients with CKD-A quality appraisal of the available Guidelines. Nutrients. (2021) 13:624. doi: 10.3390/nu13020624

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Andrade, R, Pereira, R, van Cingel, R, Staal, JB, and Espregueira-Mendes, J. How should clinicians rehabilitate patients after ACL reconstruction? A systematic review of clinical practice guidelines (CPGs) with a focus on quality appraisal (AGREE II). Br J Sport Med. (2020) 54:512–9. doi: 10.1136/bjsports-2018-100310

CrossRef Full Text | Google Scholar

30. Haran, C, van Driel, M, Mitchell, BL, and Brodribb, WE. Clinical guidelines for postpartum women and infants in primary care-a systematic review. BMC Pregnancy Childbirth. (2014) 14:51. doi: 10.1186/1471-2393-14-51

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Bouwmeester, W, van Enst, A, and van Tulder, M. Quality of low back pain guidelines improved. Spine. (2009) 34:2562–7. doi: 10.1097/BRS.0b013e3181b4d50d

CrossRef Full Text | Google Scholar

32. Kramer, MS, and Feinstein, AR, Clinical biostatistics LIV. The biostatistics of concordance. Clin Pharmacol Ther. (1981) 29:111–23. doi: 10.1038/clpt.1981.18

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Luu, NN, Chorath, KT, May, BR, Bhuiyan, N, Moreira, AG, and Rajasekaran, K. Clinical practice guidelines in idiopathic facial paralysis: systematic review using the appraisal of guidelines for research and evaluation (AGREE II) instrument. J Neurol. (2021) 268:1847–56. doi: 10.1007/s00415-020-10345-0

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Corp, N, Mansell, G, Stynes, S, Wynne-Jones, G, Morsø, L, Hill, JC, et al. Evidence-based treatment recommendations for neck and low back pain across Europe: a systematic review of guidelines. Eur J Pain. (2021) 25:275–95. doi: 10.1002/ejp.1679

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Finkel, RS, Mercuri, E, Meyer, OH, Simonds, AK, Schroth, MK, Graham, RJ, et al. Diagnosis and management of spinal muscular atrophy: part 2: pulmonary and acute care; medications, supplements and immunizations; other organ systems; and ethics. Neuromuscul Disord. (2018) 28:197–207. doi: 10.1016/j.nmd.2017.11.004

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Miller, RG, Jackson, CE, Kasarskis, EJ, England, JD, Forshew, D, Johnston, W, et al. Practice parameter update: the care of the patient with amyotrophic lateral sclerosis: multidisciplinary care, symptom management, and cognitive/behavioral impairment (an evidence-based review): report of the quality standards Subcommittee of the American Academy of neurology. Neurology. (2009) 73:1227–33. doi: 10.1212/WNL.0b013e3181bc01a4

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Miller, RG, Jackson, CE, Kasarskis, EJ, England, JD, Forshew, D, Johnston, W, et al. Practice parameter update: the care of the patient with amyotrophic lateral sclerosis: drug, nutritional, and respiratory therapies (an evidence-based review): report of the quality standards Subcommittee of the American Academy of neurology. Neurology. (2009) 73:1218–26. doi: 10.1212/WNL.0b013e3181bc0141

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Georges, M, Perez, T, Rabec, C, Jacquin, L, Finet-Monnier, A, Ramos, C, et al. Proposals from a French expert panel for respiratory care in ALS patients. Respir Med Res. (2022) 81:100901. doi: 10.1016/j.resmer.2022.100901

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Corcia, P, Lumbroso, S, Cazeneuve, C, Mouzat, K, Camu, W, and Vourc'H, P. Pre-symptomatic diagnosis in ALS. Rev Neurol. (2020) 176:166–9. doi: 10.1016/j.neurol.2019.07.027

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Pattee, GL, Plowman, EK, (Focht) Garand, KL, Costello, J, Brooks, BR, Berry, JD, et al. Provisional best practices guidelines for the evaluation of bulbar dysfunction in amyotrophic lateral sclerosis. Muscle Nerve. (2019) 59:531–6. doi: 10.1002/mus.26408

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Motor neurone disease: assessment and management. London: National Institute for health and care excellence (NICE) (2019). Available at: https://www.nice.org.uk/guidance/ng

Google Scholar

42. Silvinato, A, and Bernardo, WM. Spinal muscular atrophy 5Q - treatment with nusinersen. Rev Assoc Med Bras. (2018) 64:484–91. doi: 10.1590/1806-9282.64.06.484

CrossRef Full Text | Google Scholar

43. Cuisset, JM, and Estournet, B. Recommendations for the diagnosis and management of typical childhood spinal muscular atrophy. Rev Neurol. (2012) 168:902–9. doi: 10.1016/j.neurol.2012.07.020

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Pitarch Castellano, I, Cabrera-Serrano, M, Calvo Medina, R, Cattinari, MG, Espinosa García, S, Fernández-Ramos, JA, et al. Delphi consensus on recommendations for the treatment of spinal muscular atrophy in Spain (RET-AME consensus). Neurologia (Engl Ed). (2022) 37:216–28. doi: 10.1016/j.nrleng.2021.07.002

PubMed Abstract | CrossRef Full Text | Google Scholar

45. EFNS Task Force on Diagnosis and Management of Amyotrophic Lateral SclerosisAndersen, PM, Abrahams, S, Borasio, GD, de Carvalho, M, Chio, A, et al. EFNS guidelines on the clinical management of amyotrophic lateral sclerosis (MALS)--revised report of an EFNS task force. Eur J Neurol. (2012) 19:360–75. doi: 10.1111/j.1468-1331.2011.03501.x

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Filippi, M, Agosta, F, Abrahams, S, Fazekas, F, Grosskreutz, J, Kalra, S, et al. EFNS guidelines on the use of neuroimaging in the management of motor neuron diseases. Eur J Neurol. (2010) 17:526–e20. doi: 10.1111/j.1468-1331.2010.02951.x

CrossRef Full Text | Google Scholar

47. ACOG committee opinion No. 432: spinal muscular atrophy. Obstet Gynecol. (2009) 113:1194–6. doi: 10.1097/AOG.0b013e3181a6d03a

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Lo, JK, and Robinson, LR. Post-polio syndrome and the late effects of poliomyelitis: part 2. Treatment, management, and prognosis. Muscle Nerve. (2018) 58:760–9. doi: 10.1002/mus.26167

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Lo, JK, and Robinson, LR. Postpolio syndrome and the late effects of poliomyelitis. Part 1. Pathogenesis, biomechanical considerations, diagnosis, and investigations. Muscle Nerve. (2018) 58:751–9. doi: 10.1002/mus.26168

PubMed Abstract | CrossRef Full Text | Google Scholar

50. de Carvalho, M, Dengler, R, Eisen, A, England, JD, Kaji, R, Kimura, J, et al. Electrodiagnostic criteria for diagnosis of ALS. Clin Neurophysiol. (2008) 119:497–503. doi: 10.1016/j.clinph.2007.09.143

CrossRef Full Text | Google Scholar

51. Armstrong, JJ, Goldfarb, AM, Instrum, RS, and MacDermid, JC. Improvement evident but still necessary in clinical practice guideline quality: a systematic review. J Clin Epidemiol. (2017) 81:13–21. doi: 10.1016/j.jclinepi.2016.08.005

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Abdul, WS, Law, ZK, Ismail, NA, Azman, AR, and Lai, NM. Cell-based therapies for amyotrophic lateral sclerosis/motor neuron disease. Cochrane Database Syst Rev. (2016) 11:D11742. doi: 10.1002/14651858.CD011742.pub2

CrossRef Full Text | Google Scholar

53. Gallardo, CR, Rigau, D, Irfan, A, Ferrer, A, Caylà, JA, Bonfill, X, et al. Quality of tuberculosis guidelines: urgent need for improvement. Int J Tuberc Lung Dis. (2010) 14:1045–51.

Google Scholar

54. Hoydonckx, Y, Kumar, P, Flamer, D, Costanzi, M, Raja, SN, Peng, P, et al. Quality of chronic pain interventional treatment guidelines from pain societies: assessment with the AGREE II instrument. Eur J Pain. (2020) 24:704–21. doi: 10.1002/ejp.1524

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Acuña-Izcaray, A, Sánchez-Angarita, E, Plaza, V, Rodrigo, G, de Oca, MM, Gich, I, et al. Quality assessment of asthma clinical practice guidelines: a systematic appraisal. Chest. (2013) 144:390–7. doi: 10.1378/chest.12-2005

CrossRef Full Text | Google Scholar

56. Burgers, JS, Cluzeau, FA, Hanna, SE, Hunt, C, and Grol, R. Characteristics of high-quality guidelines: evaluation of 86 clinical guidelines developed in ten European countries and Canada. Int J Technol Assess Health Care. (2003) 19:148–57. doi: 10.1017/s026646230300014x

CrossRef Full Text | Google Scholar

57. Jiang, M, Liao, LY, Liu, XQ, He, WQ, Guan, WJ, Chen, H, et al. Quality assessment of clinical practice Guidelines for respiratory diseases in China: a systematic appraisal. Chest. (2015) 148:759–66. doi: 10.1378/chest.14-3201

CrossRef Full Text | Google Scholar

58. Free, C, Phillips, G, Watson, L, Galli, L, Felix, L, Edwards, P, et al. The effectiveness of mobile-health technologies to improve health care service delivery processes: a systematic review and meta-analysis. PLoS Med. (2013) 10:e1001363. doi: 10.1371/journal.pmed.1001363

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Ventola, CL. Mobile devices and apps for health care professionals: uses and benefits. P T. (2014) 39:356–64.

PubMed Abstract | Google Scholar

60. Siemieniuk, RA, Agoritsas, T, Macdonald, H, Guyatt, GH, Brandt, L, and Vandvik, PO. Introduction to BMJ rapid recommendations. BMJ. (2016) 354:i5191. doi: 10.1136/bmj.i5191

CrossRef Full Text | Google Scholar

61. Cosgrove, L, Bursztajn, HJ, Erlich, DR, Wheeler, EE, and Shaughnessy, AF. Conflicts of interest and the quality of recommendations in clinical guidelines. J Eval Clin Pract. (2013) 19:674–81. doi: 10.1111/jep.12016

CrossRef Full Text | Google Scholar

62. Detsky, AS. Sources of bias for authors of clinical practice guidelines. Can Med Assoc J. (2006) 175:1033–5. doi: 10.1503/cmaj.061181

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Neuman, J, Korenstein, D, Ross, JS, and Keyhani, S. Prevalence of financial conflicts of interest among panel members producing clinical practice guidelines in Canada and United States: cross sectional study. BMJ. (2011) 343:d5621. doi: 10.1136/bmj.d5621

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Norris, SL, Holmer, HK, Ogden, LA, and Burda, BU. Conflict of interest in clinical practice guideline development: a systematic review. PLoS One. (2011) 6:e25153. doi: 10.1371/journal.pone.0025153

PubMed Abstract | CrossRef Full Text | Google Scholar

65. Rimmer, KP, Kaminska, M, Nonoyama, M, Giannouli, E, Maltais, F, Morrison, DL, et al. Home mechanical ventilation for patients with amyotrophic lateral sclerosis: a Canadian thoracic society clinical practice guideline. Can J Respir. (2019) 3:9–27. doi: 10.1080/24745332.2018.1559644

CrossRef Full Text | Google Scholar