http://orcid.org/0000-0002-1872-5566
Figures
Abstract
Background
Due to increasing antimicrobial resistance in low-resource settings, strategies to rationalize antibiotic treatment of children unlikely to have a bacterial infection are needed. This study’s objective was to utilize a database of placebo treated Malawian children with World Health Organization (WHO) fast breathing pneumonia to develop a prognostic risk score that could aid antibiotic decision making.
Methods
We conducted a secondary analysis of children randomized to the placebo group of the Innovative Treatments in Pneumonia (ITIP) fast breathing randomized, controlled, noninferiority trial. Participants were low-risk HIV-uninfected children 2–59 months old with WHO fast breathing pneumonia in Lilongwe, Malawi. Study endpoints were treatment failure, defined as either disease progression at any time on or before Day 4 of treatment or disease persistence on Day 4, or relapse, considered as the recurrence of pneumonia or severe disease among previously cured children between Days 5 and 14. We utilized multivariable linear regression and stepwise model selection to develop a model to predict the probability of treatment failure or relapse.
Results
Treatment failure or relapse occurred in 11.5% (61/526) of children included in this analysis. The final model incorporated the following predictors: heart rate terms, mid-upper arm circumference, malaria status, water source, family income, and whether or not a sibling or other child in the household received childcare outside the home. The model’s area under the receiver operating characteristic score was 0.712 (95% confidence interval 0.66, 0.78) and it explained 6.1% of the variability in predicting treatment failure or relapse (R2, 0.061). For the model to categorize all children with treatment failure or relapse correctly, 77% of children without treatment failure or relapse would require antibiotics.
Citation: McCollum ED, Brown SP, Nkwopara E, Mvalo T, May S, Ginsburg AS (2019) Development of a prognostic risk score to aid antibiotic decision-making for children aged 2-59 months with World Health Organization fast breathing pneumonia in Malawi: An Innovative Treatments in Pneumonia (ITIP) secondary analysis. PLoS ONE 14(6): e0214583. https://doi.org/10.1371/journal.pone.0214583
Editor: Peter Collignon, Canberra Hospital, AUSTRALIA
Received: September 13, 2018; Accepted: March 14, 2019; Published: June 20, 2019
Copyright: © 2019 McCollum et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The Bill & Melinda Gates Foundation (OPP1105080) funded this study. EDM received support from the National Institutes of Health through the Fogarty International Center of the National Institutes of Health (K01TW009988). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
According to the most recent global estimates, the greatest numbers of children under five years old who die each year live in sub-Saharan Africa [1]. Presumed bacterial pneumonia is the leading cause of child mortality with 490,000 annual deaths in Africa, and 921,000 annual deaths globally [1]. World Health Organization (WHO) guidelines define child pneumonia as a clinical syndrome with either fast breathing, lower chest wall indrawing, or general clinical danger signs in children aged 2–59 months with cough or difficult breathing [2]. Introduction of these pragmatic guidelines have markedly improved antibiotic access, contributing substantially to the decline in child pneumonia mortality over the past two decades in low- and middle-income countries (LMICs) [3].
The more recent introduction of vaccines for Haemophilus influenzae and Streptococcus pneumoniae, two important etiologies for fatal bacterial pneumonia, to the sub-Saharan African region, have additionally accelerated this mortality decline [4]. However, these vaccines are also speculated to be driving the transition of pneumonia etiology away from more severe bacterial and viral co-infections toward less severe viral causes [5, 6]. This epidemiologic shift is suggested to be reducing the specificity of the WHO guidelines such that increasing numbers of children with non-bacterial pneumonia are unnecessarily treated with antibiotics [7]. In parallel to this is the concerning rise of antimicrobial resistance (AMR) in LMICs [8]. Recent estimates indicate that high proportions of pathogens in sub-Saharan Africa have developed AMR to first-line antibiotics [9–12]. Innovative, pragmatic strategies that improve antibiotic stewardship among children at low risk of mortality with presumed pneumonia and are feasible to implement in LMICs are a priority [13].
In this secondary analysis of the Innovative Treatments in Pneumonia (ITIP) fast breathing randomized, controlled, noninferiority trial [14], we sought to develop a novel prognostic risk score to aid antibiotic decision making by leveraging a unique dataset that included blinded placebo treatment of children aged 2–59 months with WHO fast breathing pneumonia in the sub-Saharan African country of Malawi. The objective of this score would be to identify those children at initial clinical presentation who are at low risk of treatment failure or relapse without antibiotic treatment. This score would serve to complement current WHO guidelines in deciding whether or not to prescribe antibiotics to low risk children in LMICs with pneumonia characterized by fast breathing only and who are without other established risk factors associated with poor outcomes.
Materials and methods
Setting
ITIP (ClinicalTrials.gov registration: NCT02760420) was a prospective, double-blind, randomized, controlled, noninferiority trial of 1,126 HIV-uninfected children aged 2–59 months with WHO fast breathing pneumonia conducted between 2016–2017 in Lilongwe, Malawi. The trial’s primary aim was to determine if placebo treatment was noninferior to three days of amoxicillin treatment. This secondary analysis utilized data from children enrolled in the placebo group of this trial with outcome data. ITIP1 took place in the capital city of Lilongwe within the outpatient department of Bwaila District Hospital (BDH), which provides primary healthcare, and the outpatient department and pediatric wards of Kamuzu Central Hospital (KCH), the tertiary referral hospital for the central region of Malawi.
Participants
All caregivers of participant children provided written informed consent. The ITIP1 fast breathing pneumonia case definition was consistent with WHO guidelines, and as shown in the Table 1, eligibility criteria included low risk children aged 2–59 months with cough or difficult breathing <14 days who also had fast breathing for age. Enrolled children who did not meet all eligibility criteria were excluded from this secondary analysis, as were children missing the outcome or key variables.
Data collection
ITIP study clinicians, called clinical and medical officers, as well as nurses trained to a standardized screening protocol under the supervision of an ITIP study pediatrician, were based at BDH and KCH, and assessed children aged 2–59 months with fast breathing pneumonia for trial eligibility. If children were found eligible, they were randomized double-blinded in a 1:1 ratio to three days of either placebo or amoxicillin treatment. To optimize child safety, all study children were monitored at KCH for 2–8 hours after enrollment. Study staff discontinued monitoring after two hours for children without fever who were breathing slower than at enrollment. Those children <6 months old, with a mid-upper arm circumference (MUAC) 11.5–13.5cm, or febrile with a negative malaria test were monitored overnight. At any point during monitoring, if a child clinically deteriorated, then parenteral antibiotics were initiated, and the child was classified as treatment failure (Table 2).
Study staff contacted caregivers of children not under hospital monitoring on Days 1–3 by phone, and assessed children in person at the study clinic or at home on Days 2–4 and Day 14. At all in person assessments, whether scheduled or not, study staff evaluated children for treatment failure or relapse, and study drug adherence.
Outcome definitions
Treatment failure was defined as occurring anytime on or before study Day 4. For children without treatment failure on Day 4, they were considered to have relapsed if they were found to have recurrence of signs of pneumonia or severe disease on or before Day 14 (Table 2).
Analysis
Continuous data were summarized with means and standard deviations, and differences between children with and without treatment failure or relapse were tested with a student’s t-test. For categorical data, we gave the number and proportion falling into each category and used Pearson chi-squared tests to compare proportions.
We developed a prognostic model for the probability of treatment failure or relapse using multivariable linear regression through stepwise model selection [15]. Linear regression was used to allow direct estimate of the failure probability, with robust standard error to accommodate the binary nature of the outcome, reasonable with the given sample size. The same model was used in the primary analysis of the study. Variables with a significance level of 0.10 or lower entered the model; those with a significance level below 0.20 remained in the final model. We chose cut-points on the high end of what is used in stepwise model building as we hoped to maximize the predictive performance of the model rather than focusing on the interpretation of the selected variables [16, 17]. Covariates to be considered for the prognostic model were selected a priori based on previous knowledge supported either by published literature or expert input. For continuous variables, the functional form included in the model building was based on bivariate modeling of each factor with the outcome. For each, we considered a linear form, a spline, and categorizations selected a priori; when the spline suggested a polynomial fit, one was considered as well. The form with the highest Akaike Information Criterion (AIC) was used in the model building [18]. The use of non-linear relationships such as splines and quadratics prioritized the development of the best fitting model regardless of interpretation complexity, under the assumption that the model could be implemented during care using electronic decision support aids. Missing covariates were minimal, with the exception of family income, which was explicitly modeled. We used R2, AIC, and area under the receiver operating characteristic (AUROC) to evaluate the model performance. We also assessed the model under the presumption that the prognostic risk score would only be acceptable for implementation by policy makers, healthcare providers, and caregivers if all children with treatment failure or relapse were correctly identified for antibiotic treatment. Therefore, we additionally evaluated the model’s performance by determining what percentage of children the model would falsely predict with likely treatment failure or relapse, and therefore receive antibiotic treatment, in order for the model to identify all children who would become a true treatment failure or relapse case without antibiotics. SAS 9.4 was used for data management and the creation of calculated variables, while baseline data summaries were produced in R 2.14.1. Predictive model building was done in STATA 14.2.
Results
In ITIP, 562 children with fast breathing were randomized to the placebo group. Of these, 27 were missing treatment outcome and were excluded from this secondary analysis. A further six were excluded because they did not meet inclusion/exclusion criteria, three others were dropped because of missing covariates (Fig 1, Table 3). Overall, 11.5% (61/526) of children treated with placebo met treatment failure or relapse criteria in this dataset. Placebo recipients with treatment failure or relapse, compared to those without treatment failure or relapse, were 5.0 months younger (17.1 vs 22.1 months, p-value = 0.008), their MUAC was 0.4 cm smaller on average (14.7 cm vs 15.1 cm, p-value = 0.009), and 11.3% more were malaria negative according to rapid antigen testing (98.4% vs 87.1%, p-value = 0.018, Table 3). In addition, there was weak evidence towards a higher proportion of children with a peripheral arterial oxyhemoglobin saturation 90–95% (6.6% vs 1.9%, p-value = 0.081), a higher average respiratory rate (51.7 vs 49.8 breaths/minute, p-value = 0.061), and a difference in categories of water source between children with and without treatment failure or relapse (p-value = 0.083).
In Table 4 we report a model that predicts the probability of treatment failure or relapse among children receiving placebo. Heart rate terms, MUAC, malaria status, water source, family income, and whether or not a sibling or other child in the household received childcare outside the home were all included in the final predictive model. Specifically, an increase in MUAC, testing malaria positive, the categories of other water source and well water, relative to the piped water referent category, and missing family income were all associated with a lower risk of treatment failure or relapse. On the other hand, increasing family income and having a sibling receive childcare outside the home were associated with a statistically significant higher risk of treatment failure or relapse. Failure rates were the highest for heart rates between 120 and 170, peaking near 145 beats per minute, and reduced for higher or lower baseline heart rates. The model equation is:
We found that this model explained just 6.1% of the variability in predicting treatment failure or relapse (R2, 0.061) and achieved a goodness of fit, as measured by the c-statistic (AUROC), of 0.712 (95% confidence interval 0.66, 0.78). Both of these model fit indicators imply modest model performance overall. Furthermore, we found that in order to correctly classify all children who developed treatment failure or relapse in this dataset, we would need to incorrectly treat 77% of those children with antibiotics who did not develop treatment failure or relapse without antibiotic treatment. While we considered applying less restrictive criteria to allow some children to develop treatment failure or relapse without antibiotics, the substantial overlap in prognostic risk scores across the two outcome groups meant–even with less restrictive criteria–the proportion of false positives would change little. A risk prediction score plot visually depicts the density of the relative overlap between children with and without treatment failure or relapse, again suggesting that the model inadequately discriminates between these two outcomes (Fig 2).
Discussion
We leveraged a unique dataset of placebo treated Malawian children in an effort to develop a novel prognostic risk score to complement current WHO guidelines in deciding which 2–59 month olds with fast breathing do or do not require antibiotics in LMICs. Given incorrect non-treatment of a bacterial infection with antibiotics can be fatal in settings where many children may only access healthcare once, a prognostic risk score to facilitate a “watch and wait” antibiotic treatment approach requires high accuracy for it to be acceptable to caregivers, healthcare providers, or policy makers in LMICs. It is against this backdrop that we found our model to be inadequate for clinical application as the model performed only modestly, and did not have sufficient discrimination between outcomes. Application of our model would result in the incorrect treatment of 77% of fast breathing children with antibiotics in order to ensure no fast breathing child who would otherwise progress to treatment failure or relapse went without treatment. In other words, use of this model would theoretically reduce the number needed to treat to prevent one treatment failure or relapse episode from 33 to 26.
Prognostic risk scores have been implemented in resource rich hospital settings to identify children at risk of death [19], and similarly designed scores are now emerging from LMICs [20]. Scores from hospitalized children have been developed using databases from Columbia, Ghana, India, Kenya, Malawi, Mexico, Pakistan, South Africa, Vietnam, and Zambia [21–25]. Only one score from South Africa has been successfully externally validated [21, 23], and only one score in Malawi has been implemented, with results suggesting a reduction in hospital mortality post-implementation [26]. Notably, the South African RISC score, which included one score for HIV-infected and another score for HIV-uninfected children with pneumonia, achieved a c-statistic of 0.77 (HIV-infected model) and 0.92 (HIV-uninfected model) during development, a c-statistic of 0.74 to 0.92 (HIV-uninfected) and 0.66 to 0.73 (HIV-infected) during internal validation, and a c-statistic of 0.72 during external validation on a Malawi dataset [21, 23]. While our prognostic risk score achieved a similar c-statistic of 0.71, identifying children at low risk for poor outcomes requires much higher discrimination to be safely implementable.
Interestingly, this same standard may not apply to adult pneumonia populations. In a large study of hospitalized adults in the United States a prediction score was developed and validated to stratify patients into categories of low or high risk of death [27]. The authors reported a c-statistic of 0.84 during score development and 0.83 during external validation, and concluded that their score accurately predicted adults with pneumonia who are at low risk of death and did not require hospitalization [27]. Despite achieving high accuracy, seven deaths were still included in the model’s low risk category [27]. Furthermore, to achieve this accuracy, the score was comprised of 20 characteristics including comprehensive medical histories, six invasive blood samples including an arterial blood gas, and chest radiography [27]. This level of diagnostic complexity is not feasible in the vast majority of LMIC primary care settings where detailed medical histories are rarely available and laboratory and radiographic access is limited. Nevertheless, given that innovative approaches to improve antibiotic stewardship in LMICs remain a top priority, prognostic risk scores to identify low risk patients that can be managed without antibiotics warrants continued exploration in similar adult populations or other pediatric pneumonia patient groups.
Children with an acute illness characterized by only fast breathing may be an especially challenging population to develop and validate an accurate prognostic risk score. Fast breathing among children under the age of five years is non-specific with many possible causes, some of which are pathologic and some of which are benign. The spectrum of more pathologic causes includes bacterial lower respiratory tract infection, which requires antibiotics, while the spectrum of more benign causes includes anxiety, which does not require antibiotics. In addition, counting respirations in a child can be a vexing task to even the most skilled clinicians, requiring one full minute of focused attention on the chest’s subtle rise and fall in an often moving, agitated, anxious, or crying child. Many caregivers are reluctant to fully expose the chest of their child and many time-constrained healthcare providers do not consistently count respirations for a full minute, both of which further reduce the reliability of respiratory rate counts [28, 29]. These issues pose substantial barriers to creating a sufficiently discriminative score from demographic variables and clinical markers. One previous study, also from Malawi, sought to develop a prognostic risk score to predict treatment failure among 2–59 month old children with fast breathing [30]. In this study trained community health workers diagnosed children with fast breathing and prescribed antibiotics [30]. While our model performed better in comparison, the previous study’s authors also concluded that their model had an unacceptably low discrimination (c-statistic 0.56) that prevented clinical implementation [30]. Taken together, these results suggest that a predictive score among a population of fast breathing children is unlikely to be sufficiently accurate without collecting invasive biospecimens or imaging.
This secondary analysis has several limitations. First, the ITIP fast breathing trial excluded most children with known risk factors for death such that the dataset may not be representative of all Malawian children with fast breathing. However, current WHO guidelines require hospitalization of high risk children with fast breathing, and this population was not our target group [2]. Second, the outcomes of treatment failure and relapse are themselves relatively non-specific for death, and many children meeting criteria for these endpoints improve without any intervention [31]. Given no placebo recipients participating in this trial died, these endpoints were the only outcomes available. Additionally, treatment failure and relapse remain clinically relevant in real-world practice as current WHO guidelines recommend treatment changes when they occur [2]. Third, treatment failure and relapse occurred relatively infrequently, and this limited our ability to evaluate all potential risk factors in our model. Additional studies from Pakistan that include placebo treatment of fast breathing children have been completed, and future data pooling for a meta-analysis may help to address this limitation [32, 33]. Lastly, since the primary aim of the project was to develop a predictive model and assess its accuracy, we did not adjust for multiple comparisons or use any other methods to reduce the risk of type I error. Indeed, the model building methods we utilized are known to result in biased point estimates and inflated p-values, so any direct interpretation of those metrics should be done with caution [34].
In conclusion, while the outcomes of this study’s placebo group shed light on the natural history of children with WHO fast breathing pneumonia in a sub-Saharan African setting, model development to accurately predict treatment failure or relapse was unsuccessful. Next steps may include data pooling with other similarly unique datasets or development of prognostic risk scores based on minimally invasive biospecimens or imaging that could be feasible for implementation in LMICs.
Acknowledgments
We thank all study participant children, caregivers, and the local community in Lilongwe, Malawi for their participation. We also thank the University of North Carolina Project, Lilongwe Medical Relief Fund Trust study staff for providing dedicated patient care, Triclinium Clinical Development for providing data management and safety monitoring, and the Malawi Ministry of Health for their support.
References
- 1. Liu L, Oza S, Hogan D, Chu Y, Perin J, Zhu J, et al. Global, regional, and national causes of under-5 mortality in 2000–15: an updated systematic analysis with implications for the Sustainable Development Goals. Lancet. 2017;388:3027–35.
- 2.
World Health Organization. Integrated Management of Childhood Illness: Chart Booklet. World Health Organization: Geneva. 2014. Available from: http://apps.who.int/iris/bitstream/handle/10665/104772/9789241506823_Chartbook_eng.pdf;jsessionid=186666A47ABF139612503B0572A61B43?sequence=16. Cited 3 September 2018.
- 3. Sazawal S, Black RE. Effect of pneumonia case management on mortality in neonates, infants, and preschool children: a meta-analysis of community-based trials. Lancet Infect Dis. 2003;3:547–56. pmid:12954560
- 4. Wahl B, O'Brien KL, Greenbaum A, Majumder A, Liu L, Chu Y, et al. Burden of Streptococcus pneumoniae and Haemophilus influenzae type b disease in children in the era of conjugate vaccines: global, regional, and national estimates for 2000–15. Lancet Glob Health. 2018;6:e744–e57. pmid:29903376
- 5. Madhi SA, Klugman KP, Vaccine Trialist G. A role for Streptococcus pneumoniae in virus-associated pneumonia. Nat Med. 2004;10:811–3. pmid:15247911
- 6. McCollum ED, Nambiar B, Deula R, Zadutsa B, Bondo A, King C, et al. Impact of the 13-Valent Pneumococcal Conjugate Vaccine on Clinical and Hypoxemic Childhood Pneumonia over Three Years in Central Malawi: An Observational Study. PLoS ONE. 2017;12:e0168209. pmid:28052071
- 7. McCollum ED, King C, Hollowell R, Zhou J, Colbourn T, Nambiar B, et al. Predictors of treatment failure for non-severe childhood pneumonia in developing countries—systematic literature review and expert survey—the first step towards a community focused mHealth risk-assessment tool? BMC Pediatr. 2015;15:74. pmid:26156710
- 8.
World Health Organization. Antimicrobial Resistance Global Report on Surveillance. World Health Organization: Geneva. 2014. Available from: http://apps.who.int/iris/bitstream/handle/10665/112642/9789241564748_eng.pdf?sequence=1. Cited 3 September 2018.
- 9. Musicha P, Cornick JE, Bar-Zeev N, French N, Masesa C, Denis B, et al. Trends in antimicrobial resistance in bloodstream infection isolates at a large urban hospital in Malawi (1998–2016): a surveillance study. Lancet Infect Dis. 2017;17:1042–52. pmid:28818544
- 10. Dramowski A, Cotton MF, Rabie H, Whitelaw A. Trends in paediatric bloodstream infections at a South African referral hospital. BMC Pediatr. 2015;15:33. pmid:25884449
- 11. Mandomando I, Sigauque B, Morais L, Espasa M, Valles X, Sacarlal J, et al. Antimicrobial drug resistance trends of bacteremia isolates in a rural hospital in southern Mozambique. Am J Trop Med Hyg. 2010;83:152–7. pmid:20595494
- 12. Ginsburg AS, Tinkham L, Riley K, Kay NA, Klugman KP, Gill CJ. Antibiotic non-susceptibility among Streptococcus pneumoniae and Haemophilus influenzae isolates identified in African cohorts: a meta-analysis of three decades of published studies. Int J Antimicrob Agents. 2013;42:482–91. pmid:24139883
- 13. McCollum ED, King C, Hammitt LL, Ginsburg AS, Colbourn T, Baqui AH, et al. Reduction of childhood pneumonia mortality in the Sustainable Development era. Lancet Resp Med. 2016;4:932–3.
- 14. Ginsburg AS, Mvalo T, Nkwopara E, McCollum ED, Ndamala CB, Schmicker R, et al. Placebo versus amoxicillin for non-severe fast-breathing pneumonia in Malawian children aged 2–59 months: a double-blind, randomized controlled non-inferiority trial. JAMA Pediatr. 2018; In press.
- 15. Cheung YB. A modified least-squares regression approach to the estimation of risk difference. Am J Epidemiol. 2007;166:1337–44. pmid:18000021
- 16. Hocking RR. The Analysis and Selection of Variables in Linear Regression. Biometrics. 1976;32:1–49.
- 17. Steyerberg EW, Eijkemans MJ, Habbema JD. Stepwise Selection in Small Data Sets: A Simulation Study of Bias in Logistic Regression Analysis. J Clin Epidemiol. 1999;52:935–42. pmid:10513756
- 18. Akaike H. A new look at the statistical model identification. IEEE Trans Automat Contr. 1974;19:716–23.
- 19. Duncan H, Hutchison J, Parshuram CS. The Pediatric Early Warning System score: a severity of illness score to predict urgent medical need in hospitalized children. J Crit Care. 2006;21:271–8. pmid:16990097
- 20. Deardorff KV, McCollum ED, Ginsburg AS. Pneumonia Risk Stratification Scores for Children in Low-Resource Settings: A Systematic Literature Review. Pediatr Infect Dis J. 2018;37:743–8. pmid:29278608
- 21. Reed C, Madhi SA, Klugman KP, Kuwanda L, Ortiz JR, Finelli L, et al. Development of the Respiratory Index of Severity in Children (RISC) score among young children with respiratory infections in South Africa. PLoS ONE. 2012;7:e27793. pmid:22238570
- 22. Emukule GO, McMorrow M, Ulloa C, Khagayi S, Njuguna HN, Burton D, et al. Predicting mortality among hospitalized children with respiratory illness in Western Kenya, 2009–2012. PLoS ONE. 2014;9:e92968. pmid:24667695
- 23. Hooli S, Colbourn T, Lufesi N, Costello A, Nambiar B, Thammasitboon S, et al. Predicting Hospitalised Paediatric Pneumonia Mortality Risk: An External Validation of RISC and mRISC, and Local Tool Development (RISC-Malawi) from Malawi. PLoS ONE. 2016;11:e0168126. pmid:28030608
- 24. Mamtani M, Patel A, Hibberd PL, Tuan TA, Jeena P, Chisaka N, et al. A clinical tool to predict failed response to therapy in children with severe pneumonia. Pediatr Pulmonol. 2009;44:379–86. pmid:19330771
- 25. Olson D, Davis N, Milazi R, Lufesi N, Miller W, Preidis G, et al. Development of a severity of illness scoring system (inpatient triage, assessment and treatment) for resource-constrained hospitals in developing countries. Trop Med Int Health. 2013;18:871–8. pmid:23758198
- 26. Olson D, Preidis G, Milazi R, Spinler J, Lufesi N, Mwansambo C, et al. Task shifting an inpatient triage, assessment and treatment programme improves the quality of care for hospitalised Malawian children. Trop Med Int Health. 2013;18:879–86. pmid:23600592
- 27. Fine MJ, Auble TE, Yealy DM, Hanusa BH, Weissfeld LA, Singer DE, et al. A prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med. 1997;336:243–50. pmid:8995086
- 28. Edward A, Dam K, Chege J, Ghee AE, Zare H, Chhorvann C. Measuring pediatric quality of care in rural clinics-a multi-country assessment-Cambodia, Guatemala, Zambia and Kenya. Int J Qual Health Care. 2016;28:586–93. pmid:27488477
- 29. Bjornstad E, Preidis G, Lufesi N, Olson D, Kamthunzi P, Hosseinipour M, et al. Determining the quality of IMCI pneumonia care in Malawian children. Paediatr Int Child Health. 2014;34:29–36. pmid:24091151
- 30. King C, McCollum ED, Mankhambo L, Colbourn T, Beard J, Hay Burgess DC, et al. Can We Predict Oral Antibiotic Treatment Failure in Children with Fast-Breathing Pneumonia Managed at the Community Level? A Prospective Cohort Study in Malawi. PLoS ONE. 2015;10:e0136839. pmid:26313752
- 31. Hazir T, Qazi SA, Nisar YB, Maqbool S, Asghar R, Iqbal I, et al. Can WHO therapy failure criteria for non-severe pneumonia be improved in children aged 2–59 months? Int J Tuberc Lung Dis. 2006;10:924–31. pmid:16898379
- 32. Hazir T, Nisar YB, Abbasi S, Ashraf YP, Khurshid J, Tariq P, et al. Comparison of oral amoxicillin with placebo for the treatment of world health organization-defined nonsevere pneumonia in children aged 2–59 months: a multicenter, double-blind, randomized, placebo-controlled trial in pakistan. Clin Infect Dis. 2011;52:293–300. pmid:21189270
- 33. Jehan F, Nisar MI, Kerai S, Brown N, Balouch B, Hyder Z, et al. A double blind community-based randomized trial of amoxicillin versus placebo for fast breathing pneumonia in children aged 2–59 months in Karachi, Pakistan (RETAPP). BMC Infect Dis. 2016;16:13. pmid:26758747
- 34. Hurvich CM, Chih-Ling T. The Impact of Model Selection on Inference in Linear Regression. Am Stat. 1990;44:214–7.