http://orcid.org/0000-0002-9546-5788
Figures
Abstract
Background
Trauma is predicted to become the third leading cause of death in India by 2020, which indicate the need for urgent action. Trauma scores such as the international classification of diseases injury severity score (ICISS) have been used with great success in trauma research and in quality programmes to improve trauma care. To this date no valid trauma score has been developed for the Indian population.
Study design
This retrospective cohort study used a dataset of 16047 trauma-patients from four public university hospitals in urban India, which was divided into derivation and validation subsets. All injuries in the dataset were assigned an international classification of disease (ICD) code. Survival Risk Ratios (SRRs), for mortality within 24 hours and 30 days were then calculated for each ICD-code and used to calculate the corresponding ICISS. Score performance was measured using discrimination by calculating the area under the receiver operating characteristics curve (AUROCC) and calibration by calculating the calibration slope and intercept to plot a calibration curve.
Results
Predictions of 30-day mortality showed an AUROCC of 0.618, calibration slope of 0.269 and calibration intercept of 0.071. Estimates of 24-hour mortality consistently showed low AUROCCs and negative calibration slopes.
Conclusions
We attempted to derive and validate a version of the ICISS using SRRs calculated from an Indian population. However, the developed ICISS-scores overestimate mortality and implementing these scores in clinical or policy contexts is not recommended. This study, as well as previous reports, suggest that other scoring systems might be better suited for India and other Low- and middle-income countries until more data are available.
Citation: Attergrim J, Sterner M, Claeson A, Dharap S, Gupta A, Khajanchi M, et al. (2018) Predicting mortality with the international classification of disease injury severity score using survival risk ratios derived from an Indian trauma population: A cohort study. PLoS ONE 13(6): e0199754. https://doi.org/10.1371/journal.pone.0199754
Editor: Erin E. Fox, University of Texas Health Science Center at Houston, UNITED STATES
Received: August 17, 2017; Accepted: June 13, 2018; Published: June 27, 2018
Copyright: © 2018 Attergrim et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All datafiles are available from the figshare databese, DOI number: 10.6084/m9.figshare.5318218 and available from URL: https://figshare.com/s/984db82958a69e7d24d4.
Funding: The work was supported by the following: Swedish National Board of Health and Welfare - Block grant on research to improve trauma care outcomes; Swedish International Development Cooperation Agency - Field research grant (JKAA); Laerdal Foundation for Acute Medicine - For data collection.
Competing interests: Mr. Attergrim reports grants from Swedish National Board of Health and Welfare, grants from Swedish International Development Cooperation Agency, from Laerdal Foundation for Acute Medicine, during the conduct of the study. Dr. Gupta reports grants from Tata Institute of Social Sciences as Grants for manpower and contingencies for the collection of data from his Institution. Dr. Gerdin reports grants from Karolinska Institutet, during the conduct of the study.
Abbreviations: AUROCC, Area under the receiver operating characteristics curve; CI, Confidence interval; CLI, Calibration line intercept; CSL, Calibration line slope; ICD, International Classification of Diseases; ICISS, International Classification of Disease Injury Severity Score; LMICS, low- and middle-income; SD, Standard deviation; SRR, Survival risk ratios; TITCO, Towards Improved Trauma Care Outcomes
Introduction
Trauma was estimated to cause 4.8 million deaths in 2013, which is greater than the deaths caused by HIV/AIDS, tuberculosis, malaria and maternal conditions combined [1]. Ninety per cent of these deaths have been estimated to occur in low- and middle-income countries (LMICs) and an estimated 2 million lives could be saved annually by improving the quality of trauma care in LMICs [2,3].
India is an LMIC with more than 1 million annual trauma deaths, and by 2020, trauma is predicted to become the country’s third leading cause of death[1,4]. The majority of trauma victims are young and in their most productive phases of life, creating both physiological and psychological suffering as well as an economic drain [4–7]. Hence, efforts to strengthen trauma care and prevention in India are urgently needed [8].
Trauma patients constitute a heterogeneous population, making trauma research and outcome comparisons over time and between contexts challenging [9]. One crucial factor to consider in such comparisons is injury severity, and several scores have been developed for this purpose [7,10–13]. The use of these scores as part of quality improvement programmes has been associated with improved trauma care[14], but no optimal score exists for the Indian population [15,16].
The international classification of disease (ICD) injury severity score (ICISS) is a tool used to determine injury severity [12,17]. This score uses survival risk ratios (SRRs), empirically derived for each unique ICD code, to estimate a patient’s probability of survival. According to a recent systematic review, the ICISS outperforms many other scores [18], but little research on the ICISS is from LMICs. Since the ICISS is derived from a specific context [12], the current study sought to identify whether an Indian version of the ICISS could predict mortality in four public hospitals in urban India.
Methods
Study design
We analysed data from a previously conducted cohort study to derive and temporally validate a new version of the ICISS. This study was registered with clinicaltrials.gov with the registration number NCT02716649.
Setting
We used data from the Towards Improved Trauma Care Outcomes in India (TITCO) project that were collected at four public university hospitals across urban India. The data used in this study were collected from the four study centres between July 2013 and December 2015. The four centres included Lokmanya Tilak Municipal General Hospital in Mumbai, King Edward Memorial Hospital in Mumbai, Jai Prakash Narayan Apex Trauma Center in Delhi, and the Institute of Post-Graduate Medical Education and Research and Seth Sukhlal Karnani Memorial Hospital in Kolkata.
At each hospital, one trained project officer with a health master’s degree or higher level of education collected all the data. The project officers worked eight hours a day rotating between day, evening and night shifts, and were continuously supervised and trained by project management. Patients were followed up until discharge, death or 30 days.
Participants and eligibility criteria
Patients with a history of trauma who arrived alive to the study hospitals and were admitted or died between arrival and admission were included. Patients with isolated limb fractures without vascular injury were excluded because they are placed in the “orthopaedic pathway” instead of the “trauma pathway” in the study hospitals, which made patient follow-up was impossible for these individuals.
The project officers included all consecutive patients who fit the eligibility criteria. Data for the patients admitted during the project officers’ shifts were collected using direct observation and extraction from patient records. Data for patients admitted outside of their shifts were collected retrospectively from patient records within days of patient arrival.
Outcomes variables and covariates
The primary and secondary outcomes were in hospital death at any of the hospitals participating in this study within 30 days and 24 hours respectively. The explanatory variable was the ICISS. Other covariates included the date and time of arrival, age, sex, mechanism of injury and transfer status. All quantitative variables were analysed as continuous. We used these variables to characterize the study sample.
Data sources/measurements
Free-text injury descriptions were extracted from patient records, including imaging reports and surgical notes, and were then coded using the ICD-10. We calculated the SRR for each unique ICD-10 code using , where A denotes the number of surviving patients and B denotes the number of non-surviving patients with the same specific ICD-code. The calculated SRR wasassigned a value between 0 and 1, where 1 represents 100% survival and 0 represents 0% survival. The final ICISS score for each patient was calculated as the product of all individual SRRs. Hence, the ICISS also ranges from 0 to 1 and should be interpreted as the patient’s probability of survival. This method is commonly referred to as the conventional or multiplicative ICISS.
Bias
During the conversion from free-text injury descriptions to ICD codes, the coders were blinded to patient demographics and outcomes. ICD coding was conducted after the coders completed the World Health Organization (WHO) ICD-10 online training module and achieved more than 80% agreement with an external experienced coder over several samples of 50 injuries.
Study size
We used all available data from TITCO and created a temporally split sample, henceforth referred to as the derivation sample and validation sample, using the earlier data for derivation and the most recent data for validation. The validation sample size was estimated by including the most recent 200 consecutive events, i.e. patients who died within 24 hours, and all non-events enrolled during the same time period [19,20]. We used mortality within 24 hours for our sample size calculation because we wanted the study to be powered for secondary outcomes as well. This effective sample size allowed us to detect a significant difference in the discrimination and calibration of the ICISS between the derivation and calibration samples at 80% power and a 5% significance level. All remaining patients constituted the derivation sample.
Statistical methods and analyses
The derivation and validation of the ICISS were conducted as two separate steps using R for all statistical analyses [21]. We assessed predictive performance in terms of discrimination, by calculating the area under the receiver operating characteristics curve (AUROCC) and calibration by visually comparing the observed and predicted outcomes in a calibration plot and calculating the calibration slope and intercept. Confidence intervals for predictive performance measures were estimated using a bootstrap approach [22].
We interpreted overlapping 95% confidence intervals (CI) as evidence of lack of a statistically significant difference. Parametric and non-parametric exact tests were used as appropriate, with a 5% significance level. Our main analysis was a complete case analysis, in which we excluded observations with missing values in any of the outcomes or covariates. Observations with no injuries reported were assigned an ICISS of 1, and for each observation, the final ICISS was calculated based only on the SRRs for ICD codes that occurred in at least 10 observations in the derivation sample. Ten was chosen as a compromise between precision and the available ICD-codes.
Derivation and validation
We derived SRRs in the derivation sample for each of the outcomes, mortality within 30 days and within 24 hours, and used these ratios to calculate two ICISSs for each patient. The SRRs for 30-day mortality are referred to as SRRm30d, and the SRRs for 24-hour mortality are referred to as SRRm24h. We used similar notation to refer to the ICISS, i.e., ICISSm30d and ICISSm24h. Finally, we assessed the performance of ICISSm30d in predicting mortality within 30 days and within 24 hours and repeated this analysis for ICISSm24h. We used the SRRm30d and SRRm24h results from the derivation sample to calculate ICISSm30d and ICISSm24h in the validation sample. Performance was assessed in the same manner as in the derivation sample, and the results were compared to those from the derivation sample.
Sensitivity analyses
We conducted four sensitivity analyses. The first analysis included observations with missing values in covariates but with complete outcome data. The second excluded observations without any reported injury, which were previously assigned an ICISS value of 1 regardless of outcome. The third calculated ICISS was based on all available calculated SRRs, regardless of how frequently the corresponding ICD codes occurred in the dataset. Finally, we calculated the ICISS for each patient based only on unique ICD codes, in other words, each ICD code was allowed to contribute only one SRR to the ICISS even if it occurred more than once in the same patient.
Ethical considerations
Ethics committees at all participating centres approved the collation of the database and granted a waiver of consent for the trauma patients. This study was conducted using anonymized data. The names of the review boards and ethical approval reference numbers for the original study that collected the data were as follows: Seth GS Medical College and King Edward Memorial Hospital Institutional Ethics Committee, approval number IEC(I)/OUT/222/14; Lokmanya Tilak Municipal Medical College & Lokmanya Tilak Municipal General Hospital Institutional Ethics Committee, approval number IEC/11/13; Institute of Post-Graduate Medical Education and Research Institutional Ethics Committee, approval number IEC/279; and All India Institute of Medical Sciences Institutional Ethics Committee, approval number IEC/NP-279/2013 RP-Ol/2013.
Results
Participants
Between July 2013 and December 2015, a total of 16047 patients were included in the TITCO dataset. The complete case analysis excluded patients whose records had missing values in covariates or outcomes, leaving 15,865 patients for the analysis.
Descriptive data
The complete case sample (n = 15,865) was split into a derivation sample (n = 11,944) and a validation sample (n = 3,921). Age and gender showed the lowest numbers of missing values. Date of arrival to the hospital and mechanism of injury showed the highest number of missing values with 95 (0.6%) and 60 (0.4%) missing values, respectively (Table 1).
The mean age was 31.6 (SD 18.9, range 0–97) years in the derivation sample and 33.3 (SD 18.7, range 0–95) years in the validation sample. The mean age for non-survivors was higher than that for survivors, at 37 (SD 18.9, range 1–97) and 38 years, respectively. The majority of patients were male (73.8% and 79.3% in the derivation and validation samples, respectively). Road traffic injury (41% and 46.9%) was the most common cause of trauma followed by falls (33% and 31.2%). The majority (70.4% and 73.3%) of patients were transferred from another hospital (Table 2).
Outcome data
Out of the 15,865 patients in the complete case analysis, 3,459 (21.6%) patients died within 30 days and 1,025 (6.4%) died within 24 hours. Mortality was higher in the derivation sample (22.3%, n = 2675 for 30-day mortality; 6.9%, n = 825 for 24-hour mortality) than in the validation sample (20.0%, n = 784 for 30-day mortality and 5.1%, n = 200 for 24-hour mortality).
Main results
The different combinations of ICISSm24h and ICISSm30d with m30d and m24h were assessed using calibration and discrimination. For each measure, a 95% confidence interval was generated using a bootstrap approach with 1000 draws with replacement of the same size as the original sample (Table 3).
The AUROCC of ICISSm30d in predicting 30-day mortality in the derivation sample was 0.633 (95% CI 0.620–0.646), and the calibration slope was 0.300 (95% CI 0.271–0.333), suggesting that ICISSm30d substantially overestimated the risk of mortality (Fig 1). The calibration slope in the derivation sample for ICISSm24h on 24-hour mortality was 0.079 (95% CI 0.041–0.12) with an AUROCC of 0.519 (95% CI 0.496–0.541), indicating a prediction value similar to chance alone. The results from the derivation sample were largely mimicked in the validation sample.
Calibration plots were created in which the ICISS scores were split into deciles and plotted against mortality (Fig 2). The main analysis was conducted in the validation sample for ICISSm30d with m30d and ICISSm24h with m24h. Fig 2A shows a linear association between ICISSm30d and mortality, and the point estimates are relatively close to the solid line. The calibration slopes capture the overall trend but with substantial overestimation of mortality. The point estimates in Fig 2B are linear, but the calibration slope suggests no or even a slight inverse relationship.
Plots for the validation sample are shown for (A) within 30 days and (B) within 24 hours. Abbreviations: CSL: Calibration line slope, CLI: Calibration line intercept.
Discussion
Key results
Our aim was to derive and validate two versions of the ICISS for 30-day mortality (ICISSm30d) and 24-hour mortality (ICISSm24h) using SRRs derived from trauma patients admitted to four public university hospitals in urban India. Our results indicate that neither of the developed scores performs adequately in terms of discrimination and calibration when used to predict death at eather 30 days or 24 hours. To illustrate this point, the best discriminating score in the validation sample was for ICISSm30d when used to predict death at 30 days, and this score achieved an AUROCC of only 0.618, with accompanying poor estimates of calibration.
Limitations
This study was based on the Largest Indian trauma dataset available, but the sample size of 16,000 patients is still small compared with similar studies from other contexts in which more than 100,000 patients were used [23,24]. The comparably small sample size may have resulted in more unstable SRRs. To address this limitation, we used SRRs from injuries that occurred 10 or more times, but this cut-off should be increased in future studies when more data are available.
The clinical environment studied comes with limitations including a lack of diagnostic tools, which is likely to result in undocumented injuries[16]. As previous literature suggests, this limitation may result in lower SRRs for minor injuries due to the presence of more severe, co-existing undocumented injuries, which would in turn translate into an overestimation of mortality, especially for ICISSm24h where missed diagnoses are common [23].
We studied only the conventional ICISS in which all of a patient’s injuries are accounted for in the final score. The study by Kilgo PD et al. evaluated methods using a maximum amount of injuries or just the worst injury [24], and the methods using just the worst injury outperformed models using multiple or all injuries, suggesting that the model in this study, the conventional ICISS, was not the optimal model [24,25].
Interpretation
Our study demonstrates the challenges associated with deriving local SRRs using a relatively limited sample. It is notable however, that this dataset is to our knowledge the largest available trauma dataset from India, the world’s second most populous country. It is unlikely that much larger datasets exist from many other LMICs; thus deriving local SRRs in similar contexts may be equally challenging.
Recent research evaluated the performance of five different trauma scores in part of our dataset [16]. The scores evaluated were the injury severity score (ISS), the new injury severity score (NISS), the revised trauma score (RTS), the Kampala trauma score (KTS), and the trauma injury severity score (TRISS). Like the ICISS, the ISS and NISS are purely anatomical scores, whereas the RTS is a physiological score, and the KTS and TRISS are combined scores, i.e., including both anatomical and physiological components. In that study, TRISS discriminated best with an AUROCC of 0.82 and was closely followed by RTS at 0.81. The former score is a composite of the RTS, ISS, age and blunt or penetrating injury, indicating that the addition of ISS, age and injury type results in a negligible improvement in discrimination. These results suggest that physiological scores such as the RTS would be preferable in LMICs, provided issues with vital sign registration frequency are corrected.
Further, research on an earlier dataset from Mumbai showed that physiological scores outperformed anatomical scores [15]. The ISS was the worst discriminating score, with an AUROCC of 0.69, i.e., similar to that of the ICISS in our study. It is likely that undocumented injuries and incomplete imaging contributed to the poor performance of the ISS in that study and in our study. Interestingly however, even in more complete datasets, such as that of San Francisco General Hospital, the discrimination of physiological scores approaches that of combined scores [26], indicating that the former may be a more stable choice.
Generalizability
Transfer of ICISS scores between countries and contexts has been attempted in high-income countries with mixed results [27]. Pooling data from different countries to create a multinational ICISS also reduced performance [28]. Thus, the SRRs derived in our study are unlikely to transfer well to other settings, even if they had displayed better performance in our data. Instead, our research highlights the importance of validating an ICISS generated using locally derived SRRs in an independent sample before applying it to practice. As we show, using the SRRs derived here would result in a serious overestimation of mortality.
Conclusion
The scores developed in this study systematically overestimated 30-day mortality, and prediction of 24-hour mortality suggests that an ICISS based on the SRRs derived in this study should not be used to predict short-term mortality at all. Due to poor discrimination and calibration, the current models cannot be considered a reliable choice of trauma score in the studied context. Our study highlights the challenges associated with deriving local SRRs based on limited data, and the findings suggest that other trauma scores should be considered in settings where only small samples are available to calculate SRRs. Other LMICs face similar challenges as India, suggesting that resources and research might be better directed towards other scoring systems than the ICISS.
Supporting information
S1 Table. Sensitivity analysis I.
Including patients with missing covariates. ICISS: International classification of disease injury severity score, AUROCC: Area under the receiver operating characteristic curve, m30d: Mortality within 30 days, m24h: Mortality within 24 hours.
https://doi.org/10.1371/journal.pone.0199754.s001
(DOC)
S2 Table. Sensitivity analysis II.
Excluding patients without observed injuries. ICISS: International classification of disease injury severity score, AUROCC: Area under the receiver operating characteristic curve, m30d: Mortality within 30 days, m24h: Mortality within 24 hours.
https://doi.org/10.1371/journal.pone.0199754.s002
(DOC)
S3 Table. Sensitivity analysis III.
The ICISS was based on all SRRs independent of how many patients they were based on. ICISS: International classification of disease injury severity score, AUROCC: Area under the receiver operating characteristic curve, m30d: Mortality within 30 days, m24h: Mortality within 24 hours.
https://doi.org/10.1371/journal.pone.0199754.s003
(DOC)
S4 Table. Sensitivity analysis IV.
A given ICD code could contribute only once to the patient’s final ICISS score. ICISS: International classification of disease injury severity score, AUROCC: Area under the receiver operating characteristic curve, m30d: Mortality within 30 days, m24h: Mortality within 24 hours.
https://doi.org/10.1371/journal.pone.0199754.s004
(DOC)
Acknowledgments
We thank the Mumbai-based multidisciplinary research group known as the Thursday Truth Seekers for extensive comments and input.
References
- 1. Vos T, Barber RM, Bell B, Bertozzi-Villa A, Biryukov S, Bolliger I, et al. Global, regional, and national incidence, prevalence, and years lived with disability for 301 acute and chronic diseases and injuries in 188 countries, 1990–2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet. 2015;386: 743–800. pmid:26063472
- 2. Chandran A, Hyder AA, Peek-Asa C. The global burden of unintentional injuries and an agenda for progress. Epidemiol Rev. 2010;32: 110–20. pmid:20570956
- 3. Mock C, Joshipura M, Arreola-Risa C, Quansah R. An estimate of the Number of Lives that Could be Saved through Improvements in Trauma Care Globally. World J Surg. 2012;36: 959–963. pmid:22419411
- 4. Joshipura MK. Trauma care in India: current scenario. World J Surg. 2008;32: 1613–7. pmid:18553048
- 5. Goyal A, Yusuf S. Burden of disease in the India, NCMH. Indian J Med Res. 2006;124: 235–244. 2005
- 6.
Who. Injuries and Violence: The Facts. Geneva, Switz WHO. 2010; 2–18.
- 7. Baker SP, O’Neill B, Haddon W, Long WB. The injury severity score: a method for describing patients with multiple injuries and evaluating emergency care. J Trauma. 1974;14: 187–96. pmid:4814394
- 8. Hsiao M, Malhotra A, Thakur JS, Sheth JK, Nathens AB, Dhingra N, et al. Road traffic injury mortality and its mechanisms in India: nationally representative mortality survey of 1.1 million homes. BMJ Open. 2013;3: e002621. pmid:23959748
- 9. Rutledge R. The goals, development, and use of trauma registries and trauma data sources in decision making in injury. Surg Clin North Am. 1995;75: 305–26. pmid:7900000
- 10. Newgard CD, Fildes JJ, Wu L, Hemmila MR, Burd RS, Neal M, et al. Methodology and analytic rationale for the American College of Surgeons Trauma Quality Improvement Program. J Am Coll Surg. 2013;216: 147–57. pmid:23062519
- 11. Krumholz HM. Mathematical Models and the Assessment of Performance in Cardiology. Circulation. 1999;99: 2067–2069. pmid:10217642
- 12. Rutledge R, Osler T, Emery S, Kromhout-Schiro S. The end of the Injury Severity Score (ISS) and the Trauma and Injury Severity Score (TRISS): ICISS, an International Classification of Diseases, ninth revision-based prediction tool, outperforms both ISS and TRISS as predictors of trauma patient survival,. J Trauma. 1998;44: 41–9. pmid:9464748
- 13. Rutledge T, Deis R, Bedrick JRN. ICISS: An International Classification of Disease—9 Based Injury Severity Score. 2016;41: 1–12.
- 14. Glance LG, Osler T. Beyond the major trauma outcome study: benchmarking performance using a national contemporary, population-based trauma registry. J Trauma. 2001;51: 725–7. pmid:11586166
- 15. Laytin AD, Kumar V, Juillard CJ, Sarang B, Lashoher A, Roy N, et al. Choice of injury scoring system in low- and middle-income countries: Lessons from Mumbai. Injury. Elsevier Ltd; 2015;46: 2491–2497. pmid:26233630
- 16. Roy N, Gerdin M, Schneider E, Kizhakke Veetil DK, Khajanchi M, Kumar V, et al. Validation of international trauma scoring systems in urban trauma centres in India. Injury. 2016;47: 2459–2464. pmid:27667119
- 17. Meredith JW, Evans G, Kilgo PD, MacKenzie E, Osler T, McGwin G, et al. A comparison of the abilities of nine scoring algorithms in predicting mortality. J Trauma. 2002;53: 621-8-9. pmid:12394857
- 18. Gagné M, Moore L, Beaudoin C, Batomen Kuimi BL, Sirois M-J. Performance of ICD-based injury severity measures used to predict in-hospital mortality: a systematic review and meta-analysis. J Trauma Acute Care Surg. 2015; pmid:26713976
- 19. Vergouwe Y, Steyerberg EW, Eijkemans MJC, Habbema JDF. Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. 2005;58: 475–483. pmid:15845334
- 20. Steyerberg EW, Borsboom GJJM, Van Houwelingen HC. Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. 2004;2586: 2567–2586. pmid:15287085
- 21.
R Core Team. R. R: A language and environment for statistical computing. Vienna, Austria.: R Foundation for Statistical Computing; 2015.
- 22. Margolis DJ, Bilker W, Boston R, Localio R, Berlin JA. Statistical characteristics of area under the receiver operating characteristic curve for a simple prognostic model using traditional and bootstrapped approaches. 2002;55: 518–524.
- 23. Meredith JW, Kilgo PD, Osler TM. Independently derived survival risk ratios yield better estimates of survival than traditional survival risk ratios when using the ICISS. J Trauma. 2003;55: 933–8. pmid:14608168
- 24. Kilgo PD, Osler TM, Meredith W. The worst injury predicts mortality outcome the best: rethinking the role of multiple injuries in trauma outcome scoring. J Trauma. 2003;55: 599–606–7. pmid:14566109
- 25. Gagné M, Moore L, Sirois M-J, Simard M, Beaudoin C, Kuimi BLB. Performance of International Classification of Diseases–based injury severity measures used to predict in-hospital mortality and intensive care admission among traumatic brain-injured patients. J Trauma Acute Care Surg. 2017;82: 374–382. pmid:28107311
- 26. Laytin AD, Dicker RA, Gerdin M, Roy N, Sarang B, Kumar V, et al. Comparing traditional and novel injury scoring systems in a US level-I trauma center: an opportunity for improved injury surveillance in low- and middle-income countries. J Surg Res. 2017;215: 60–66. pmid:28688663
- 27. Stephenson S, Henley G, Harrison JE, Langley JD. Diagnosis based injury severity scaling: investigation of a method using Australian and New Zealand hospitalisations. Inj Prev. 2004;10: 379–83. pmid:15583261
- 28. Gedeborg R, Warner M, Chen L-H, Gulliver P, Cryer C, Robitaille Y, et al. Internationally comparable diagnosis-specific survival probabilities for calculation of the ICD-10-based Injury Severity Score. J Trauma Acute Care Surg. 2014;76: 358–65. pmid:24398769