https://orcid.org/0000-0003-1902-4734
Figures
Abstract
Introduction
The closest emergency department (ED) may not always be the optimal hospital for certain stable high acuity patients if further distanced ED’s can provide specialized care or are less overcrowded. Machine learning (ML) predictions may support paramedic decision-making to transport a subgroup of emergent patients to a more suitable, albeit more distanced, ED if hospital admission is unlikely. We examined whether characteristics known to paramedics in the prehospital setting were predictive of hospital admission in emergent acuity patients.
Materials and methods
We conducted a population-level cohort study using four ML algorithms to analyze ED visits of the National Ambulatory Care Reporting System from January 1, 2018 to December 31, 2019 in Ontario, Canada. We included all adult patients (≥18 years) transported to the ED by paramedics with an emergent Canadian Triage Acuity Scale score. We included eight characteristic classes as model predictors that are recorded at ED triage. All ML algorithms were trained and assessed using 10-fold cross-validation to predict hospital admission from the ED. Predictive model performance was determined using the area under curve (AUC) with 95% confidence intervals and probabilistic accuracy using the Brier Scaled score. Variable importance scores were computed to determine the top 10 predictors of hospital admission.
Results
All machine learning algorithms demonstrated acceptable accuracy in predicting hospital admission (AUC 0.77–0.78, Brier Scaled 0.22–0.24). The characteristics most predictive of admission were age between 65 to 105 years, referral source from a residential care facility, presenting with a respiratory complaint, and receiving home care.
Discussion
Hospital admission was accurately predicted based on patient characteristics known prehospital to paramedics prior to arrival. Our results support consideration of policy modification to permit certain emergent acuity patients to be transported to a further distanced ED. Additionally, this study demonstrates the utility of ML in paramedic and prehospital research.
Citation: Strum RP, Mowbray FI, Zargoush M, Jones AP (2023) Prehospital prediction of hospital admission for emergent acuity patients transported by paramedics: A population-based cohort study using machine learning. PLoS ONE 18(8): e0289429. https://doi.org/10.1371/journal.pone.0289429
Editor: Binod Acharya, Drexel University, UNITED STATES
Received: March 1, 2023; Accepted: July 18, 2023; Published: August 24, 2023
Copyright: © 2023 Strum et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The minimal dataset cannot be published due to the legal health data restrictions in Ontario, Canada. ICES' Privacy and Legal Office authorizes the use of these data for scientific research following our approved application. The use of ICES' secondary and anonymized data is exempt from review by an ethics review board, as ICES is a prescribed entity of the Personal Health Information Protection Act in Ontario. The minimum dataset can be provided with an approved application to ICES directly at das@ices.on.ca.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Canadian paramedics utilize the Canadian Triage and Acuity Scale (CTAS) to classify medical acuity and patient care needs in the prehospital setting [1, 2]. Health policy mandates paramedics transport patients with an emergent acuity CTAS score to the closest emergency department (ED) for time-sensitive emergency medicine [1]. However, the majority of emergent acuity patients do not receive emergency medicine, experience clinically deteriorate or need hospital admission (56.6%); instead, a significant proportion receive physician care resembling primary care (39.3%) [3–5]. The exclusive transport of all emergent acuity patients to the closest ED represents an operational inefficiency. All emergent acuity patients are handled similarly in the prehospital field with transport to the closest ED, regardless of patient repatriation consideration for continuity of care, patient preference, ED workload, or bed availability at further distanced hospitals [1]. In circumstances where emergent acuity patients do not require hospital admission or time-sensitive care, transport to a further distanced ED could beneficial for both the patient and Eds [6]. Such an approach could be similar to the handling of urgent and low acuity patients who may be transported to any of the three nearest hospitals [6].
Knowing which prehospital patient characteristics predict ED admission could inform policy modifications and guide paramedics to transport emergent acuity patients to the closest ED, rather than consider further distanced EDs. Utilization of machine learning (ML) classifications algorithms could identify patient characteristics that are predictive of hospital admission among emergent acuity patients, thereby aiding in the determination of which patients are unsuitable for bypassing the closest ED. ML has the potential to improve care efficiency in patient distribution, and support paramedic decision-making when clinical information is limited in the prehospital field [7, 8].
Our primary objective was to utilize ML algorithms to predict hospital admission for adult patients arriving at the ED through paramedic transport with an emergent acuity triage score. Our secondary objective was to evaluate the predictive utility of patient characteristics known to paramedics in the prehospital setting as predictors of ED outcomes in ML.
Material and methods
Study design
We conducted a population-level cohort study using administrative ED patient records to analyze model performance of four machine learning algorithms. Patient medical records completed by paramedics in the prehospital field are not available for research purposes in Ontario; we approximated characteristics that would be known to paramedics prior to hospital arrival by using ED data collected upon entry to the department at nurse triage. The STROBE statement was followed for the reporting of results (S1 Table) [9].
Population
All Ontario adult patients (≥18 years) transported to the ED by ground paramedics after use of the emergency alert system (9-1-1) were eligible for inclusion. We included high acuity patients with an ‘emergent’ CTAS triage score (two). High acuity patients are classified as having a CTAS score of one and two, of which patients with a CTAS one score could not be considered for any plausible hospital bypass due to their life-threatening medical severity, excluding them. CTAS is an ordinal scale that ranges from one to five, with a score of one indicating the highest acuity (resuscitation) and five the lowest acuity (non-urgent) [2]. We excluded ED visits transported by air paramedic medical services, and visits with an outcome other than hospital admission or ED discharge due to data availability. All ED visits between January 1, 2018 to December 31, 2019 that met the inclusion criteria were included, as this timeframe represents the most recently available two-year period prior to the Covid-19 pandemic when paramedic and ED utilization trends changed [10].
Data source
We used population-level administrative ED patient data from the National Ambulatory Care Reporting System (NACRS) database. NACRS is a hospital and community-based ambulatory care administrative database that collects patient and visit-related data at ED triage and the time care is provided [11, 12]. All Ontario hospitals are required to submit electronic patient abstracts from the ED to comply with reporting and quality measures and these data are recorded in NACRS. NACRS data was accessed through the ICES (formerly known as the Institute for Clinical Evaluative Sciences); ICES is a non-profit, independent corporation that supports the study of health service and population-wide outcomes in Ontario. All data were provided in November 2021, and accessed through ICES’s secure server IDAVE.
Ethics
ICES’ collection and use of NACRS secondary ambulatory data are authorized under Section 45 of Ontario’s Personal Health Information Protection Act (PHIPA) as a prescribed entity, which is exempt from review by a Research Ethics Board [13, 14]. The use of the data in this study is authorized and approved by ICES’ Privacy and Legal Office, which does not require participant consent according to national regulations in Ontario, Canada [13]. All data were anonymized prior to access for this research.
Predictor eligibility and outcome variables
To be pragmatic, we extracted all patient characteristics collected at ED triage from the NACRS database for eligibility in the study. These characteristics constitute a comprehensive representation of patients, their conditions, rationale for visit prior to ED arrival, and would be known to paramedics in the prehospital setting (paramedics routinely provide this data to ED triage). We used hospital admission from the ED as the outcome, defined as a binary variable where a patient is hospitalized to receive further clinical treatment and assessment [15].
Predictor selection and data preparation
We selected eight characteristic classes for inclusion in the study based on data availability and completeness in NACRS, previous literature and clinical importance determined a priori by emergency physicians at our local academic hospital centre. Selected characteristic classes were termed the predictors in the study. Missing data for each included characteristic were scant (≤0.5%), and directly stated where applicable; complete case analysis was utilized. We dummy-coded categorical predictor variables for non-tree-based algorithm models [16].
All predictors were measured and recorded upon entry to the ED at triage, and prior to any ED diagnostic tools being employed. Patient age was originally extracted as twenty-two categorical levels due to personal health information privacy restrictions, and further collapsed into three categories to parallel major age progressions: 18–39, 40–64, 65–105 years. Access to primary healthcare is defined as having access to any of a family physician, family health team, walk-in clinic or other primary care setting. Geographic location of the ED is classified as either urban or rural; considered urban when the community (defined by regional postal codes) are equal to or greater than 10,000 residents. The presenting complaint refers to the primary reason for seeking emergency medical care, categorized using the Canadian Emergency Department Information System (CEDIS). Referral source indicates the entity that advised the patient to seek emergency care in the ED.
Statistical analysis and machine learning algorithms
We reported descriptive statistics using measures of central tendency and frequency. We trained four independent ML classification algorithms in this study: logistic regression (LR), lasso logistic regression (Lasso LR), random forest (RF) and gradient boosted trees (GBT). We selected algorithms based on previous literature to predict patient ED visit outcomes in consultation with health care machine learning experts [17–19]. We used 10-fold cross-validation to calculate predictive performance for the models using the same series of predictors. Cross-validation is a reliable measure of model performance and an approach commonly utilized in machine learning health care prediction research [20]. We used a dichotomous threshold of 50% likelihood for our predictions. We managed and analyzed the data in R software, V.3.6 and built the machine learning algorithms using ‘mlr’ and ‘mlr3’ packages [21, 22]. The R script that defined the study cohort is shown in S1 File; the R script that computed the models could not be extracted due to ICES privacy restrictions.
Hyperparameter tuning
We utilized hyperparameter optimization in RF and GBT algorithms to maximize the average 10-fold cross validation area under the receiver operating curve (AUC) through tuning prior to computation of the final models [23]. For the RF algorithm, we used an exhaustive grid search for eligible tuning parameters and further trained models by optimizing the minimum node size and number of splitting variables. For GBT, we employed a broad random search of parameter space, followed by a focused random search within the best performing areas of the broad search. The hyperparameters trained in the GBT algorithm consisted of maximum number of iterations, maximum tree depth, shrinkage rate, minimum sum of instance weights required to stop splitting and gamma. A comprehensive depiction of all hyperparameter tuning used for the RF and GBT algorithms are shown in the S2 Table.
Performance measurement and predictor importance
We measured the performance of all algorithms using the AUC that resulted from 10-fold cross-validation with corresponding 95% confidence intervals (CIs). We assessed calibration accuracy using the Brier Scaled score, a measure of probabilistic predictions on a scale of 0 (perfect accuracy) to 1 (perfect inaccuracy) [24]. Additionally, we reported the sensitivity, specificity, positive predictive value, and negative predicted value for each algorithm. To determine the importance of each predictor, we used permutation methods to calculate the average decrease in AUC in the absence of a predictor. The predictors with the highest variable importance corresponded to the largest declines in AUC. To construct a comprehensive list of variable importance across all algorithms, we computed the mean reduction in AUC for all predictors by averaging their AUC reduction from all models. We ranked the mean AUC reductions to produce a top 10 most important predictors list.
Results
Descriptive results
Fig 1 shows exclusions used to construct the study cohort. A total of 584,925 ED patients transported by paramedics were included, yielding 261,072 (44.6%) patients admitted to hospital and 323,853 (55.4%) patients discharged directly from ED. Overall, our cohort represented 90.4% of all adult emergently triaged paramedic transported ED visits in Ontario.
Table 1 shows a summary of the clinical profile of all patients in our study cohort. The majority of patients were older adults aged 65 years or greater (296,111; 50.6%), male (296,902; 50.8%), referred to ED by self of family (490,242; 83.3%), previously diagnosed with hypertension (336,174; 57.5%) and had their ED visit occur in an urban region of Ontario (535,270; 91.5%). The most common presenting complaints on ED arrival were cardiovascular (178,524; 30.5%), neurologic (99,119; 16.9%) and gastrointestinal (61,104; 10.4%). Missing data was negligible and non-informative, presenting only in the geographic location of the ED (0.5%).
Hyperparameter tuning and comparison of predictive model performance
Table 2 displays the final predictive model performance of all machine learning algorithms used in this study after benchmarking. Overall, each algorithm performed similarly with an AUC ranging from 0.77–0.78. The GBT model computed the best overall performance in classifying paramedic transported emergently triaged visits that were admitted to hospital, achieving an AUC of 0.78 (95% CI 0.78–0.78). The LR and lasso LR models computed the most accurate predictions with a Brier Scaled score of 0.22. S2 Table shows the details of hyperparameter tuning ranges and optimal values used to produce the final RF and GBT algorithm outputs. Calibration plots of predicted probabilities were assessed for quality of algorithm predictions; all algorithms received excellent calibration with slopes ranging from 0.997 to 1.086, shown in S1–S4 Figs.
Variable importance
Fig 2 shows the mean variable importance score for the top 10 predictors of hospital admission from the ED following paramedic transport. The top 10 most important predictors were (1) age 65 to 105 years, (2) referred to ED from a residential care facility, (3) presenting complaint is respiratory, (4) receiving home care, (5) presenting complaint is general or minor, (6) aged 40 to 64 years, (7) presenting complaint is cardiovascular, (8) presenting complaint is gastrointestinal, (9) presenting complaint is mental health and (10) comorbidity of hypertension. S3 Table shows the relative importance of each predictor across all machine learning algorithms, presented as the reduction in AUC in absence of the predictor. S4 Table shows the variable-relevance ranking of the top 10 predictors for each ML algorithm. Aged 65 to 105 years was consistently the most important variable ranked amongst all ML algorithms. Referral from residential care was ranked as either the second or third most important predictor, and presenting with a respiratory complaint was ranked between second to fourth position. Receiving home care’s rank was spread from second to sixth in the algorithms. All remaining top 10 predictors were distributed variably in each algorithm, with less uniformity in rankings.
Discussion
We predicted hospital admission using readily available clinical characteristics (age, gender, presenting complaint, comorbidities) and non-clinical characteristics (geographic location of ED, referral source, receiving home care services) known to paramedics prior to ED arrival. The acceptable algorithm results demonstrated that prehospital characteristics have predictive validity to forecast ED visit dispositions of emergent acuity. In the absence of additional important clinical and non-clinical characteristics not captured in administrative data, we computed acceptable AUC outputs of 0.77–0.78 and probabilistic accuracy scores of 0.22–0.24 across all algorithms. All machine learning algorithms performed similarly in all measures of statistical precision (sensitivity, specificity).
Our study identified older age, living in a residential facility, presenting to ED with a respiratory complaint, and receiving home care as the most informative predictors of hospital admission. These findings align with previous literature that examined predictors of admission for unscheduled ED visits [25–28]. A plethora of research has examined patient characteristics predictive of admission in the older adult cohorts, yielding consistent results with our predictors [29–32]. Receiving home care was included less frequently as an admission predictor in the literature, suggesting a potential underreporting of this characteristics influence on ED visit dispositions. Our results highlight that later-stages of life and requiring support care (residential or institutional) are the driving forces of ED admission from prehospital patient characteristics.
Several characteristics were non-informative in predicting hospital admissions, such as ED location, presenting complaints related to environmental reasons or the skin, comorbidities of bowel disease or asthma, and referral from ambulatory care services or private practices. These non-informative predictors align somewhat with prior work [4, 33]. Specifically, comorbidities of asthma and diabetes have been reported as risk markers for hospital admission, although not consistently in the literature [34, 35]. In our study, these two predictors did not contribute significantly to the prediction of admission. This result may be attributed to the inclusion of only paramedic transported visits, as paramedics frequently treat asthma or diabetes conditions prior to ED arrival in the prehospital field, thereby decreasing the likelihood of subsequent admission.
Considering the similarities in discriminative and calibration accuracy between the LR model and more advanced ML algorithms, it is worth noting that this model may be best suited for pragmatic implementation and decision-making at the clinical and policy levels. Interpretation of relative odds and confidence intervals are standard in medical education, whereas ML interpretation is not. The simplicity of the LR supports an easier comprehension of the study findings by medical and health policy professionals, who are most likely to influence policy changes.
Our study holds relevance for the scientific community as it employs ML algorithms to predict ED outcomes of paramedic transported patients using only characteristics known before hospital arrival. Given these predictors in the study are easily accessed by paramedics in the prehospital setting, policymakers could consider modifications to patient distribution legislation and clinical decision rules to transport certain patients to further distanced EDs. Our study’s most important predictors could identify ineligible patients for transport to a further distanced hospital due to their increased likelihood for admission.
One limitation of our study was the absence of additional clinical and non-clinical characteristics available to paramedics, which could improve prediction performance. Our data was limited to translating ED data collected at triage to what would be known in the prehospital field. This pragmatic approach was necessary due to the challenges associated with accessing paramedic administrative data resources, which are held independently by each of the 52 paramedic service independently and do not regularly permit research access [8]. Some degree of heterogeneity may exist between prehospital and ED clinicians categorizing the primary complaint, although we hypothesize that this is minimal given the broad categorizations of the conditions. Additionally, we used ED disposition as the outcome of our primary objective as a proxy for poor clinical condition and the need for further treatment due to data availability. Alternative strategies, such as emergency procedures or administration of emergency medicine, could provide more precise indicators of poor clinical condition, but are not routinely available for research.
With advanced knowledge of which patient characteristics are most predictive of admission, a paramedic distribution protocol could be developed to stratify emergent patients into a two-tier system. The first group could comprise patients who must be transported to the closest ED, as they have characteristics associated with a high predicted probability of admission. The second group could consist of patients eligible for transport to the most appropriate hospital, even if further distanced, as their characteristics are not associated with an increased predicted probability of admission. Future research is needed to determine the inclusion criteria of these groups within emergently triaged patients. Nevertheless, our study serves as a first step towards investigating the concept of hospital redistribution for high acuity patients. Employing threshold tuning to identify an ideal cut-point in the predicted probability with an acceptable sensitivity could be beneficial in defining these groups. Stratification of patients assigned an emergent triage score into groups could optimize patient distribution, improve ED congestion and plausibly improve patient-important outcomes. However, integration of our results should not be readily translated into the clinical setting without external validation or examining temporal validation data.
An overarching goal of this study was to determine if patient characteristics known prehospital could predict ED visit dispositions prior to arrival. Our study demonstrated that prehospital characteristics have predictive utility for forthcoming ED visits. We established that ML has ample potential for integration into prehospital research and could be an important methodology for optimizing paramedic clinical practices and logistical effectiveness. Including paramedic medical reports, linked to ED administrative data resources, would undoubtedly increase the probabilistic accuracy of predictive algorithms, and contribute variables that encompass a more complete prehospital patient presentation than ED resources alone.
Future research should leverage ML to analyze the utility of patient characteristics in predicting ED visit dispositions in low acuity transports. Ontario paramedics could be granted the capacity to transport low acuity patients to non-ED community-based care alternatives to ease ED patient congestion, though identifying and classifying of suitable patients for redirection from an ED poses challenging in the community setting [36]. Future research on low acuity transports should utilize ML to investigate which patient characteristics are most predictive of ED visits where patients are discharged and did not require any the emergency medicine intervention [36]. Knowledge of these predictors would be beneficial in supporting the development of paramedic clinical guidelines to identify patients potentially suitable for redirection, considering the limited clinical information available to paramedics. Additionally, future research could replicate our results using emergency medicine procedures as the primary outcome to provide a plausibly more precise depiction of patients with poor clinical conditions in the ED. Lastly, external validity research using multijurisdictional data outside of Ontario could further examine our results for clinical implementation purposes.
Conclusion
Hospital admission was predicted with acceptable accuracy using patient characteristics of paramedic transported patients with an emergent acuity. All ML algorithm performed similarly with an acceptable AUC and probabilistic accuracy score. Our results could support the exploration of policy modification to consider the transport of certain emergently triaged patients to a further distanced EDs when appropriate. Our study highlights the abundant predictive utility of prehospital patient characteristics in ML, emphasizing its potential in future paramedic and prehospital research.
Supporting information
S1 File. R script.
R script to define the study cohort from raw data.
https://doi.org/10.1371/journal.pone.0289429.s001
(DOCX)
S2 Fig. Calibration of lasso logistic regression.
https://doi.org/10.1371/journal.pone.0289429.s003
(TIF)
S4 Fig. Calibration of gradient boosted trees.
https://doi.org/10.1371/journal.pone.0289429.s005
(TIF)
S1 Table. STROBE statement.
Checklist of items that should be included in reports of cohort studies.
https://doi.org/10.1371/journal.pone.0289429.s006
(DOCX)
S3 Table. Variable importance.
Results of variable importance predictors for each machine learning algorithm.
https://doi.org/10.1371/journal.pone.0289429.s008
(DOCX)
S4 Table. Variable-relevance ranking.
Variable-relevance ranking of top 10 predictors for each machine learning algorithm.
https://doi.org/10.1371/journal.pone.0289429.s009
(DOCX)
References
- 1.
Emergency Health Services Branch, Ministry of Health and Long-Term Care. Prehospital Canadian Triage & Acuity Scale: Prehospital CTAS Paramedic Guide [Internet]. Government of Ontario; 2016. https://www.lhsc.on.ca/media/2904/download
- 2. Bullard MJ, Unger B, Spence J, Grafstein E, the CTAS National Working Group. Revisions to the Canadian Emergency Department Triage and Acuity Scale (CTAS) adult guidelines. CJEM [Internet]. 2008 Mar [cited 2020 Dec 5];10(02):136–42. Available from: https://www.cambridge.org/core/product/identifier/S1481803500009854/type/journal_article
- 3. Strum RP, Tavares W, Worster A, Griffith LE, Costa AP. Identifying patient characteristics associated with potentially redirectable paramedic transported emergency department visits in Ontario, Canada: a population-based cohort study. BMJ Open [Internet]. 2021 Dec 1 [cited 2021 Dec 31];11(12):e054625. Available from: https://bmjopen.bmj.com/content/11/12/e054625 pmid:35225823
- 4. Strum RP, Mowbray FI, Worster A, Tavares W, Leyenaar MS, Correia RH, et al. Examining the association between paramedic transport to the emergency department and hospital admission: a population-based cohort study. BMC Emerg Med [Internet]. 2021 Dec [cited 2021 Nov 16];21(1):1–9. Available from: https://bmcemergmed.biomedcentral.com/articles/10.1186/s12873-021-00507-2
- 5. Strum RP, Drennan IR, Mowbray FI, Mondoux S, Worster A, Babe G, et al. Increased demand for paramedic transports to the emergency department in Ontario, Canada: a population-level descriptive study from 2010 to 2019. Can J Emerg Med [Internet]. 2022 Aug 19 [cited 2022 Aug 24]; Available from: pmid:35984572
- 6. Shen Y, Lee LH. Improving the wait time to consultation at the emergency department. BMJ Open Qual [Internet]. 2018 Jan 3 [cited 2022 Mar 22];7(1):e000131. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5759711/ pmid:29333496
- 7. Hong WS, Haimovich AD, Taylor RA. Predicting hospital admission at emergency department triage using machine learning. PLOS ONE [Internet]. 2018 Jul 20 [cited 2022 Mar 13];13(7):e0201016. Available from: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0201016 pmid:30028888
- 8. Strum RP, Costa AP. Implementing machine learning in paramedicine. CMAJ [Internet]. 2022 Jan 17 [cited 2022 Mar 13];194(2):E54–E54. Available from: https://www.cmaj.ca/content/194/2/E54 pmid:35039390
- 9. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ [Internet]. 2007 Oct 18 [cited 2020 Dec 4];335(7624):806–8. Available from: https://www.bmj.com/content/335/7624/806 pmid:17947786
- 10. Ferron R, Agarwal G, Cooper R, Munkley D. The effect of COVID-19 on emergency medical service call volumes and patient acuity: a cross-sectional study in Niagara, Ontario. BMC Emerg Med [Internet]. 2021 Mar 29 [cited 2022 Jan 1];21(1):39. Available from: pmid:33781229
- 11. Li G, Lau JT, McCarthy ML, Schull MJ, Vermeulen M, Kelen GD. Emergency Department Utilization in the United States and Ontario, Canada. Acad Emerg Med [Internet]. 2007 [cited 2020 Dec 4];14(6):582–4. Available from: https://onlinelibrary.wiley.com/doi/abs/10.1197/j.aem.2007.02.030 pmid:17470903
- 12.
National Ambulatory Care Reporting System metadata (NACRS) | CIHI [Internet]. [cited 2020 Nov 14]. https://www.cihi.ca/en/national-ambulatory-care-reporting-system-metadata-nacrs
- 13.
Government of Ontario. Personal Health Information Protection Act, 2004, S.O. 2004, c. 3, Sched. A [Internet]. Jul 24, 2014. https://www.ontario.ca/laws/statute/04p03#BK63
- 14.
Government of Ontario. Ontario Regulation 329/04: General, Personal Health Information Protection Act [Internet]. Sect. 18. Prescribed entitites for the purposes of s. 45 (1) of the Act 02, 2021. https://www.ontario.ca/laws/regulation/040329#BK19
- 15. Calder LA, Arnason T, Vaillancourt C, Perry JJ, Stiell IG, Forster AJ. How do emergency physicians make discharge decisions? Emerg Med J [Internet]. 2015 Jan 1 [cited 2022 Jan 5];32(1):9–14. Available from: https://emj.bmj.com/content/32/1/9 pmid:24045050
- 16.
Johnson MK and K. Feature Engineering and Selection: A Practical Approach for Predictive Models [Internet]. [cited 2022 Jan 6]. http://www.feat.engineering/
- 17. Jones A, Costa AP, Pesevski A, McNicholas PD. Predicting hospital and emergency department utilization among community-dwelling older adults: Statistical and machine learning approaches. PLOS ONE [Internet]. 2018 Nov 1 [cited 2022 Mar 15];13(11):e0206662. Available from: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0206662 pmid:30383850
- 18. Graham B, Bond R, Quinn M, Mulvenna M. Using Data Mining to Predict Hospital Admissions From the Emergency Department. IEEE Access. 2018;6:10458–69.
- 19. Raita Y, Goto T, Faridi MK, Brown DFM, Camargo CA, Hasegawa K. Emergency department triage prediction of clinical outcomes using machine learning models. Crit Care [Internet]. 2019 Feb 22 [cited 2022 Mar 15];23(1):64. Available from: pmid:30795786
- 20. Doupe P, Faghmous J, Basu S. Machine Learning for Health Services Researchers. Value Health [Internet]. 2019 Jul 1 [cited 2022 Mar 15];22(7):808–15. Available from: https://www.sciencedirect.com/science/article/pii/S1098301519301469 pmid:31277828
- 21. Binder M, Pfisterer F, Lang M, Schneider L, Kotthoff L, Bischl B. mlr3pipelines–Flexible Machine Learning Pipelines in R. J Mach Learn Res. 2021 Jun;22:1–7.
- 22. Lang M, Binder M, Richter J, Schratz P, Pfisterer F, Coors S, et al. mlr3: A modern object-oriented machine learning framework in R. J Open Source Softw [Internet]. 2019 Dec 11 [cited 2022 Jan 11];4(44):1903. Available from: https://joss.theoj.org/papers/10.21105/joss.01903
- 23.
Abbott D. Applied Predictive Analytics: Principles and Techniques for the Professional Data Analyst. John Wiley & Sons; 2014. 456 p.
- 24. Rufibach K. Use of Brier score to assess binary predictions. J Clin Epidemiol [Internet]. 2010 Aug 1 [cited 2022 Jan 6];63(8):938–9. Available from: https://www.jclinepi.com/article/S0895-4356(09)00363-1/fulltext pmid:20189763
- 25. Hayward J, Hagtvedt R, Ma W, Gauri A, Vester M, Holroyd BR. Predictors of Admission in Adult Unscheduled Return Visits to the Emergency Department. West J Emerg Med. 2018 Nov;19(6):912–8. pmid:30429921
- 26. Lin D, Worster A. Predictors of admission to hospital of patients triaged as nonurgent using the Canadian Triage and Acuity Scale. CJEM [Internet]. 2013 Nov [cited 2020 Dec 11];15(06):353–8. Available from: https://www.cambridge.org/core/product/identifier/S148180350000244X/type/journal_article pmid:24176459
- 27. Mowbray FI, Aryal K, Mercier E, Heckman G, Costa AP. Older Emergency Department Patients: Does Baseline Care Status Matter? Can Geriatr J CGJ. 2020 Dec;23(4):289–96. pmid:33282049
- 28. Mowbray F, Zargoush M, Jones A, de Wit K, Costa A. Predicting hospital admission for older emergency department patients: Insights from machine learning. Int J Med Inf [Internet]. 2020 Aug 1 [cited 2022 May 31];140:104163. Available from: https://www.sciencedirect.com/science/article/pii/S1386505619314510 pmid:32474393
- 29. Marcusson J, Nord M, Dong HJ, Lyth J. Clinically useful prediction of hospital admissions in an older population. BMC Geriatr [Internet]. 2020 Dec [cited 2020 Dec 11];20(1):95. Available from: https://bmcgeriatr.biomedcentral.com/articles/10.1186/s12877-020-1475-6 pmid:32143637
- 30. Gabayan GZ, Sarkisian CA, Liang LJ, Sun BC. Predictors of admission after emergency department discharge in older adults. J Am Geriatr Soc. 2015 Jan;63(1):39–45. pmid:25537073
- 31. Vetrano DL, Landi F, De Buyser SL, Carfì A, Zuccalà G, Petrovic M, et al. Predictors of length of hospital stay among older adults admitted to acute care wards: a multicentre observational study. Eur J Intern Med [Internet]. 2014 Jan 1 [cited 2022 Mar 16];25(1):56–62. Available from: https://www.sciencedirect.com/science/article/pii/S0953620513009187 pmid:24054859
- 32. Aliyu MH, Adediran AS, Obisesan TO. Predictors of hospital admissions in the elderly: analysis of data from the Longitudinal Study on Aging. J Natl Med Assoc [Internet]. 2003 Dec [cited 2022 Mar 16];95(12):1158–67. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2594833/ pmid:14717472
- 33. Rahimian F, Salimi-Khorshidi G, Payberah AH, Tran J, Solares RA, Raimondi F, et al. Predicting the risk of emergency admission with machine learning: Development and validation using linked electronic health records. PLOS Med [Internet]. 2018 Nov 20 [cited 2022 Mar 16];15(11):e1002695. Available from: https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1002695 pmid:30458006
- 34. Donnan PT, Dorward DWT, Mutch B, Morris AD. Development and Validation of a Model for Predicting Emergency Admissions Over the Next Year (PEONY): A UK Historical Cohort Study. Arch Intern Med [Internet]. 2008 Jul 14 [cited 2022 Mar 16];168(13):1416–22. Available from: pmid:18625922
- 35. Ownby DR, Abarzua J, Anderson JA. Attempting to predict hospital admission in acute asthma. Am J Dis Child 1960. 1984 Nov;138(11):1062–6. pmid:6496423
- 36. Strum RP, Tavares W, Worster A, Griffith LE, Costa AP. Emergency department interventions that could be conducted in subacute care settings for patients with nonemergent conditions transported by paramedics: a modified Delphi study. Can Med Assoc Open Access J [Internet]. 2022 Jan 1 [cited 2022 Jan 11];10(1):E1–7. Available from: https://www.cmajopen.ca/content/10/1/E1 pmid:35017171